| Server IP : 127.0.0.1 / Your IP : 216.73.216.109 Web Server : Apache/2.4.54 (Win64) OpenSSL/1.1.1q PHP/8.1.10 System : Windows NT DESKTOP-E5T4RUN 10.0 build 19045 (Windows 10) AMD64 User : SERVERWEB ( 0) PHP Version : 8.1.10 Disable Function : NONE MySQL : OFF | cURL : ON | WGET : OFF | Perl : OFF | Python : OFF | Sudo : OFF | Pkexec : OFF Directory : C:/cygwin64/lib/python3.9/site-packages/sphinx/builders/__pycache__/ |
Upload File : |
a
�abq � @ s� d Z ddlZddlZddlZddlZddlZddlmZmZ ddlm Z ddl
m Z ddl m
Z
ddlmZmZ ddlmZ dd lmZmZmZmZmZmZmZmZm Z m Z m Z dd
l m Z m!Z!m"Z" dd l#m$Z$ dd l%m&Z& dd
l'm(Z( ddl)m*Z*m+Z+m,Z, ddl-m.Z. ddl/m0Z0 ddl1m2Z2 ddl3m4Z4 ddl5m6Z6 ddl7m8Z8 ddl9m:Z: ddl;m<Z<m=Z=m'Z' ddl>m?Z?m@Z@mAZAmBZBmCZC ddlDmEZE e=�FeG�ZHe�Id�ZJG dd� de�ZKG d d � d e�ZLG d d � d e�ZMG d d!� d!e�ZNe eLe eOePePeQf f ZRd"d#iZSdZTd$ZUd%ZVe&eQd&�d'd(�ZWG d)d*� d*e �ZXe'j'j(ePeYd+�d,d-�ZZG d.d/� d/e0�Z[G d0d1� d1�Z\G d2d3� d3e�Z]G d4d5� d5e:�Z^e.ePeeP d6�d7d8�Z_e.e2dd9�d:d;�Z`e.eePef d<�d=d>�ZadS )?z�
sphinx.builders.linkcheck
~~~~~~~~~~~~~~~~~~~~~~~~~
The CheckExternalLinksBuilder class.
:copyright: Copyright 2007-2022 by the Sphinx team, see AUTHORS.
:license: BSD, see LICENSE for details.
� N)�datetime�timezone)�parsedate_to_datetime)�
HTMLParser)�path)�
PriorityQueue�Queue)�Thread) �Any�Dict� Generator�List�
NamedTuple�Optional�Pattern�Set�Tuple�Union�cast)�unquote�urlparse�
urlunparse)�nodes)�Element)�Response)�ConnectionError� HTTPError�TooManyRedirects)�Sphinx)� DummyBuilder)�Config)�RemovedInSphinx50Warning)�BuildEnvironment)�__)�SphinxPostTransform)�
encode_uri�logging�requests)�darkgray� darkgreen�purple�red� turquoise)�
get_node_linez ([a-z]+:)?//c @ s* e Zd ZU eed< eed< ee ed< dS )� Hyperlink�uri�docname�linenoN)�__name__�
__module__� __qualname__�str�__annotations__r �int� r8 r8 �=/usr/lib/python3.9/site-packages/sphinx/builders/linkcheck.pyr. / s
r. c @ s" e Zd ZU eed< ee ed< dS )� CheckRequest�
next_check� hyperlinkN)r2 r3 r4 �floatr6 r r. r8 r8 r8 r9 r: 5 s
r: c @ s> e Zd ZU eed< eed< eed< eed< eed< eed< dS )� CheckResultr/ r0 r1 �status�message�codeN)r2 r3 r4 r5 r6 r7 r8 r8 r8 r9 r> : s
r> c @ s e Zd ZU eed< eed< dS )� RateLimit�delayr; N)r2 r3 r4 r= r6 r8 r8 r8 r9 rB C s
rB ZAcceptz/text/html,application/xhtml+xml;q=0.9,*/*;q=0.8� g N@)�node�returnc C s t jdtdd� t| �pdS )z�
PriorityQueue items must be comparable. The line number is part of the
tuple used by the PriorityQueue, keep an homogeneous type for comparison.
z node_line_or_0() is deprecated.� ��
stacklevelr )�warnings�warnr! r- )rE r8 r8 r9 �node_line_or_0S s �rL c s: e Zd ZdZedd�� fdd� Zeedd�dd�Z� ZS ) �AnchorCheckParserz9Specialized HTML parser that looks for a specific anchor.N)�
search_anchorrF c s t � �� || _d| _d S �NF)�super�__init__rN �found)�selfrN �� __class__r8 r9 rQ ` s
zAnchorCheckParser.__init__)�tag�attrsrF c C s. |D ]$\}}|dv r|| j krd| _ q*qd S )N)�id�nameT)rN rR )rS rV rW �key�valuer8 r8 r9 �handle_starttagf s z!AnchorCheckParser.handle_starttag) r2 r3 r4 �__doc__r5 rQ r
r\ �
__classcell__r8 r8 rT r9 rM ] s rM )�response�anchorrF c C sP t |�}| jddd�D ]*}t|t�r,|�� }|�|� |jr qBq|�� |jS )z�Reads HTML data from a response object `response` searching for `anchor`.
Returns True if anchor was found, False otherwise.
i T)Z
chunk_sizeZdecode_unicode)rM Z iter_content�
isinstance�bytes�decodeZfeedrR �close)r_ r` �parser�chunkr8 r8 r9 � check_anchorm s
rg c @ sb e Zd ZdZdZed�Zdd�dd�Zee e
d�dd ��Z ee e e
e
f d�d
d ��Zee e
d�d d
��Zeee d�dd��Zeeeef d�dd��Zeeee eef f d�dd��Zdd�dd�Zeee d�dd�Zee d�dd�Z ee e d�dd �Z ee d�d d �Z e!dd �d d!�Z"eeeeedd"�d#d$�Z#e$dd%�d&d'�Z%dd�d(d)�Z&dS )*�CheckExternalLinksBuilderz+
Checks for broken external links.
� linkcheckzCLook for any errors in the above output or in %(outdir)s/output.txtN�rF c C s8 i | _ t� | _i | _i | _t�d� t� | _t � | _
d S )Ng @) �
hyperlinks�set�_good�_broken� _redirected�socketZsetdefaulttimeoutr �_wqueuer �_rqueue�rS r8 r8 r9 �init� s
z CheckExternalLinksBuilder.initc C s. t jd| jjdf tdd� dd� | jjD �S )N�%s.%s is deprecated.�anchors_ignorerG rH c S s g | ]}t �|��qS r8 ��re�compile��.0�xr8 r8 r9 �
<listcomp>� � z<CheckExternalLinksBuilder.anchors_ignore.<locals>.<listcomp>)rJ rK rU r2 r! �config�linkcheck_anchors_ignorers r8 r8 r9 rv � s �z(CheckExternalLinksBuilder.anchors_ignorec C s. t jd| jjdf tdd� dd� | jjD �S )Nru �authrG rH c S s g | ]\}}t �|�|f�qS r8 rw �r{ �pattern� auth_infor8 r8 r9 r} � r~ z2CheckExternalLinksBuilder.auth.<locals>.<listcomp>)rJ rK rU r2 r! r �linkcheck_authrs r8 r8 r9 r� � s ��z CheckExternalLinksBuilder.authc C s. t jd| jjdf tdd� dd� | jjD �S )Nru � to_ignorerG rH c S s g | ]}t �|��qS r8 rw rz r8 r8 r9 r} � r~ z7CheckExternalLinksBuilder.to_ignore.<locals>.<listcomp>)rJ rK rU r2 r! r �linkcheck_ignorers r8 r8 r9 r� � s �z#CheckExternalLinksBuilder.to_ignorec C s" t jd| jjdf tdd� | jS )Nru �goodrG rH )rJ rK rU r2 r! rm rs r8 r8 r9 r� � s �z CheckExternalLinksBuilder.goodc C s" t jd| jjdf tdd� | jS )Nru �brokenrG rH )rJ rK rU r2 r! rn rs r8 r8 r9 r� � s �z CheckExternalLinksBuilder.brokenc C s" t jd| jjdf tdd� | jS )Nru �
redirectedrG rH )rJ rK rU r2 r! ro rs r8 r8 r9 r� � s �z$CheckExternalLinksBuilder.redirectedc C s t jd| jjdf tdd� d S )Nru � check_threadrG rH �rJ rK rU r2 r! rs r8 r8 r9 r� � s
�z&CheckExternalLinksBuilder.check_thread�r_ rF c C s: t jd| jjdf tdd� t| j| jd d i �}|�|�S )Nru �
limit_raterG rH ) rJ rK rU r2 r! � HyperlinkAvailabilityCheckWorker�envr r� )rS r_ Zworkerr8 r8 r9 r� � s �
�z$CheckExternalLinksBuilder.limit_ratec C s" t jd| jjdf tdd� | jS )Nru �rqueuerG rH )rJ rK rU r2 r! rr �rS r_ r8 r8 r9 r� � s �z CheckExternalLinksBuilder.rqueuec C s t jd| jjdf tdd� g S )Nru �workersrG rH r� r� r8 r8 r9 r� � s �z!CheckExternalLinksBuilder.workersc C s" t jd| jjdf tdd� | jS )Nru �wqueuerG rH )rJ rK rU r2 r! rq r� r8 r8 r9 r� � s �z CheckExternalLinksBuilder.wqueue)�resultrF c C s� | j �|jd �}t||j|j|j|j|jd�}| � |� |jdkrFd S |jdkr^|jdkr^d S |jrzt
j d|j|jdd� |jdkr�|jr�t
� t d �|j d
|j � nt
� t d �|j � �n�|jd kr�t
� t d �|j � | �
d |j||j|j� �n�|jdk�r&t
� td
�|j |j � �n�|jdk�r�| jj�sF| jj�rjt
jtd�|j|j|j|jfd� n"t
� td�|j td|j � � | �
d|j||j|jd
|j � �n |jdk�r�z2dtfdtfdtfdtfdtfd�|j \}}W n t�y dt }}Y n0 ||d< | jj�rRt
jd|j d | d |j |j|jfd� n*t
� |d�|j |d| d |j � � | �
d | |j||j|jd |j � ntd |j ��d S ) N)�filenamer1 r? rA r/ �info� unchecked�working�oldz(%16s: line %4d) T)Znonl�ignoredz
-ignored- z: �localz
-local- z
ok r� zbroken link: %s (%s))�locationz
broken z - r� Z permanentlyz
with Foundzwith See OtherZ temporarily)i- i. i/ i3 i4 zwith unknown code�textz
redirect z to z redirected zUnknown status %s.)r� �doc2pathr0 �dictr1 r? rA r/ r@ �write_linkstat�loggerr� r( � write_entryr) �app�quietZwarningiserror�warningr# r+ r* r, �KeyErrorr �linkcheck_allowed_redirects�
ValueError)rS r� r� Zlinkstatr� Zcolorr8 r8 r9 �process_result� sp �
�"� ��
�
���z(CheckExternalLinksBuilder.process_result)�whatr0 r� �liner/ rF c C s | j �d||||f � d S )Nz%s:%s: [%s] %s
)� txt_outfile�write)rS r� r0 r� r� r/ r8 r8 r9 r� + s z%CheckExternalLinksBuilder.write_entry)�datarF c C s" | j �t�|�� | j �d� d S )N�
)� json_outfiler� �json�dumps)rS r� r8 r8 r9 r� / s z(CheckExternalLinksBuilder.write_linkstatc C s� t | j| j| �}t�d� tt�| jd�d��d| _ tt�| jd�d��.| _
|� | j �D ]}| �
|� qVW d � n1 sz0 Y W d � n1 s�0 Y | jr�d| j_d S )N� z
output.txt�wz output.jsonrD )� HyperlinkAvailabilityCheckerr� r r� r� �openr �joinZoutdirr� r� �checkrk r� rn r� Z
statuscode)rS Zcheckerr� r8 r8 r9 �finish3 s
�Hz CheckExternalLinksBuilder.finish)'r2 r3 r4 r] rY r# �epilogrt �propertyr
r rv r r
r� r� r r5 r� r r� r7 r� r� r r r= r� r r� r r� r� r> r� r� r� r� r� r8 r8 r8 r9 rh s6
7� rh c @ sp e Zd Zdeeedd�dd�Zdd�dd�Zdd�dd �Ze e
e f e e
ddf d
�d d �Ze
ed
�dd�ZdS )r� N)r� r �builderrF c C s\ || _ || _|| _i | _g | _dd� | jjD �| _|rH|j| _|j | _
nt � | _t � | _
d S )Nc S s g | ]}t �|��qS r8 rw rz r8 r8 r9 r} L r~ z9HyperlinkAvailabilityChecker.__init__.<locals>.<listcomp>)
r� r r� � rate_limitsr� r� r� rr r� rq r� r r )rS r� r r� r8 r8 r9 rQ A s
z%HyperlinkAvailabilityChecker.__init__rj c C sH t | jj�D ]6}t| j| j| j| j| j| j�}|� � | j
� |� q d S �N) �ranger �linkcheck_workersr� r� r� r� r� r� �startr� �append)rS Z_i�threadr8 r8 r9 �invoke_threadsU s
�z+HyperlinkAvailabilityChecker.invoke_threadsc C s. | j �� | jD ]}| j �ttd �d� qd S rO )r� r� r� �putr: �CHECK_IMMEDIATELY)rS Z_workerr8 r8 r9 �shutdown_threads] s
z-HyperlinkAvailabilityChecker.shutdown_threads)rk rF c c s� | � � d}|�� D ]H}| �|j�r@t|j|j|jddd�V q| j�t t
|�d� |d7 }qd}||k r�| j � � V |d7 }qb| �
� d S )Nr r� r� FrD )r� �values�is_ignored_urir/ r> r0 r1 r� r� r: r� r� �getr� )rS rk Z total_linksr< Zdoner8 r8 r9 r� b s �
z"HyperlinkAvailabilityChecker.check)r/ rF c s t � fdd�| jD ��S )Nc 3 s | ]}|� � �V qd S r� )�match)r{ �pat�r/ r8 r9 � <genexpr>v r~ z>HyperlinkAvailabilityChecker.is_ignored_uri.<locals>.<genexpr>)�anyr� �rS r/ r8 r� r9 r� u s z+HyperlinkAvailabilityChecker.is_ignored_uri)N)r2 r3 r4 r" r rh rQ r� r� r r5 r. r r> r� �boolr� r8 r8 r8 r9 r� @ s �� "r� c s^ e Zd ZdZd eeeeeee f e
dd�� fdd�
Z dd�dd�Z e
ee d �d
d �Z� ZS )
r� z;A worker class for checking the availability of hyperlinks.N)r� r r� r� r� r� rF c s� || _ || _|| _|| _|| _dd� | j jD �| _dd� | j jD �| _dd� | j j D �| _
|rx|j | _ |j | _ |j
| _
nt� | _ i | _ i | _
t� jdd� d S )Nc S s g | ]}t �|��qS r8 rw rz r8 r8 r9 r} � s �z=HyperlinkAvailabilityCheckWorker.__init__.<locals>.<listcomp>c S s g | ]}t �|��qS r8 rw )r{ �docr8 r8 r9 r} � s �c S s g | ]\}}t �|�|f�qS r8 rw r� r8 r8 r9 r} � r~ T)�daemon)r r� r� r� r� r� rv �linkcheck_exclude_documents�documents_excluder� r� rm rn ro rl rP rQ )rS r� r r� r� r� r� rT r8 r9 rQ | s, ���
z)HyperlinkAvailabilityCheckWorker.__init__rj c
s� i ��j jr�j j�d< td���fdd� �ttttf d�� �����fdd� �tttd��fdd � � tttttf d
����fd d � }�j�� }z$|\}��d u r�W �q��\�}}W n t y� |\}�}}Y n0 �d u r�q�t
��j }z�j | j
}W n t�y Y n0 |t�� k�rLt�t� �j�t|��d
� �j�� q�||�\}}} |dk�r�t�td�� td� � n�j�t�||||| �� �j�� q�d S )N�timeoutrj c sh t ��} d| j| jf d| j| jf �dg}|D ]2}|� jjv r0tt�}|�� jj| � | S q0i S )Nz%s://%sz%s://%s/�*)r �scheme�netlocr �linkcheck_request_headersr� �DEFAULT_REQUEST_HEADERS�update)�url�
candidates�u�headersr� r8 r9 �get_request_headers� s �
zAHyperlinkAvailabilityCheckWorker.run.<locals>.get_request_headersc
s: d�v r8�� dd�\} }�jD ]}|�|�r d } q@q n�} d }z| �d� W n tyh t| �} Y n0 �jD ]\}}|���rp q�qpd }�� �d< z�|r�jjr�t j
| fd�j|d����}|� � t |t
|��}|s�ttd�| ��n�z(t j| fd�j|d����}|� � W nl tttf�y� } zLt|t��rH|jjd k�rH� t j
| fd�j|d����}|� � W Y d }~n
d }~0 0 W �n t�yj } z�|jjd
k�r�W Y d }~d S |jjd k�r��|j�}|d u�r��j�t|��d � W Y d }~d
S dt|�dfW Y d }~S |jjdk�r<dt|�dfW Y d }~S dt|�dfW Y d }~S W Y d }~nhd }~0 t�y� } zdt|�dfW Y d }~S d }~0 0 t | �j } z �j | = W n t �y� Y n0 |j �!d�| �!d�k�r�dS |j }
|�r�|
d| 7 }
� | |
��rdS |j"�r,|j"d j} d|
| fS d|
dfS d S )N�#rD �asciir� T)�streamr r� zAnchor '%s' not found)Zallow_redirectsr r� i� i� )r� z - unauthorizedr F)� rate-limitedr� r r� r i� r� �/�r� r� r ���r� )#�splitrv r� �encode� UnicodeErrorr% r� r �linkcheck_anchorsr' r� Zraise_for_statusrg r � Exceptionr# �headr r r ra r_ Z status_coder� r� r� r: r5 r r� r� r� r� �rstrip�history) Zreq_urlr` Zrexr� r� r_ rR �errr; r� �new_urlrA )�allowed_redirectr� r< �kwargsrS r/ r8 r9 � check_uri� s�
��
��
��$
.$
z7HyperlinkAvailabilityCheckWorker.run.<locals>.check_uri)r� r� rF c s4 � j j�� D ]"\}}|�| �r |�|�r dS q dS )NTF)r r� �itemsr� )r� r� Zfrom_urlZto_urlrs r8 r9 r� s z>HyperlinkAvailabilityCheckWorker.run.<locals>.allowed_redirect)r0 rF c sp �j D ].}|�| �r| � d|j� d�}d|df S qt��dksL��d�rPdS ��d�s�t���rhdS t��j� | ��}t�
t� |���r�dS d �j �< d
S nP��j
v r�d S ��j v r�d �j � dfS ��jv r�d
�j� d �j� d fS t�jj�D ]"}� � \}}}|d k� r� �q � q�|dk�r8�j
��� n.|d k�rN|�j �< n|d
k�rf||f�j�< |||fS )Nz matched z! from linkcheck_exclude_documentsr� r )r� zmailto:ztel:)r� r� r )zhttp:zhttps:r� r� )r� r� r )r� r� r r� r� rD r� )r� r� r� �len�
startswith�uri_rer �dirnamer� r� �existsr� rn rm ro r� r �linkcheck_retries�add)r0 Z doc_matcherr� Zsrcdir�_r? rA )r� rS r/ r8 r9 r� s@
�
z3HyperlinkAvailabilityCheckWorker.run.<locals>.checkFr� z-rate limited- z | sleeping...)r �linkcheck_timeoutr r r5 r7 r� r� r� r� r r� r� r; r� �time�sleep�QUEUE_POLL_SECSr� r: Z task_doner� r� r( r� r> )
rS r� Z
check_requestr; r0 r1 r� r? r� rA r8 )r� r� r� r< r� rS r/ r9 �run� s@ &`"/
z$HyperlinkAvailabilityCheckWorker.runr� c
C s d }|j �d�}|r�z t|�}W nT tyt z t|�}W n ttfyP Y n 0 t�|�}|t�t j
� � � }Y n0 t � � | }t
|j�j}|d u r�| jj}z| j| }W n ty� t}Y n$0 |j} d| }||kr�| |k r�|}||kr�d S t � � | }t||�| j|< |S )Nz Retry-Afterg @)r� r� r= r� r � TypeErrorr Z timestampZnowr ZutcZ
total_secondsr r r� r� r � linkcheck_rate_limit_timeoutr� r� �
DEFAULT_DELAYrC rB )
rS r_ r; Z retry_afterrC Zuntilr� Z max_delayZ
rate_limitZlast_wait_timer8 r8 r9 r� j s:
z+HyperlinkAvailabilityCheckWorker.limit_rate)N)r2 r3 r4 r] r" r r r r5 rB rh rQ r r r r= r� r^ r8 r8 rT r9 r� y s � � Or� c @ s$ e Zd ZdZdZedd�dd�ZdS )�HyperlinkCollector)ri � N)r� rF c
K s� t t| jj�}|j}| j�tj�D ]T}d|vr0q"|d }| j� d|�}|rN|}t
|�}t || j j
|�}||vr"|||< q"| j�tj�D ]\} | d �d�}|r�d|v r�| j� d|�}|r�|}t
| �}t || j j
|�}||vr�|||< q�d S )NZrefuri�linkcheck-process-urir� �?z://)r rh r� r� rk Zdocument�findallr Z referenceZemit_firstresultr- r. r� r0 Zimager� )
rS r� r� rk Zrefnoder/ Znewurir1 Zuri_infoZimgnoder8 r8 r9 r � s.
zHyperlinkCollector.run)r2 r3 r4 ZbuildersZdefault_priorityr
r r8 r8 r8 r9 r � s r )r� r/ rF c C sH t |�}|jdkrD|jrD|j�d�}|sDd|j� �}t|j|d��S dS )z�Rewrite anchor name of the hyperlink to github.com
The hyperlink anchors in github.com are dynamically generated. This rewrites
them before checking and makes them comparable.
z
github.comz
user-content-)�fragmentN)r �hostnamer
r� r �_replace)r� r/ �parsedZprefixedr
r8 r8 r9 �rewrite_github_anchor� s r )r� r rF c C s� t | jj�� �D ]�\}}zlz t�|�| jjt�|�< W n< tjyr } z"t�t d�|j
|j � W Y d}~n
d}~0 0 W | jj� |� q| jj� |� 0 qdS )zFCompile patterns in linkcheck_allowed_redirects to the regexp objects.z=Failed to compile regex in linkcheck_allowed_redirects: %r %sN)
�listr r� r� rx ry �errorr� r� r# r� �msg�pop)r� r r� r� �excr8 r8 r9 �#compile_linkcheck_allowed_redirects� s
� r )r� rF c C s� | � t� | �t� | �dg d � | �dg d � | �di d � | �dg d � | �di d � | �ddd � | �dd d tg� | �d d
d � | �d d d � | �d
dgd � | �ddd � | �d� | jdtdd� dd d d�S )Nr� r� r� r� r� r� rD r r� � r� Tr� z^!r g �r@r
z
config-initedr )�priorityZbuiltin)�versionZparallel_read_safeZparallel_write_safe) Z add_builderrh Zadd_post_transformr Zadd_config_valuer7 Z add_eventZconnectr )r� r8 r8 r9 �setup� s&
�r )br] r� rx rp r rJ r r Z email.utilsr Z html.parserr �osr Zqueuer r � threadingr �typingr
r r r
r r r r r r r � urllib.parser r r Zdocutilsr Zdocutils.nodesr r' r Zrequests.exceptionsr r r Zsphinx.applicationr Zsphinx.builders.dummyr Z
sphinx.configr Zsphinx.deprecationr! Zsphinx.environmentr" Z
sphinx.localer# Z!sphinx.transforms.post_transformsr$ Z sphinx.utilr% r& Zsphinx.util.consoler( r) r* r+ r, Zsphinx.util.nodesr- Z getLoggerr2 r� ry r� r. r: r> rB r= r5 r7 ZCheckRequestTyper� r� r r rL rM r� rg rh r� r� r r r r r8 r8 r8 r9 �<module> sh
4
�
B9 $