| Server IP : 127.0.0.1 / Your IP : 216.73.216.109 Web Server : Apache/2.4.54 (Win64) OpenSSL/1.1.1q PHP/8.1.10 System : Windows NT DESKTOP-E5T4RUN 10.0 build 19045 (Windows 10) AMD64 User : SERVERWEB ( 0) PHP Version : 8.1.10 Disable Function : NONE MySQL : OFF | cURL : ON | WGET : OFF | Perl : OFF | Python : OFF | Sudo : OFF | Pkexec : OFF Directory : C:/cygwin64/lib/python3.9/site-packages/sphinx/search/__pycache__/ |
Upload File : |
a
�abE � @ sP U d Z ddlZddlZddlZddlmZ ddlmZ ddlm Z m
Z
m Z m Z m
Z
mZmZmZmZ ddlmZ ddlmZmZ ddlmZmZ dd lmZ dd
l m Z dd l m Z G d d
� d
�Z ddl!m"Z" e#ee# d�dd�Z$dde"ddddddddd d d d d d!d"�Z%e e#e
f e&d#< G d$d%� d%�Z'e'� Z(G d&d'� d'ej)�Z*G d(d)� d)�Z+dS )*z�
sphinx.search
~~~~~~~~~~~~~
Create a full-text search index for offline search.
:copyright: Copyright 2007-2022 by the Sphinx team, see AUTHORS.
:license: BSD, see LICENSE for details.
� N)�
import_module)�path) �IO�Any�Dict�Iterable�List�Optional�Set�Tuple�Type)�nodes)�Element�Node)�addnodes� package_dir)�BuildEnvironment)�
splitter_code)�jsdumpc @ s� e Zd ZU dZdZeed< dZeed< e� Z e
e ed< dZ eed< dZ eed< dZ
e�d �Zedd
�d d �Zedd
�d
d�Zeee d�dd�Zeed�dd�Zeed�dd�ZdS )�SearchLanguagea<
This class is the base class for search natural language preprocessors. If
you want to add support for a new language, you should override the methods
of this class.
You should override `lang` class property too (e.g. 'en', 'fr' and so on).
.. attribute:: stopwords
This is a set of stop words of the target language. Default `stopwords`
is empty. This word is used for building index and embedded in JS.
.. attribute:: js_splitter_code
Return splitter function of JavaScript version. The function should be
named as ``splitQuery``. And it should take a string and return list of
strings.
.. versionadded:: 3.0
.. attribute:: js_stemmer_code
Return stemmer class of JavaScript version. This class' name should be
``Stemmer`` and this class must have ``stemWord`` method. This string is
embedded as-is in searchtools.js.
This class is used to preprocess search word which Sphinx HTML readers
type, before searching index. Default implementation does nothing.
N�lang�
language_name� stopwords�js_splitter_code�js_stemmer_rawcodez�
/**
* Dummy stemmer for languages without stemming rules.
*/
var Stemmer = function() {
this.stemWord = function(w) {
return w;
}
}
z(?u)\w+)�options�returnc C s || _ | �|� d S �N)r �init��selfr � r! �:/usr/lib/python3.9/site-packages/sphinx/search/__init__.py�__init__J s zSearchLanguage.__init__c C s dS )zK
Initialize the class with the options the user has given.
Nr! r r! r! r" r N s zSearchLanguage.init)�inputr c C s | j �|�S )z�
This method splits a sentence into words. Default splitter splits input
at white spaces, which should be enough for most languages except CJK
languages.
)�_word_re�findall)r r$ r! r! r" �splitS s zSearchLanguage.split��wordr c C s |S )a�
This method implements stemming algorithm of the Python version.
Default implementation does nothing. You should implement this if the
language has any stemming rules.
This class is used to preprocess search words before registering them in
the search index. The stemming of the Python version and the JS version
(given in the js_stemmer_code attribute) must be compatible.
r! �r r) r! r! r" �stem[ s zSearchLanguage.stemc C sT t |�dkpRt |�dk r8dt|d � k o2dk n pPt|d �dk oP|| jv S )z�
Return true if the target word should be registered in the search index.
This method is called after stemming.
r � iA0 i�0 � )�len�ordr r* r! r! r" � word_filterh s ,��zSearchLanguage.word_filter)�__name__�
__module__� __qualname__�__doc__r �str�__annotations__r �setr r
r r �js_stemmer_code�re�compiler% r r# r r r' r+ �boolr0 r! r! r! r" r s
r )�
SearchEnglish)�sourcer c C s4 t � }| �� D ] }|�d�d }|�|�� � q|S )zs
Parse snowball style word list like this:
* http://snowball.tartarus.org/algorithms/finnish/stop.txt
�|r )r7 �
splitlinesr' �update)r= �result�liner! r! r" �parse_stop_wordy s
rC z sphinx.search.da.SearchDanishz sphinx.search.de.SearchGermanz sphinx.search.es.SearchSpanishz sphinx.search.fi.SearchFinnishz sphinx.search.fr.SearchFrenchz sphinx.search.hu.SearchHungarianz sphinx.search.it.SearchItalianz sphinx.search.ja.SearchJapanesez sphinx.search.nl.SearchDutchz sphinx.search.no.SearchNorwegianz!sphinx.search.pt.SearchPortuguesez sphinx.search.ro.SearchRomanianz sphinx.search.ru.SearchRussianz sphinx.search.sv.SearchSwedishz sphinx.search.tr.SearchTurkishz sphinx.search.zh.SearchChinese)�da�de�en�es�fi�fr�hu�it�ja�nl�no�pt�ro�ru�sv�tr�zh� languagesc @ sZ e Zd ZdZdZdZeed�dd�Zeed�dd �Z ee
d
d �d d
�Z e
ed�dd�Z d
S )�_JavaScriptIndexz�
The search index as JavaScript file that calls a function
on the documentation search object to register the index.
zSearch.setIndex(�))�datar c C s | j t�|� | j S r )�PREFIXr �dumps�SUFFIX)r rX r! r! r" rZ � s z_JavaScriptIndex.dumps)�sr c C sH |t | j�t | j� � }|r6|�| j�r6|�| j�s>td��t�|�S )Nz invalid data)r. rY r[ �
startswith�endswith�
ValueErrorr �loads)r r\ rX r! r! r" r` � s
�z_JavaScriptIndex.loadsN)rX �fr c C s |� | �|�� d S r )�writerZ )r rX ra r! r! r" �dump� s z_JavaScriptIndex.dump)ra r c C s | � |�� �S r )r` �read)r ra r! r! r" �load� s z_JavaScriptIndex.load)
r1 r2 r3 r4 rY r[ r r5 rZ r` r rc re r! r! r! r" rV � s rV c sL e Zd ZdZejedd�� fdd� Zee d�dd�Z
e dd�d d
�Z � Z
S ) �
WordCollectorzG
A special visitor that collects words for the `IndexBuilder`.
N)�documentr r c s" t � �|� g | _g | _|| _d S r )�superr# � found_words�found_title_wordsr )r rg r �� __class__r! r" r# � s zWordCollector.__init__)�noder c C sJ t |tjtjf�rF|�d�dkrF|�d�}|d u r6dS || jjkrFdS dS )N�name�keywordsr TF)�
isinstancer �metaZ
docutils_meta�getr )r rm Z meta_langr! r! r" �is_meta_keywords� s �
z WordCollector.is_meta_keywordsc C s t |tj�rtj�n�t |tj�r~d|�dd��� v rvt�dd|� � �}t�dd|�}t�dd|�}| j
� | j �|�� tj�n�t |tj
�r�| j
� | j �|� � �� nft |tj�r�| j� | j �|� � �� n@t |t��r
| �|��r
|d }dd � |�d
�D �}| j
� |� d S ) N�html�format� z(?is)<style.*?</style>z(?is)<script.*?</script>z<[^<]+?>Zcontentc S s g | ] }|� � �qS r! )�strip)�.0�keywordr! r! r" �
<listcomp>� � z0WordCollector.dispatch_visit.<locals>.<listcomp>�,)rp r
ZcommentZSkipNode�rawrr r' r9 �subZastextri �extendr �Text�titlerj r rs )r rm Znodetextro r! r! r" �dispatch_visit� s" z WordCollector.dispatch_visit)r1 r2 r3 r4 r
rg r r# r r; rs r r� �
__classcell__r! r! rk r" rf � s rf c
@ sN e Zd ZdZeed�Zeee edd�dd�Z
e e dd�dd �Z
e e dd�d
d �Ze eef e eeeeeeeef f d �d
d�Ze ee eee f e eee f f d �dd�Ze ee f d�dd�Zed�dd�Zee dd�dd�Zeeeejdd�dd�Ze ee f d�d d �Zee d�d d �Z e e d�d d!�Z ed�d"d#�Z dS )$� IndexBuilderzi
Helper class that creates a search index based on the doctrees
passed to the `feed` method.
)r �pickleN)�envr r �scoringr c C s� || _ i | _i | _i | _i | _i | _i | _i | _t� |�}|d u r^d|v r^t� |�
d�d �}|d u rrt |�| _ n>t
|t�r�|�dd�\}}tt|�|�}||�| _ n
||�| _ |r�t|d�� }|�� �� | _W d � q�1 s�0 Y nd| _t| _d S )N�_r �.� �rbrv )r� �_titles�
_filenames�_mapping�_title_mapping� _stem_cache� _objtypes� _objnamesrU rr r' r< r rp r5 �rsplit�getattrr �openrd �decode�js_scorer_coder r ) r r� r r r� Z
lang_class�moduleZ classname�fpr! r! r" r# � s.
.zIndexBuilder.__init__)�streamru r c s� t |t�r| j| }|�|�}t |t�r:|�d�| jjkrBtd��|d � tt � |d ��| _
tt � |d ��| _ t tt
f t ttt f d�� fdd� }||d �| _||d
�| _d S ) z Reconstruct from frozen data.�
envversionz
old format�docnames� filenames�titles)�mappingr c sJ i }| � � D ]8\}}t|t�r.� | h||< q � fdd�|D �||< q |S )Nc s h | ] }� | �qS r! r! )rx �i�Zindex2fnr! r" � <setcomp>& r{ z8IndexBuilder.load.<locals>.load_terms.<locals>.<setcomp>)�itemsrp �int)r� �rv�k�vr� r! r" �
load_terms s
z%IndexBuilder.load.<locals>.load_terms�terms�
titletermsN)rp r5 �formatsre �dictrr r� �versionr_ �zipr� r� r r r
r� r� )r r� ru �frozenr� r! r� r" re s
�( zIndexBuilder.loadc C s( t |t�r| j| }|�| �� |� dS )z"Dump the frozen index to a stream.N)rp r5 r� rc �freeze)r r� ru r! r! r" rc - s
zIndexBuilder.dump)�fn2indexr c C sH i }| j }| j}t| jj�� �D �] \}}t|�� �D �]\}}} }
} } |
|vrRq6| dk r\q6t�|�}t�|�}|� d�\}
}}|�
|
g �}z||| f }W n^ t y� t |�}|||| f< |j
�| �}|r�|| t|�|��f||< n|| | f||< Y n0 | |k�r d}n | | d | k�r$d}n| }|�||
|| ||f� q6q |S )Nr r� rv �-)r� r� �sortedr� Zdomainsr� � get_objectsrt �escape�
rpartition�
setdefault�KeyErrorr. Z object_typesrr r5 Z
get_type_name�append)r r� r� ZotypesZonamesZ
domainname�domain�fullnameZdispname�type�docname�anchorZprio�prefixr� rn ZplistZ typeindexZotypeZ shortanchorr! r! r" r� 3 s@
�
�
zIndexBuilder.get_objectsc s| i i f}t || j| jf�D ]\\}}|�� D ]J\}}t|�dkrZ|\}|� v rt� | ||< q*t� fdd�|D ��||< q*q|S )Nr� c s g | ]}|� v r� | �qS r! r! )rx �fn�r� r! r" rz a r{ z*IndexBuilder.get_terms.<locals>.<listcomp>)r� r� r� r� r. r� )r r� Zrvsr� r� r� r� r� r! r� r" � get_termsX s zIndexBuilder.get_terms)r c
s� t t� j�� �� \}}� fdd�|D �}dd� t|�D �}� �|�\}}� �|�}dd� � j�� D �}� j} t ||||||| |� j
j d� S )z/Create a usable data structure for serializing.c s g | ]}� j �|��qS r! )r� rr )rx r� �r r! r" rz g r{ z'IndexBuilder.freeze.<locals>.<listcomp>c S s i | ]\}}||�qS r! r! )rx r� ra r! r! r" �
<dictcomp>h r{ z'IndexBuilder.freeze.<locals>.<dictcomp>c S s& i | ] \}}||d d |d �qS )r �:r� r! )rx r� r� r! r! r" r� l r{ ) r� r� r� r� �objects�objtypes�objnamesr� r� ) r� r� r� r� � enumerater� r� r� r� r� r� r� )
r r� r� r� r� r� Z title_termsr� r� r� r! r� r" r� d s
�zIndexBuilder.freezec C s d| j j| j j f S )Nz
%s (code: %s))r r r� r! r! r" �labelr s zIndexBuilder.label)r� r c C s| i }i }|D ]*}|| j v r | j | ||< | j| ||< q || _ || _| j�� D ]}|�|� qN| j�� D ]}|�|� qhdS )z-Remove data for all docnames not in the list.N)r� r� r� �values�intersection_updater� )r r� Z
new_titlesZ
new_filenamesr� Z wordnamesr! r! r" �pruneu s
zIndexBuilder.prune)r� �filenamer� �doctreer c s� |� j |< |� j|< t|� j�}|�|� ttd�� fdd� }� jj}|jD ]J}||�} || �rx� j� | t
� �� |� qL||�rL� j� |t
� �� |� qL|j D ]V}||�} || �s�||�r�|} |� j�
| t
� �v }
|| �r�|
s�� j� | t
� �� |� q�dS )z Feed a doctree to the index.r( c sD z � j | W S ty> � j�| ��� � j | < � j | Y S 0 d S r )r� r� r r+ �lower)r) r� r! r" r+ � s
z IndexBuilder.feed.<locals>.stemN)r� r� rf r �walkr5 r0 rj r� r� r7 �addri rr r� ) r r� r� r� r� Zvisitorr+ �_filterr) Z stemmed_wordZalready_indexedr! r� r" �feed� s&
zIndexBuilder.feedc C s: | j jr| j j}n| j}| �� t�t| j j��| j|d�S )N)Z search_language_stemming_codeZsearch_language_stop_wordsZsearch_scorer_toolZsearch_word_splitter_code)r r �get_js_stemmer_coder rZ r� r r� )r r r! r! r" �context_for_searchtool� s
�z#IndexBuilder.context_for_searchtoolc C s& | j jr dd� d| j jfD �S g S dS )z8Returns a list of non-minified stemmer JS files to copy.c S s g | ]}t �td d|��qS )�searchznon-minified-js)r �joinr )rx Zfnamer! r! r" rz � s �z8IndexBuilder.get_js_stemmer_rawcodes.<locals>.<listcomp>�base-stemmer.jsN)r r r� r! r! r" �get_js_stemmer_rawcodes� s
�z$IndexBuilder.get_js_stemmer_rawcodesc C s d S r r! r� r! r! r" �get_js_stemmer_rawcode� s z#IndexBuilder.get_js_stemmer_rawcodec C s� | j jr�t�tdd�}tt�|d���}|�� }W d � n1 sD0 Y tt�|| j j���}|�� }W d � n1 s�0 Y d||| j jf S | j jS dS )z<Returns JS code that will be inserted into language_data.js.r� z minified-jsr� Nz%s
%s
Stemmer = %sStemmer;) r r r r� r r� rd r r8 )r Zjs_dirZjs_fileZbase_jsZ language_jsr! r! r" r� � s && �z IndexBuilder.get_js_stemmer_code) r1 r2 r3 r4 r r� r� r r5 r r# r r re rc r� r r r� r� r� r� r r� r
rg r� r� r� r r� r� r! r! r! r" r� � s& �#
� %0 !
r� ),r4 rt r� r9 � importlibr �osr �typingr r r r r r r
r r Zdocutilsr
Zdocutils.nodesr r Zsphinxr r Zsphinx.environmentr Zsphinx.search.jssplitterr Z sphinx.utilr r Zsphinx.search.enr< r5 rC rU r6 rV Zjs_indexZ NodeVisitorrf r� r! r! r! r" �<module> sJ , \ �-