h ddlZddlZddlmZmZmZddlmZddlm Z ddl m Z ddl m Z dd lmZmZmZdd lmZdd lmZdd lmZdd lmZdZdZdZdZdZdZdZ dZ!dZ"dZ#dZ$dZ%dZ&dZ'dZ(dZ)dZ*dZ+d Z,d!Z-d"Z.d#Z/d$Z0d%Z1d&Z2d'Z3d(Z4d)Z5d*Z6d+Z7d,Z8d-Z9d.Z:d/Z;d0Zd3Z?d4Z@d5ZAd6ZBd7ZCd8ZDGd9d:ZEGd;d| t t#|gz } t#|dkr| t|z } | d | tt%jd|Dz } n'#t($r| |z } YnwxYw|j| S)a Create the search index. The index must not already exist. ### Parameters: - **fields**: a list of TextField or NumericField objects - **no_term_offsets**: If true, we will not save term offsets in the index - **no_field_flags**: If true, we will not save field flags that allow searching in specific fields - **stopwords**: If not None, we create the index with this custom stopword list. The list can be empty - **max_text_fields**: If true, we will encode indexes as if there were more than 32 text fields which allows you to add additional fields (beyond 32). - **temporary**: Create a lightweight temporary index which will expire after the specified period of inactivity (in seconds). The internal idle timer is reset whenever the index is searched or added to. - **no_highlight**: If true, disabling highlighting support. Also implied by no_term_offsets. - **no_term_frequencies**: If true, we avoid saving the term frequencies in the index. - **skip_initial_scan**: If true, we do not scan and index. For more information see `FT.CREATE `_. NrSCHEMAc3>K|]}|VdSN redis_args.0fs r' z.SearchCommands.create_index..**J*Ja1<<>>*J*J*J*J*J*Jr)) CREATE_CMD index_nameargsappendr isinstanceintrrrrrrlisttuplesetrlen itertoolschain TypeErrorr/execute_command) r&fieldsno_term_offsetsno_field_flags stopwords definitionmax_text_fields temporary no_highlightno_term_frequenciesskip_initial_scanr7s r' create_indexzSearchCommands.create_indexHsPDO,  ! JO #D  ' KK & & &  Z 3%?%? KK " " " KK " " "  # KK " " "   KK     " KK ! ! !  ! KK  ) KK ( ( (  Z D%;M%N%N YI/ /D9~~!!Y' H ( D*J*J6*J*J*JKLL LDD ( ( ( F%%'' 'DDD ($t#T**s(F**!G Gct|jddg} |ttjd|Dz }n'#t $r||z }YnwxYw|j|S)a Alter the existing search index by adding new fields. The index must already exist. ### Parameters: - **fields**: a list of Field objects to add for the index For more information see `FT.ALTER `_. r+ADDc3>K|]}|VdSr-r.r0s r'r3z2SearchCommands.alter_schema_add..r4r)) ALTER_CMDr6r;r?r@rAr/rB)r&rCr7s r'alter_schema_addzSearchCommands.alter_schema_adds4?He< ( D*J*J6*J*J*JKLL LDD ( ( ( F%%'' 'DDD ($t#T**s(;!AAcP|rdnd}|t|j|S)ai Drop the index if it exists. Replaced `drop_index` in RediSearch 2.0. Default behavior was changed to not delete the indexed documents. ### Parameters: - **delete_documents**: If `True`, all documents will be deleted. For more information see `FT.DROPINDEX `_. DD)rB DROPINDEX_CMDr6)r&delete_documents delete_strs r' dropindexzSearchCommands.dropindexs,.5TT2 ##M4?JOOOr)?c  |s| rd}t|j||g} |r| d|*| d| ||rC| d|r| d| r| d|r| d|gz } | d | tt j| z } | |j| S|j| S) zS Internal add_document used for both batch and single doc indexing TNOSAVENPAYLOADREPLACEPARTIALNOCREATELANGUAGEFIELDS)ADD_CMDr6r8r;r?r@itemsrB) r&doc_idconnnosavescorepayloadreplacepartiallanguage no_createrCr7s r' _add_documentzSearchCommands._add_documents*"  i G&%8  " KK ! ! !   KK " " " KK  ( KK " " " ' I&&& ( J'''  + Z* *D H Y_fllnn5666  '4'. .#t#T**r)ct|j||g}|r|d|r|d|gz }| |j|S|j|S)zX Internal add_document_hash used for both batch and single doc indexing r^ra) ADDHASH_CMDr6r8rB)r&rerfrhrlrjr7s r'_add_document_hashz!SearchCommands._add_document_hashskT_fe<  # KK " " "  + Z* *D  '4'. .#t#T**r)z2.0.0z2deprecated since redisearch 2.0, call hset instead)versionreasonc 0|j|fd|||||||d| S)a Add a single document to the index. ### Parameters - **doc_id**: the id of the saved document. - **nosave**: if set to true, we just index the document, and don't save a copy of it. This means that searches will just return ids. - **score**: the document ranking, between 0.0 and 1.0 - **payload**: optional inner-index payload we can save for fast i access in scoring functions - **replace**: if True, and the document already is in the index, we perform an update and reindex the document - **partial**: if True, the fields specified will be added to the existing document. This has the added benefit that any fields specified with `no_index` will not be reindexed again. Implies `replace` - **language**: Specify the language used for document tokenization. - **no_create**: if True, the document is only updated and reindexed if it already exists. If the document does not exist, an error will be returned. Implies `replace` - **fields** kwargs dictionary of the document fields to be saved and/or indexed. NOTE: Geo points shoule be encoded as strings of "lon,lat" N)rfrgrhrirjrkrlrm)rn) r&rergrhrirjrkrlrmrCs r' add_documentzSearchCommands.add_documentsKV"t!         r)c6||d|||S)a Add a hash document to the index. ### Parameters - **doc_id**: the document's id. This has to be an existing HASH key in Redis that will hold the fields the index needs. - **score**: the document ranking, between 0.0 and 1.0 - **replace**: if True, and the document already is in the index, we perform an update and reindex the document - **language**: Specify the language used for document tokenization. N)rfrhrlrj)rq)r&rerhrlrjs r'add_document_hashz SearchCommands.add_document_hash(s, && UXw'   r)czt|j|g}|r|d| |j|S|j|S)a Delete a document from index Returns 1 if the document was deleted, 0 if not ### Parameters - **delete_actual_document**: if set to True, RediSearch also delete the actual document if it is in the index rT)DEL_CMDr6r8rB)r&rerfdelete_actual_documentr7s r'delete_documentzSearchCommands.delete_document<sR&1 !  KK     '4'. .#t#T**r)c|j|}d|D}|} |d=n#t$rYnwxYwt dd|i|S). Load a single document by id cNi|]"\}}t|t|#Sr r1kvs r' z0SearchCommands.load_document..T* D D DTQillIaLL D D Dr)idrclienthgetallrdKeyErrorrr&rrCf2s r' load_documentzSearchCommands.load_documentOs$$R(( D DV\\^^ D D D t     D ((2((((sA A  A c4|jt|jg|RS)z Returns the full contents of multiple documents. ### Parameters - **ids**: the ids of the saved documents. )rBMGET_CMDr6)r&idss r'getzSearchCommands.get^s#$t#HdoDDDDDr)c|t|j}tt|}t t ||S) Get info an stats about the the current index, including the number of documents, memory consumption, etc For more information see `FT.INFO `_. rBINFO_CMDr6mapr dictzipr&resits r'infozSearchCommands.infojs@""8T_== C CBKK   r) query_paramsc4|gSg}t|dkr~|d|t|dz|D]/\}}||||0|S)Nrparamsr)r>r8rd)r&rr7keyvalues r'get_params_argszSearchCommands.get_params_argsvs  I |  q KK ! ! ! KKL))A- . . .*0022 # # U C    E"""" r)c(|jg}t|trt|}t|tst dt |||z }|||z }||fS)NzBad query type )r6r9strr ValueErrortypeget_argsr)r&queryrr7s r'_mk_query_argszSearchCommands._mk_query_argss  eS ! ! !%LLE%'' >`_. r@@duration has_payload with_scores rtimerB SEARCH_CMDr9rr _no_content_with_payloads _with_scoresr&rrr7strs r'searchzSearchCommands.searchs ))%l)KK e Y[["d":5555 c8 $ $ J ! !ikkB&&0,*     r)c\|||\}}|jtg|RS)zReturns the execution plan for a complex query. For more information see `FT.EXPLAIN `_. r)rrB EXPLAIN_CMD)r&rrr7 query_texts r'explainzSearchCommands.explains< ..u<.PPj#t#K7$7777r)c td)Nz#EXPLAINCLI will not be implemented.)NotImplementedError)r&rs r' explain_clizSearchCommands.explain_clis!"GHHHr)ct|tr8t|j}t|jg|z}nLt|tr'd}td|jg|z}ntd||| |z }|j |}| |||S) Issue an aggregation query. ### Parameters **query**: This can be either an `AggregateRequest`, or a `Cursor` An `AggregateResult` object is returned. You can access the rows from its `rows` property, which will always yield the rows of the result. For more information see `FT.AGGREGATE `_. TREAD Bad query r9r bool_cursor AGGREGATE_CMDr6 build_argsr CURSOR_CMDrrrB_get_aggregate_resultr&rr has_cursorcmdraws r' aggregatezSearchCommands.aggregates" e- . . 1em,,J $/2U5E5E5G5GGCC v & & 1Jvt7%:J:J:L:LLCC[%00 0 t##L111"d"C())#ujAAAr)c(|rCt|tr|d|_|}nt|d}|d}nd}t|tr|jr|d}|dd}n d}|dd}t |||S)Nr rr)r9rcidr _with_schemar)r&rrrcursorschemarowss r'rz$SearchCommands._get_aggregate_results  %(( (F Aa&CCF e- . . 53E VFqrr7DDFqrr7DtVV444r)limitedctj}t|jdg}|r|d|dt |t rd|d<||z }nYt |tr5d|d<||z }|| |z }ntd|j |}t |t r#| |d||j }nAt|d|j tj|z d z|j|j }|t%|d fS) a Performs a search or aggregate command and collects performance information. ### Parameters **query**: This can be either an `AggregateRequest`, `Query` or string. **limited**: If set to True, removes details of reader iterator. **query_params**: Define one or more value parameters. Each parameter has a name and a value. rULIMITEDQUERY AGGREGATErSEARCHz5Must provide AggregateRequest object or Query object.rrrr )r PROFILE_CMDr6r8r9r rrrrrrBrrrrrrr )r&rrrrrrresults r'profilezSearchCommands.profilest$Y[[DOR0  " JJy ! ! ! 7 e- . . V CF 5##%% %CC u % % VCF 5>>## #C 4'' 55 5CCTUU U"d"C( e- . . //Au}MMFFA%%)++*f4!0!. F}SV,,,,r)ct|j|g}|r|d|g|r|dd|g|r|dd|g|j|}i}|dkr|S|D]e}t |t r|dkrt |dkr2|ds;|ddsJd|dD||d <f|S)  Issue a spellcheck query ### Parameters **query**: search query. **distance***: the maximal Levenshtein distance for spelling suggestions (default: 1, max: 4). **include**: specifies an inclusion custom dictionary. **exclude**: specifies an exclusion custom dictionary. For more information see `FT.SPELLCHECK `_. DISTANCETERMSINCLUDEEXCLUDErrc0g|]}|d|ddSrr )rh suggestionrr1_items r' z-SearchCommands.spellcheck..[5+++@E%(%(;;+++r)r SPELLCHECK_CMDr6extendrBr9r:r> r&rdistanceincludeexcluderr corrections _corrections r' spellcheckzSearchCommands.spellcheck#s4t6  / JJ H- . . .  6 JJG4 5 5 5  6 JJG4 5 5 5"d"C( !88   K+s++  q0@0@;1$$q> q>!$ "++ITUV+++K A ' 'r)cRt|g}|||j|S)zAdds terms to a dictionary. ### Parameters - **name**: Dictionary name. - **terms**: List of items for adding to the dictionary. For more information see `FT.DICTADD `_. ) DICT_ADD_CMDrrBr&nametermsrs r'dict_addzSearchCommands.dict_adda0T" 5#t#S))r)cRt|g}|||j|S)aDeletes terms from a dictionary. ### Parameters - **name**: Dictionary name. - **terms**: List of items for removing from the dictionary. For more information see `FT.DICTDEL `_. ) DICT_DEL_CMDrrBrs r'dict_delzSearchCommands.dict_delorr)c(t|g}|j|S)zDumps all terms in the given dictionary. ### Parameters - **name**: Dictionary name. For more information see `FT.DICTDUMP `_. ) DICT_DUMP_CMDrB)r&rrs r' dict_dumpzSearchCommands.dict_dump}sd##t#S))r)c8td||g}|j|}|dkS)Set runtime configuration option. ### Parameters - **option**: the name of the configuration option. - **value**: a value for the configuration option. For more information see `FT.CONFIG SET `_. SETOK CONFIG_CMDrBr&optionrrrs r' config_setzSearchCommands.config_sets+5&%0"d"C(d{r)cbtd|g}i}|j|}|r|D]}|d||d<|S)Get runtime configuration option value. ### Parameters - **option**: the name of the configuration option. For more information see `FT.CONFIG GET `_. GETr rr r&r rrrkvss r' config_getzSearchCommands.config_getsS5&)"d"C(  % % %!!fCF  r)cD|t|j|S)z Return a list of all possible tag values ### Parameters - **tagfield**: Tag field name For more information see `FT.TAGVALS `_. )rB TAGVALS_CMDr6)r&tagfields r'tagvalszSearchCommands.tagvalss##K(KKKr)cD|t||jS)z Alias a search index - will fail if alias already exists ### Parameters - **alias**: Name of the alias to create For more information see `FT.ALIASADD `_. )rB ALIAS_ADD_CMDr6r&aliass r'aliasaddzSearchCommands.aliasadds##M5$/JJJr)cD|t||jS)z Updates an alias - will fail if alias does not already exist ### Parameters - **alias**: Name of the alias to create For more information see `FT.ALIASUPDATE `_. )rBALIAS_UPDATE_CMDr6rs r' aliasupdatezSearchCommands.aliasupdates##$4eT_MMMr)c8|t|S)z Removes an alias to a search index ### Parameters - **alias**: Name of the alias to delete For more information see `FT.ALIASDEL `_. )rB ALIAS_DEL_CMDrs r'aliasdelzSearchCommands.aliasdels##M5999r)cj|d}|D]}t||j|jg}|dr|d|jr/|d||j|j||dS)aJ Add suggestion terms to the AutoCompleter engine. Each suggestion has a score and string. If kwargs["increment"] is true and the terms are already in the server's dictionary, we increment their scores. For more information see `FT.SUGADD `_. F transaction incrementINCRr] pipelineSUGADD_COMMANDstringrhrr8rirBexecuter&r suggestionskwargspipesugr7s r'sugaddzSearchCommands.sugadds}}}// ( (C"CSY?Dzz+&& $ F###{ ) I&&& CK((( D $ ' ' '||~~b!!r)c8|t|S)z Return the number of entries in the AutoCompleter index. For more information see `FT.SUGLEN `_. )rBSUGLEN_COMMAND)r&rs r'suglenzSearchCommands.suglens ##NC888r)c:|t||S)z Delete a string from the AutoCompleter index. Returns 1 if the string was found and deleted, 0 otherwise. For more information see `FT.SUGDEL `_. )rBSUGDEL_COMMAND)r&rr,s r'sugdelzSearchCommands.sugdels##NC@@@r) ct||d|g}|r|t|r|t|r|t|j|}g} |s| St |||} d| DS)t Get a list of suggestions from the AutoCompleter, for a given prefix. Parameters: prefix : str The prefix we are searching. **Must be valid ascii or utf-8** fuzzy : bool If set to true, the prefix search is done in fuzzy mode. **NOTE**: Running fuzzy searches on short (<3 letters) prefixes can be very slow, and even scan the entire index. with_scores : bool If set to true, we also return the (refactored) score of each suggestion. This is normally not needed, and is NOT the original score inserted into the index. with_payloads : bool Return suggestion payloads num : int The maximum number of results we return. Note that we might return less. The algorithm trims irrelevant suggestions. Returns: list: A list of Suggestion objects. If with_scores was False, the score of all suggestions is 1. For more information see `FT.SUGGET `_. MAXcg|]}|Srrr1ss r'rz)SearchCommands.sugget..3"""a"""r)SUGGET_COMMANDr8rrrrBr r&rprefixfuzzynumr with_payloadsr7retresultsparsers r'suggetzSearchCommands.suggetsDVUC8   KK     $ KK # # #  & KK % % %"d"D) N!+}cBB""6""""r)ct|j|g}|r|dg|||j|S)a Updates a synonym group. The command is used to create or update a synonym group with additional terms. Only documents which were indexed after the update will be affected. Parameters: groupid : Synonym group id. skipinitial : bool If set to true, we do not scan and index. terms : The terms. For more information see `FT.SYNUPDATE `_. r) SYNUPDATE_CMDr6rrB)r&groupid skipinitialrrs r' synupdatezSearchCommands.synupdate5sR$dow7  , JJ)* + + + 5#t#S))r)c|t|jfdtdt dDS)a Dumps the contents of a synonym group. The command is used to dump the synonyms data structure. Returns a list of synonym terms and their synonym group ids. For more information see `FT.SYNDUMP `_. c4i|]}||dzS)r r)r1irs r'rz*SearchCommands.syndump..Ws'BBBqAAE BBBr)rr)rB SYNDUMP_CMDr6ranger>)r&rs @r'syndumpzSearchCommands.syndumpMsH"";@@BBBBE!SXXq,A,ABBBBr))r") FFNNFNFFF)F)NFrZNFFNF)NrZNF)FrZNFFNF)rZNF)NFr-)FNNNNFr:FF)2__name__ __module__ __qualname____doc__r(rMrRrYrnrqrrurwr{rrrrrrr:floatrrrrrrrrr rrrrrrrr rrrrr"r3r6r9rLrQrWrr)r'r!r!?s>>>>!E+E+E+E+N+++( P P P P$(+(+(+(+VDI++++( T 3 3 3 3 j T    "++++& ) ) ) E E E ! ! ! !$sE#sE/,B'B"CT"IJ     $sE#sE/.rr)r rrs r'rzAsyncSearchCommands.spellchecksJt6  / JJ H- . . .  6 JJG4 5 5 5  6 JJG4 5 5 5(D(#....... !88   K+s++  q0@0@;1$$q> q>!$ ++ITUV+++K A ' 'r)cHKtd||g}|j|d{V}|dkS)rrNrr r s r'r zAsyncSearchCommands.config_setsA5&%0(D(#.......d{r)crKtd|g}i}|j|d{V}|r|D]}|d||d<|S)rrNr rr rs r'rzAsyncSearchCommands.config_getsi5&)(D(#.......  % % %!!fCF  r)cK|j|d{V}d|D}|} |d=n#t$rYnwxYwt dd|i|S)r}NcNi|]"\}}t|t|#Srr rs r'rz5AsyncSearchCommands.load_document..rr)rrrrs r'rz!AsyncSearchCommands.load_documents{**2........ D DV\\^^ D D D t     D ((2((((sA AAczK|d}|D]}t||j|jg}|dr|d|jr/|d||j|j||d{VdS)aI Add suggestion terms to the AutoCompleter engine. Each suggestion has a score and string. If kwargs["increment"] is true and the terms are already in the server's dictionary, we increment their scores. For more information see `FT.SUGADD `_. Fr$r&r'r]Nr(r)r.s r'r3zAsyncSearchCommands.sugadds}}}// ( (C"CSY?Dzz+&& $ F###{ ) I&&& CK((( D $ ' ' 'llnn$$$$$$b))r)Fr:c,Kt||d|g}|r|t|r|t|r|t|j|d{V}g} |s| St |||} d| DS)r<r=Ncg|]}|Srrr?s r'rz.AsyncSearchCommands.sugget..HrAr)rBrDs r'rLzAsyncSearchCommands.suggetsDVUC8   KK     $ KK # # #  & KK % % %(D($/////// N!+}cBB""6""""r)r-rXrY)rZr[r\rrrrrr:r^rrrr rrr3rLrr)r'r`r`Zs3 ! ! !;?  S%Z  3c3o 667    D;?BBS%Z B3c3o 667BBBB<----^   " ) ) )***0RW0#0#0#0#0#0#r)r`)Gr?rtypingrrr redis.clientr redis.utilsrhelpersr _utilr aggregationr rrdocumentrrrrrrrrr5rQrrcrpDROP_CMDrVrEXPLAINCLI_CMDryrrrrrrrGET_CMDrr rrrr!rr+r8r5rCrNrUrrrrrrrrrrrr!r`rr)r'rws5 ((((((((((!!!!!!++++++######BBBBBBBBBB((((((                      #         #   X CX CX CX CX CX CX CX Cvn#n#n#n#n#.n#n#n#n#n#r)