dhtSrSSKJr SSKrSSKrSSKrSSKJrJrJ r J r J r J r J r SSKrSSKJr SSKJr SSKJr SSKJr SS KJr \"S /5rS rS rS r\R>"\R@"S 55RBr"\RF"\R@"S55RBr$\RJRBr&SSjr'SSjr(SSjr)SSjr*SSjr+"SS\5r,g)z&Wrapper around TileDB vector database.) annotationsN)AnyDictIterableListMappingOptionalTuple)Document) Embeddings guard_import) VectorStore)maximal_marginal_relevance euclidean documentsvectorsuint64float32c.[S5[S54$)z@Import tiledb-vector-search if available, otherwise raise error.tiledb.vector_searchtiledbr _/var/www/html/shao/venv/lib/python3.13/site-packages/langchain_community/vectorstores/tiledb.pydependable_tiledb_importrs +,X rc(U[R$) Get the URI of the vector index.)VECTOR_INDEX_NAMEurigroups rget_vector_index_uri_from_groupr##s " # ' ''rc(U[R$)zGet the URI of the documents array from group. Args: group: TileDB group object. Returns: URI of the documents array. )DOCUMENTS_ARRAY_NAMEr r!s r"get_documents_array_uri_from_groupr&(s % & * **rcUS[3$)r/)rr s rget_vector_index_urir*4sU!%& ''rcUS[3$)z#Get the URI of the documents array.r()r%r)s rget_documents_array_urir,9sU!() **rc\rSrSrSrSSSSSS.S'Sjjr\S(Sj5rS S\S .S)S jjr S SS S .S*Sjjr S SS S .S+Sjjr S,S-Sjjr S,S.Sjjr S S SSS.S/SjjrS0S1SjjrS0S2Sjjr\SSS.S3Sjj5r\SS\SSSS.S4Sjj5rS5S6SjjrS7S8Sjjr\SS\S SSS4S9S!jj5r\SS\SSSS.S:S"jj5r\\SSS#.S;S$jj5rSaTileDB vector store. To use, you should have the ``tiledb-vector-search`` python package installed. Example: .. code-block:: python from langchain_community import TileDB embeddings = OpenAIEmbeddings() db = TileDB(embeddings, index_uri, metric) NF)vector_index_uridocs_array_uriconfig timestampallow_dangerous_deserializationc U(d [S5eXlURUlX lX0lX`l[S5[S5pU RUS9 U RURS5n US:waUO [U 5Ul US:waUO [U 5Ul U R5 U RURS5n U RR!S5UlU R5 XplUR"S:XaDU R&R("S URUR UR$S .U D6UlOSUR"S :XaCU R,R."S URUR UR$S .U D6UlS S S 5 g !,(df  g =f) aNInitialize with necessary components. Args: allow_dangerous_deserialization: whether to allow deserialization of the data which involves loading data using pickle. data can be modified by malicious actors to deliver a malicious payload that results in execution of arbitrary code on your machine. aTileDB relies on pickle for serialization and deserialization. This can be dangerous if the data is intercepted and/or modified by malicious actors prior to being de-serialized. If you are sure that the data is safe from modification, you can set allow_dangerous_deserialization=True to proceed. Loading of compromised data using pickle can result in execution of arbitrary code on your machine.rr ctx_or_configrr0 index_typeFLAT)r r3r4IVF_FLATNr) ValueError embedding embed_queryembedding_function index_urimetricr3r scope_ctxGroupr#r1r&r2closemetagetr:r4 flat_index FlatIndex vector_indexivf_flat_index IVFFlatIndex)selfr>rArBr1r2r3r4r5kwargs tiledb_vsr index_groupr"s r__init__TileDB.__init__Ls,/2 #"+"7"7"   / 0  "  F  3 ,,t~~s;K$r)!4[A  !"R'7 D      LL!6!6)rMs r embeddingsTileDB.embeddingss ~~r)kfilterscore_thresholdc ^[S5n/nURURSURURS9n[ X5GH}upU S:XaU S:XaMU [ :Xa U [:XaM*Xn U b[U S5S:Xa[SU SU 35eU RS 5n [[U SS5S 9mU bj[R"[R "U R#55R%[R&5R)55n U TlUb}UR-5VVs0sH upU[/U[05(dU/OU_M" nnn[3U4S jUR-555(aUR5TU 45 GMgGMjUR5TU 45 GM UR75 UV V s/sHupX::dM X4PM nn n USU$s snnfs sn n f) aTurns TileDB results into a list of documents and scores. Args: ids: List of indices of the documents in the index. scores: List of distances of the documents in the index. k: Number of Documents to return. Defaults to 4. filter (Optional[Dict[str, Any]]): Filter by metadata. Defaults to None. score_threshold: Optional, a floating point value to filter the resulting set of retrieved docs Returns: List of Documents and scores. rr9r4r3rNtextzCould not find document for id z, got metadata) page_contentc3b># UH%upTRRU5U;v M' g7frT)r^rG).0keyvalue result_docs r /TileDB.process_index_results..s0&4 ''++C0E9&4s,/)ropenr2r4r3zip MAX_UINT64 MAX_FLOAT_32lenr=rGr strpickleloadsnparraytolistastypeuint8tobytesr^items isinstancelistallappendrE)rMidsscoresrXrYrZrdocs docs_arrayidxscoredocpickled_metadatar^rbrcrds @rprocess_index_resultsTileDB.process_index_resultss*h'[[   t{{! c*JCaxEQJj Ul%:/C{c#f+.!3 #B3%vcU!STT"wwz2 !s3v;q>/BCJ+!<<HH-4467>>rxxHPPR'/ #!'-lln&4  5$(?(?%UJ&4&,llnKKU 34   Z/05+6 /3Ptu7O  tPBQxQs &G>! H/HrXrYfetch_kc SU;aURS5nO[nURR"[R "[R "U5R [R5/5R [R54SUcUOU0UD6upxURUSUSX2US9$)aReturn docs most similar to query. Args: embedding: Embedding vector to look up documents similar to. k: Number of Documents to return. Defaults to 4. filter (Optional[Dict[str, Any]]): Filter by metadata. Defaults to None. fetch_k: (Optional[int]) Number of Documents to fetch before filtering. Defaults to 20. **kwargs: kwargs to be passed to similarity search. Can include: nprobe: Optional, number of partitions to check if using IVF_FLAT index score_threshold: Optional, a floating point value to filter the resulting set of retrieved docs Returns: List of documents most similar to the query text and distance in float for each. Lower score represents more similarity. rZrXrrzr{rYrXrZ) pop MAX_FLOATrJqueryrorprrrr) rMr>rXrYrrNrZdis r&similarity_search_with_score_by_vector-TileDB.similarity_search_with_score_by_vectors4  &$jj):;O'O  && HHbhhy)00<= > E Ebjj Q >aw   ))!QqT&*  rc VURU5nUR"U4UUUS.UD6nU$)aReturn docs most similar to query. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. fetch_k: (Optional[int]) Number of Documents to fetch before filtering. Defaults to 20. Returns: List of documents most similar to the query text with Distance as float. Lower score represents more similarity. r)r@r)rMrrXrYrrNr>r|s rsimilarity_search_with_score#TileDB.similarity_search_with_scoresF,++E2 ::       rc hUR"U4UUUS.UD6nUVVs/sHupxUPM snn$s snnf)aReturn docs most similar to embedding vector. Args: embedding: Embedding to look up documents similar to. k: Number of Documents to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. fetch_k: (Optional[int]) Number of Documents to fetch before filtering. Defaults to 20. Returns: List of Documents most similar to the embedding. r)r) rMr>rXrYrrNdocs_and_scoresr_s rsimilarity_search_by_vector"TileDB.similarity_search_by_vectorsK(EE      #22//222s .c fUR"U4X#US.UD6nUVVs/sHupxUPM snn$s snnf)aReturn docs most similar to query. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. filter: (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. fetch_k: (Optional[int]) Number of Documents to fetch before filtering. Defaults to 20. Returns: List of Documents most similar to the query. r)r) rMrrXrYrrNrrrs rsimilarity_searchTileDB.similarity_search7sE(;;  w :@ #22//222s -?rXr lambda_multrYc SU;aURS5nO[nURR"[R "[R "U5R [R5/5R [R54SUcUOUS-0UD6upURU SUSUUcUOUS-US9n U V V s/sH.upURRU R/5SPM0 n n n [[R "U/[RS9U UUS9n/nUHnURU U5 M U$s sn n f)aReturn docs and their similarity scores selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: embedding: Embedding to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch before filtering to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. Returns: List of Documents and similarity scores selected by maximal marginal relevance and score for each. rZrXrrdtype)rXr)rrrJrrorprrrrr>embed_documentsr_rry)rMr>rXrrrYrNrZr{indicesresultsrrrU mmr_selectedrrs r2max_marginal_relevance_search_with_score_by_vector9TileDB.max_marginal_relevance_search_with_score_by_vectorPsY:  &$jj):;O'O++11 HHbhhy)00<= > E Ebjj Q gWq[   ,, !9gWq[+ - QX PWfcDNN * *C,<,<+= >q APW  2 HHi[ 3 #  A  " "71: . s4Ec jUR"U4UUUUS.UD6nUVV s/sHupUPM sn n$s sn nf)aReturn docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: embedding: Embedding to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch before filtering to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. Returns: List of Documents selected by maximal marginal relevance. r)r) rMr>rXrrrYrNrrrs r'max_marginal_relevance_search_by_vector.TileDB.max_marginal_relevance_search_by_vectorsN4QQ  #    #22//222s /c XURU5nUR"U4UUUUS.UD6nU$)aReturn docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch before filtering (if needed) to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. Returns: List of Documents selected by maximal marginal relevance. r)r@r) rMrrXrrrYrNr>r|s rmax_marginal_relevance_search$TileDB.max_marginal_relevance_searchsI4++E2 ;;  #     rT) metadatasr3c[S5[S5pURUS9 URU5 UR US5n [ U R 5n [U R 5n US:XaURRU UUUS9 O"US:XaURRU UUUS9 U RU [S9 URS S [S - 4[R "[R"5S 9n UR%U 5nUR'S [R "S5SS9nU/nU(a0UR'S[R(SS9nUR+U5 UR-USSUS9nUR.RU U5 U RU [0S9 U R35 SSS5 g!URan U eSn A ff=f!,(df  g=f)Nrrr7wr;)r dimensions vector_typer3r<)nameidr)rdomainrr]U1T)rrvarr^F)rsparseallows_duplicatesattrs)rrC group_create TileDBErrorrDr*r r,rHcreaterKaddrDimrirorrDomainAttrrsry ArraySchemaArrayr%rE)clsrAr:rrrr3rOrerrr"r1docs_uridimdom text_attrr metadata_attrschemas rr TileDB.creates / 0  "  F  3 ##I.LLC0E3EII> .uyy9HV#$$++() +! , z)((//() +! 0 II&->I ? **:>*hhryy)C --$C rxx~4 PIKE & 288QU V  ]+''"' (F LL  & 1 IIh%9I : KKM_4 3%%   4 3s.G.GFG.G+$G&&G++G.. G<r;r)rrzrBr:r3index_timestampc PU[;a[SUS[[535e[S5[S5p[R "U5R [R5nURUUURSURUSLU S9 U RU S9 U(d [S5e[U5n[U5nUc6UVs/sH*n[[R "S ["S- 55PM, nn[R "U5R [R$5nU R&R("SUUUUU S :waU OSU S .U D6 U R+US 5nUcL[R,"[/U5[R$S 9n[1[/U55HnUUU'M 0n[R "U5US 'Ubr[R2"[/U5/[4S 9nS nUHBn[R6"[8R:"U5[R<S 9UU'US- nMD UUS'UUU'SSS5 SSS5 U"SUUUU S.U D6$s snf!,(df  N(=f!,(df  N1=f)NzUnsupported distance metric: z. Expected one of rrr)rAr:rrrr3r7z3embeddings must be provided to build a TileDB indexr)r:rA input_vectors external_idsrr3rrr]r^)r>rArBr3r) INDEX_METRICSr=rwrrorprrrrshaperrCr*r,rlrandomrandintrir ingestioningestrgzerosrkrangeemptyobject frombufferrmdumpsrs)rtextsrUr>rArrzrBr:r3rrNrOrrr1rrrArdatarr^s r__from TileDB.__froms  &3F8<''+M':&;=  / 0  ",33BJJ?  !$**1-%++t+    F  3 !VWW3I> .y9H{GLMu!s6>>!Z!^<=uM88C=// :L    & & %*+)3Ba3GT   Xs+q'#%88CJbii#HL"3u:.*+ Q/!xxV ($&HHc)n-=V$LMA$-+-=="LL2"((, a(Q %. (5D$"&,#,'4J      =N,+'4 3s8-0J0J A2J?C%J$JJ J J J%c [R"U5R[R5nURR XBS:waUOSS9 g)a-Delete by vector ID or other criteria. Args: ids: List of ids to delete. timestamp: Optional timestamp to delete with. **kwargs: Other keyword arguments that subclasses might use. Returns: Optional[bool]: True if deletion is successful, False otherwise, None if not implemented. rN)rr4T)rorprrrrJ delete_batch)rMrzr4rNrs rdelete TileDB.deleteesJxx}++BII6  &&%1nRV ' rc [S5nURR[U55nUc6UVs/sH*n[ [ R "S[S- 55PM, nn[R"U5R[R5n [R"[U5SS9n [[U55H*n [R"X{[RS9X'M, UR R#U U US:waUOSS9 0n [R"U5U S'Ubp[R"[U5/[$S9n Sn UHAn[R&"[(R*"U5[R,S9X'U S- n MC XS 'UR/UR0S US:waUOSUR2S 9nXU 'UR55 U$s snf) aRun more texts through the embeddings and add to the vectorstore. Args: texts: Iterable of strings to add to the vectorstore. metadatas: Optional list of metadatas associated with the texts. ids: Optional ids of each text object. timestamp: Optional timestamp to write new texts with. kwargs: vectorstore specific parameters Returns: List of ids from adding the texts into the vectorstore. rNrrOr)rrr4r]r^rr\)rr>rrwrlrrrirorprrrrrkrrrJ update_batchrrrmrrsrgr2r3rE)rMrrrzr4rNrrUrrrrr|rr^r}s r add_textsTileDB.add_textszs(h'^^33DK@ ;CHI5a3v~~aa895CIxx}++BII6 ((C OC8s:'A*-rzzBGJ( &&%#,>it ' xxV  HHc)n%5VDMA%#%==h1Grxx#X  Q& - [[    #,>it;; ! $(<  =Js0G1z/tmp/tiledb_arrayc  b/n URU5n UR"SUU UUUUUUUU S. U D6$)a Construct a TileDB index from raw documents. Args: texts: List of documents to index. embedding: Embedding function to use. metadatas: List of metadata dictionaries to associate with documents. ids: Optional ids of each text object. metric: Metric to use for indexing. Defaults to "euclidean". index_uri: The URI to write the TileDB arrays index_type: Optional, Vector index type ("FLAT", IVF_FLAT") config: Optional, TileDB config index_timestamp: Optional, timestamp to write new texts with. Example: .. code-block:: python from langchain_community import TileDB from langchain_community.embeddings import OpenAIEmbeddings embeddings = OpenAIEmbeddings() index = TileDB.from_texts(texts, embeddings) rrUr>rrzrBrAr:r3rr)r _TileDB__from) rrr>rrzrBrAr:r3rrNrUs r from_textsTileDB.from_textssXF ..u5 zz  !!+    rc UV s/sHoSPM n n UV s/sHoSPM n n UR"SU U UUUUUUUU S. U D6$s sn fs sn f)aConstruct TileDB index from embeddings. Args: text_embeddings: List of tuples of (text, embedding) embedding: Embedding function to use. index_uri: The URI to write the TileDB arrays metadatas: List of metadata dictionaries to associate with documents. metric: Optional, Metric to use for indexing. Defaults to "euclidean". index_type: Optional, Vector index type ("FLAT", IVF_FLAT") config: Optional, TileDB config index_timestamp: Optional, timestamp to write new texts with. Example: .. code-block:: python from langchain_community import TileDB from langchain_community.embeddings import OpenAIEmbeddings embeddings = OpenAIEmbeddings() text_embeddings = embeddings.embed_documents(texts) text_embedding_pairs = list(zip(texts, text_embeddings)) db = TileDB.from_embeddings(text_embedding_pairs, embeddings) rrrr)r)rtext_embeddingsr>rArrzrBr:r3rrNtrrUs rfrom_embeddingsTileDB.from_embeddingsswJ //!1/$34OqdO 4zz  !!+    04s A A )rBr3r4c U"SUUUUUS.UD6$)aELoad a TileDB index from a URI. Args: index_uri: The URI of the TileDB vector index. embedding: Embeddings to use when generating queries. metric: Optional, Metric to use for indexing. Defaults to "euclidean". config: Optional, TileDB config timestamp: Optional, timestamp to use for opening the arrays. )r>rArBr3r4rr)rrAr>rBr3r4rNs rload TileDB.loads/(      rc FURR"S0UD6Ulg)Nr)rJconsolidate_updates)rMrNs rrTileDB.consolidate_updates7s --AAKFKr) r3r2r>r@r:rArBr4rJr1)r>r rArlrBrlr1rlr2rlr3Optional[Mapping[str, Any]]r4rr5boolrNr)returnzOptional[Embeddings]) rzz List[int]r{ List[float]rXintrYOptional[Dict[str, Any]]rZfloatrList[Tuple[Document, float]]) r>rrXrrYrrrrNrrr) rrlrXrrYrrrrNrrr)rWNr) r>rrXrrYrrrrNrrList[Document]) rrlrXrrYrrrrNrrr)r>rrXrrrrrrYrrNrrr)rWrrN)r>rrXrrrrrrYrrNrrr)rrlrXrrrrrrYrrNrrr)rArlr:rlrrrznp.dtyperrr3rrNone)r List[str]rUzList[List[float]]r>r rArlrOptional[List[dict]]rzOptional[List[str]]rBrlr:rlr3rrrrNrrr.)Nr)rzrr4rrNrrzOptional[bool])NNr) rz Iterable[str]rrrzrr4rrNrrr)rrr>r rrrzrrBrlrArlr:rlr3rrrrNrrr.)rzList[Tuple[str, List[float]]]r>r rArlrrrzrrBrlr:rlr3rrrrNrrr.)rArlr>r rBrlr3rr4rrNrrr.)rNrrr)__name__ __module__ __qualname____firstlineno____doc__rQpropertyrUrrrrrrrrr classmethodrDEFAULT_METRICrrrrrrr__static_attributes__rrrr.r.>s- &!# .205HHH H  HH,HH*.HHT+/!*7 77  7 ) 77 &7z+/ % %   % ) %  % %  &% V+/    )    &F+/ 33 3) 3  3  3 3@+/ 33 3) 3  3  3 3: +/99  9  9  9)99 &9| +/ "3"3 "3 "3  "3 ) "3"3 "3N +/ ## # #  # ) ## #J.2=== =  ==,= ==~+/#'$ .2 N N &N  N  N (N !N N N ,N N N  N N bAB&:=MP 0+/#' 55(5! 5  5  5 5n +/#'$, .2 0 0 0 ( 0 ! 0  0 0 0 ,0 0 0  0 0 d+/#'$ .2 3 63 3  3 ( 3 !3 3 3 ,3 3 3  3 3 j %.2     ,       8Lrr.)rr)r"rrrl)r rlrrl)-r  __future__rrmrsystypingrrrrrr r numpyrolangchain_core.documentsr langchain_core.embeddingsr langchain_core.utilsrlangchain_core.vectorstoresr&langchain_community.vectorstores.utilsr frozensetrrr%riinformaxrifinforj float_inforrr#r&r*r,r.rrrr s," FFF-0-3M;-( " XXbhhx( ) - - xx+,00 NN   ( +( + z L[z Lr