dhYSSKJr SSKrSSKrSSKJr SSKJr SSKJ r J r J r J r J r Jr SSKJr SSKJr SSKJr SS KJrJr \R0"5rSS jr"S S \5r"S S\5r"SS\5rg)) annotationsN)sha1)Thread)AnyDictIterableListOptionalTuple)Document) Embeddings) VectorStore) BaseSettingsSettingsConfigDictc$UH nX ;dM g g)z Check if a string contains multiple substrings. Args: s: string to check. *args: substrings to check. Returns: True if all substrings are in the string, False otherwise. FT)sargsas `/var/www/html/shao/venv/lib/python3.13/site-packages/langchain_community/vectorstores/myscale.pyhas_mul_sub_strrs : c\rSrSr%SrSrS\S'SrS\S'S rS \S 'S r S \S 'S r S\S'S r S\S'SSSSS.r S\S'Sr S\S'SrS\S'SrS\S'S%Sjr\"SS S!S"S#9rS$rg )&MyScaleSettings!aJMyScale client configuration. Attribute: myscale_host (str) : An URL to connect to MyScale backend. Defaults to 'localhost'. myscale_port (int) : URL port to connect with HTTP. Defaults to 8443. username (str) : Username to login. Defaults to None. password (str) : Password to login. Defaults to None. index_type (str): index type string. index_param (dict): index build parameter. database (str) : Database name to find the table. Defaults to 'default'. table (str) : Table name to operate on. Defaults to 'vector_table'. metric (str) : Metric to compute distance, supported are ('L2', 'Cosine', 'IP'). Defaults to 'Cosine'. column_map (Dict) : Column type map to project column name onto langchain semantics. Must have keys: `text`, `id`, `vector`, must be same size to number of columns. For example: .. code-block:: python { 'id': 'text_id', 'vector': 'text_embedding', 'text': 'text_plain', 'metadata': 'metadata_dictionary_in_json', } Defaults to identity map. localhoststrhosti intportN Optional[str]usernamepasswordMSTG index_typezOptional[Dict[str, str]] index_paramidtextvectormetadata)r'r(r)r*zDict[str, str] column_mapdefaultdatabase langchaintableCosinemetricc[X5$N)getattr)selfitems r __getitem__MyScaleSettings.__getitem__Us t""rz.envutf-8myscale_ignore)env_fileenv_file_encoding env_prefixextrar)r6rreturnr)__name__ __module__ __qualname____firstlineno____doc__r__annotations__r r"r#r%r&r+r-r/r1r7r model_config__static_attributes__rrrrr!s>D#D#"Hm""Hm"J,0K)0 "JHcE3FC#&! Lrrc^\rSrSrSrSSU4Sjjjr\SSj5rSSjrSSjr SSjr SSSjjr \ SSS jj5r SS jrSSS jjrSS S jjrSS!S jjrSS"SjjrS#SjrS$S%Sjjr\SSj5rSrU=r$)&MyScale`aq`MyScale` vector store. You need a `clickhouse-connect` python package, and a valid account to connect to MyScale. MyScale can not only search with simple vector indexes. It also supports a complex query with multiple conditions, constraints and even sub-queries. For more information, please visit [myscale official site](https://docs.myscale.com/en/overview/) c $ >SSKJn SSKJn XPl[ T U]5 UbX lO[5UlUR(deURR(aURR(deURR(aQURR(a6URR(aURR(deSHnX`RR;aMe URRR5S;deURRS ;a[ R#S 5 [%UR'S 55nURR((aQS S R+URR(R-5VVs/sHuphSUSUS3PM snn5-OSn SURRSURRSURRSSURRSSURRSSURRSSURRSSUSURRSSURR.SURRSU SURRSS 3n XplS!UlS"UlXlURRR5S#;aS$OS%UlU"S+URRURRURR:URR<S&.UD6UlUR>RAS'5 UR>RAS*5 UR>RAU 5 g![a [S5ef=f![a SUlGN.f=fs snnf![Ba7n [ RES(UR>RFS)35 Sn A NSn A ff=f),zMyScale Wrapper to LangChain embedding (Embeddings): config (MyScaleSettings): Configuration to MyScale Client Other keyword arguments will pass into [clickhouse-connect](https://docs.myscale.com/) r) get_clientzlCould not import clickhouse connect python package. Please install it with `pip install clickhouse-connect`.)tqdmcU$r3r)xs r"MyScale.__init__..s1rN)r'r)r(r*)IPCOSINEL2)ipcosinel2z_Lower case metric types will be deprecated the future. Please use one of ('IP', 'Cosine', 'L2')z try this out, ,'=z( CREATE TABLE IF NOT EXISTS .z( r'z String, r(r)z! Array(Float32), r*zP JSON, CONSTRAINT cons_vec_len CHECK length( z) = z$, VECTOR INDEX vidx z TYPE z&( 'metric_type=z,) ) ENGINE = MergeTree ORDER BY z \)r_r[)rTrUASCDESC)rr r"r#z"SET allow_experimental_json_type=1zClickhouse version=z6 - There is no allow_experimental_json_type parameter.z$SET allow_experimental_object_type=1r)$clickhouse_connectrM ImportErrorrNpgbarsuper__init__configrrr r+r-r/r1upperloggerwarninglen embed_queryr&joinitemsr%dimBS must_escape _embeddings dist_orderr"r#clientcommand Exceptiondebugserver_version) r5 embeddingrgkwargsrMrNkrov index_paramsschema__ __class__s rrfMyScale.__init__ns  5  % !J    K)+DK{{{{{DKK$4$444 KK " " $$ !! ""   # 6A ... ..6{{!!'')-CCCC ;;  !7 7 NNG  )''78{{&& 388dkk6M6M6S6S6UV6Udaq1QCq\6UVW W  ((, (<(<'=Qt{{?P?P>QR''-./''/01''123'' 345[[++H56d3%@##';;#9#9(#C"DE++0012&&*kk&8&8%9<.I++/;;+A+A$+G*HI   &$[[''--/3CCE  ! !!!![[))[[))       KK   D E BC G$_ K   %$DJ %:WF  LL%dkk&@&@%ABFF   s:P P/Q QP,/QQ R-R  RcUR$r3)rrr5s r embeddingsMyScale.embeddingssrc:^SRU4SjU55$)Nr]c3d># UH&oTR;aTRU3OUv M( g7fr3)rqrp).0cr5s r %MyScale.escape_str..s/VPU1t/?/?*?$''1#QFPUs-0)rm)r5values` r escape_strMyScale.escape_strswwVPUVVVrc xSRU5n/nUHUnSRUVs/sH!nSUR[U55S3PM# sn5nURSUS35 MW SURR SURR SUSSRU5S3 nU$s snf) NrZr[()z8 INSERT INTO TABLE r^z)) VALUES z )rmrrappendrgr-r/)r5transac column_namesks_datan_ni_strs r _build_istrMyScale._build_istrs XXl #AAFAbAdooc"g67q9AFGA LL1QCq "[[))*!DKK,=,=+>atD%!"   Gs'B7 c\URX5nURRU5 gr3)rrtru)r5rr_i_strs r_insertMyScale._inserts$!!'8 F#rc U=(d8 UVs/sH+n[URS55R5PM- snnURRn/nUSUUSUUS[ UR RU50n U=(d UV s/sHn 0PM sn n[ [RU5XS'[[U5[U 5- 5S:de[U R56upSnUR[U 6S[U5S 9Hn [XRURRS55UR :XdeUR#U 5 [U5U:XdMhU(aUR%5 ['UR(X/S 9nUR+5 /nM [U5S:a(U(aUR%5 UR)X5 UVs/sHoPM sn$s snfs sn fs snf![,a:n[.R1S [3U5S [5U5S 35 /sSnA$SnAff=f)a_Run more texts through the embeddings and add to the vectorstore. Args: texts: Iterable of strings to add to the vectorstore. ids: Optional list of ids to associate with the texts. batch_size: Batch size of insertion metadata: Optional column data to be inserted Returns: List of ids from adding the texts into the vectorstore. r9r'r(r)r*rNzInserting data...)desctotal)targetr   )rencode hexdigestrgr+maprrrljsondumpsrksetziprnrdindexrorrmrrstartrvrierrortyper)r5texts metadatas batch_sizeidsrztcolmap_rrrkeysvaluesr|ies r add_textsMyScale.add_textss*I5I5ad188G,-7795I++(( DM3 FOU H s4#3#3#?#?G 4e!4e"e!4 ,/ I,F Z()3w<#l"334999L..01  AZZV #6c)n 1ZZ (>(>x(HIJKtxxWWWq!w<:-dll'IAGGI G7|aFFH W+"#s!As# #=J"5*$  LL?47)3CCF87S TI sC1H HBH% A=H% H H% H%% I)//I$I)$I)c 8U"X$40UD6nURXXcS9 U$)aCreate Myscale wrapper with existing texts Args: texts (Iterable[str]): List or tuple of strings to be added embedding (Embeddings): Function to extract text embedding config (MyScaleSettings, Optional): Myscale configuration text_ids (Optional[Iterable], optional): IDs for the texts. Defaults to None. batch_size (int, optional): Batchsize when transmitting data to MyScale. Defaults to 32. metadata (List[dict], optional): metadata to texts. Defaults to None. Other keyword arguments will pass into [clickhouse-connect](https://clickhouse.com/docs/en/integrations/python#clickhouse-connect-driver-api) Returns: MyScale Index )rrr)r) clsrryrrgtext_idsrrzctxs r from_textsMyScale.from_textss(6).v. ej V rcSURRSURRS3nXRRSURRS3- nUSURR S3- nUS- nUR RS URRSURR35R5HnUS US S S USS S3- nM US- nU$)zText representation for myscale, prints backends, username and schemas. Easy to use with `str(Myscale())` Returns: repr: string to show connection info and data schema z r^z @ :z zusername: z Table Schema: z4--------------------------------------------------- zDESC z|name24sz |rz| ) rgr-r/rr r"rtquery named_results)r5_reprrs r__repr__MyScale.__repr__>s "$++"6"6!7q9J9J8K3O KK$$%Qt{{'7'7&8 DD $T[[%9%9$::TUU  ""DKK(()4;;+<+<*= > -/A AfIc?*:1V9S/T E   rcSR[[U55nU(aSU3nOSnSURRSSURRSSURR S URR S US URRS S USURSUS 3nU$)NrZ PREWHERE r] SELECT r(z, r*z, dist FROM r^  ORDER BY distance(r), []) AS dist  LIMIT )rmrrrgr+r-r/rsr5q_embtopk where_str q_emb_strq_strs r _build_qstrMyScale._build_qstrRsHHSe_- #I;/IIKK**6234'' 345++&&'q):):(;< K#{{55h?@I;O)*&  rc \UR"URRU5X#40UD6$)aIPerform a similarity search with MyScale Args: query (str): query string k (int, optional): Top K neighbors to retrieve. Defaults to 4. where_str (Optional[str], optional): where condition string. Defaults to None. NOTE: Please do not let end-user to fill this and always be aware of SQL injection. When dealing with metadatas, remember to use `{self.metadata_column}.attribute` instead of `attribute` alone. The default name for it is `metadata`. Returns: List[Document]: List of Documents )similarity_search_by_vectorrrrl)r5rr{rrzs rsimilarity_searchMyScale.similarity_searchfs5&//    ( ( / AG  rc URXU5nURRU5R5Vs/sH?n[ X`R R SX`R R SS9PMA sn$s snf![a:n[RS[U5S[U5S35 /sSnA$SnAff=f)aPerform a similarity search with MyScale by vectors Args: query (str): query string k (int, optional): Top K neighbors to retrieve. Defaults to 4. where_str (Optional[str], optional): where condition string. Defaults to None. NOTE: Please do not let end-user to fill this and always be aware of SQL injection. When dealing with metadatas, remember to use `{self.metadata_column}.attribute` instead of `attribute` alone. The default name for it is `metadata`. Returns: List[Document]: List of (Document, similarity) r(r* page_contentr*rrrN) rrtrrr rgr+rvrirrrr5ryr{rrzrrrs rr#MyScale.similarity_search_by_vector}s.  y9  **51??A  BA !";;#9#9&#A!B{{55jABB    LL?47)3CCF87S TI s/+B ABB B C/C C Cc URURRU5X#5nURR U5R 5Vs/sHDn[ X`RRSX`RRSS9US4PMF sn$s snf![a:n[RS[U5S[U5S35 /sSnA$SnAff=f) Perform a similarity search with MyScale Args: query (str): query string k (int, optional): Top K neighbors to retrieve. Defaults to 4. where_str (Optional[str], optional): where condition string. Defaults to None. NOTE: Please do not let end-user to fill this and always be aware of SQL injection. When dealing with metadatas, remember to use `{self.metadata_column}.attribute` instead of `attribute` alone. The default name for it is `metadata`. Returns: List[Document]: List of documents most similar to the query text and cosine distance in float for each. Lower score represents more similarity. r(r*rdistrrrN)rrrrlrtrrr rgr+rvrirrrr5rr{rrzrrrs r'similarity_search_with_relevance_scores/MyScale.similarity_search_with_relevance_scoress*  !1!1!=!=e!DaS **51??A BA %&{{'='=f'E%F!";;#9#9*#E!FfI B     LL?47)3CCF87S TI s0+B*A B%"B*%B** C.4/C)#C.)C.cURRSURRSURR35 g)z Helper function: Drop data zDROP TABLE IF EXISTS r^N)rtrurgr-r/rs rdrop MyScale.drops< #DKK$8$8#94;;;L;L:M N rc zUc UcS5e/nU(ab[U5S:aSSRUVs/sH nSUS3PM sn5nURURRSSUS35 U(aURU5 [U5S:deS RU5nS URR S URR S U3nURRU5 g s snf![a(n [R[U 55 Sn A gSn A ff=f)zDelete by vector ID or other criteria. Args: ids: List of ids to delete. **kwargs: Other keyword arguments that subclasses might use. Returns: Optional[bool]: True if deletion is successful, False otherwise, None if not implemented. NzIYou need to specify where to be deleted! Either with `ids` or `where_str`rrYr[r'z IN (rz AND z DELETE FROM r^z WHERE TF) rkrmrrgr+r-r/rtrurvrirr) r5rrrzcondsr'id_listwhere_str_finalqstrrs rdeleteMyScale.deletes* KI$5 W 6 3s8a<iiS 9Sr1RDS 9:G LLDKK22489wiqI J  LL #5zA~~!,,u-4;;//0$++2C2C1DE$% '   KK   %!:  LLQ  sD'D D:D55D:c4URRS$)Nr*)rgr+rs rmetadata_columnMyScale.metadata_columns{{%%j11r)rprrrtrgrorsrqrdr3)ryr rgOptional[MyScaleSettings]rzrr@None)r@r )rrr@r)rrr Iterable[str]r@r)rrrrr@r)N N) rrrzOptional[List[dict]]rrrOptional[Iterable[str]]rzrr@ List[str])NNNr)rrryr rzOptional[List[Dict[Any, Any]]]rgrrrrrrzrr@rJr@rr List[float]rrrr!r@rN) rrr{rrr!rzrr@List[Document] ryrr{rrr!rzrr@r rrr{rrr!rzrr@zList[Tuple[Document, float]])r@r)NN)rzOptional[List[str]]rr!rzrr@zOptional[bool])rArBrCrDrErfpropertyrrrrr classmethodrrrrrrrrrrH __classcell__rs@rrJrJ`sH  -1^%^%*^% ^%  ^%^%@  W $+/'+ 66(6 6 % 6  6 6p 59,0,02  *  *  <*IM (+8E *BF    1> QT  4#' "" "! "  "  "JBF## #1>#QT# %#J $(#'$ $!$ $  $L22rrJc^\rSrSrSrS/4S U4SjjjrS S SjjrS SSjjrS SSjjr\ SSj5r S r U=r $)MyScaleWithoutJSONizkMyScale vector store without metadata column This is super handy if you are working to a SQL-native table Nc 4>[TU]"X40UD6 X0lg)a7Building a myscale vector store without metadata column embedding (Embeddings): embedding model config (MyScaleSettings): Configuration to MyScale Client must_have_cols (List[str]): column names to be included in query Other keyword arguments will pass into [clickhouse-connect](https://docs.myscale.com/) N)rerfmust_have_cols)r5ryrgrrzrs rrfMyScaleWithoutJSON.__init__s 5f5)7rcSR[[U55nU(aSU3nOSnSURRSSSRUR 5SURR SURRS US URRS S US URSUS 3nU$)NrZrr]rr(z, dist, z FROM r^rrr)rrr) rmrrrgr+rr-r/rsrs rrMyScaleWithoutJSON._build_qstrsHHSe_- #I;/IIKK**6234$--./0++&&'q):):(;< K#{{55h?@I;O)*&  rc URXU5nURRU5R5VVs/sHBn[ X`R R SURVs0sHo"Xb_M snS9PMD snn$s snfs snnf![a:n[RS[U5S[U5S35 /sSnA$SnAff=f)rr(rrrrN) rrtrrr rgr+rrvrirrrrs rr.MyScaleWithoutJSON.similarity_search_by_vector%s.  y9  **51??A  BA !";;#9#9&#A!B/3/B/BC/B!g/BCB  D  LL?47)3CCF87S TI sA,B2B2 B ? BB BB C!/CCCc URURRU5X#5nURR U5R 5VVs/sHGn[ X`RRSURVs0sHo"Xb_M snS9US4PMI snn$s snfs snnf![a:n[RS[U5S[U5S35 /sSnA$SnAff=f)rr(rrrrrN)rrrrlrtrrr rgr+rrvrirrrrs rr:MyScaleWithoutJSON.similarity_search_with_relevance_scoresIs*  !1!1!=!=e!DaS **51??A BA %&{{'='=f'E%F373F3F!G3FaQT'3F!GfI B  "H   LL?47)3CCF87S TI sA,B52B/ B*B/&B5*B//B55 C9?/C4.C94C9cg)Nr]rrs rr"MyScaleWithoutJSON.metadata_columnnsr)r) ryr rgrrrrzrr@rr3rrrrr) rArBrCrDrErfrrrr rrHr r s@rrrs-1$& 88*8" 8  8  88&IM (+8E .#' "" "! "  "  "JBF## #1>#QT# %#Jrr)rrrrr@bool) __future__rrlogginghashlibr threadingrtypingrrrr r r langchain_core.documentsr langchain_core.embeddingsr langchain_core.vectorstoresrpydantic_settingsrr getLoggerrirrrJrrrrr%sc" ==-03>      <l<~V2kV2r wwr