j=ddlmZddlZddlZddlmZmZmZmZm Z m Z m Z m Z m Z ddlmZmZmZmZmZmZmZddlmZmZmZ ddlmZddlmZddl m!Z!dd l"m#Z#dd l$m%Z%d Z&d Z'eZ(Gd de%Z)y#e$r ddlmZY?  'OLL17# "# "s%E",BEE"E E""E+c\|jr|j|jyr )r%delete_collectionrTr6s r-r3zAnalyticDB.create_collections#  % %  " " $ '')r/c6|jjdtd|jd}|jj 5}|j 5|j|ddddddy#1swYxYw#1swYyxYw)NzTrying to delete collectionzDROP TABLE IF EXISTS ;)r)debugrr$r2rJrKrM)r+drop_statementrOs r-rVzAnalyticDB.delete_collectionss 78 5d6J6J5K1MN [[ "d ^,# "# "s$B B2BB BBc |*|Dcgc]}ttj!}}|jj t |}|s|Dcgc]}i}}t |jtjtdtdtdtttdtdtdtdd }g} |j j#5} | j%5t'||||D]e\} } } }| j)| || | d t+| |k(s-| j-t/|j1| | j3g| r)| j-t/|j1| dddddd|Scc}wcc}w#1swYxYw#1swY|SxYw) arRun more texts through the embeddings and add to the vectorstore. Args: texts: Iterable of strings to add to the vectorstore. metadatas: Optional list of metadatas associated with the texts. kwargs: vectorstore specific parameters Returns: List of ids from adding the texts into the vectorstore. Nr<Tr=r?r@rArCrD)r<r?r@rC)strrHrIr"embed_documentslistrr$rGrCrrrr rrr2rJrKzipappendlenrMrvaluesclear)r+texts metadatasids batch_sizekwargs_r7 chunks_tablechunks_table_datarOr@rCchunk_idr?s r- add_textszAnalyticDB.add_textss$ ;.34e3tzz|$eC4,,<"G >G G  Gc b|jj|}|j|||S)a`Run similarity search with AnalyticDB with distance. Args: query (str): Query text to search for. k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar to the query. )rr?kfilter)r" embed_querysimilarity_search_by_vector)r+queryrqrrrir?s r-similarity_searchzAnalyticDB.similarity_searchs>"++77U7C //0  r/cd|jj|}|j|||}|S)abReturn docs most similar to query. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar to the query and score for each rp)r"rs&similarity_search_with_score_by_vector)r+rurqrrr?docss r-similarity_search_with_scorez'AnalyticDB.similarity_search_with_scores= ++77> ::1V;  r/cJ ddlm}d}|<|jDcgc] \}}d|d|}}}ddj |}d |j d |d } ||d } |j j5} | jt| | j} ddd D cgc]=} t| j| j |j | jndf?}} |S#t$r tdwxYwcc}}w#1swYqxYwcc} w)Nr)RowzRCould not import Row from sqlalchemy.engine. Please 'pip install sqlalchemy>=1.4'.z metadata->>z = zWHERE z AND zW SELECT *, l2_distance(embedding, :embedding) as distance FROM z zL ORDER BY embedding <-> :embedding LIMIT :k )r?rq) page_contentrC)sqlalchemy.enginer| ImportErroritemsjoinr$r2rJrMrfetchallrr@rCr"distance)r+r?rqrrr|filter_conditionkeyvalue conditions sql_queryparamsrOresultsrRdocuments_with_scoress r-rxz1AnalyticDB.similarity_search_with_score_by_vectors_  -  DJLLNDRjc5+cWCy1N "( Z(@'AB &&'(     )q1[[ "d%)\\$y/6%J%S%S%UG#" ! " !'#__$(#:#:#FD  "  ! %$O 8  $# " ! s#C6D:*D0AD 6D Dc `|j|||}|Dcgc]\}}| c}}Scc}}w)akReturn docs most similar to embedding vector. Args: embedding: Embedding to look up documents similar to. k: Number of Documents to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar to the query vector. rp)rx)r+r?rqrrridocs_and_scoresdocrjs r-rtz&AnalyticDB.similarity_search_by_vector-s?"EE1VF #22/Q/222s *c | tdt|jtjt dt dt dttt dtdt d tdd } |jj5}|j5|jjj!|}|j#|j%j'| ddddddy#1swYnxYw dddy#1swYyxYw#t($r}t+d t-|Yd}~y d}~wwxYw) zUDelete by vector IDs. Args: ids: List of ids to delete. NzNo ids provided to delete.r<Tr\r?r@rArCrDzDelete operation failed:F) ValueErrorrr$rGrCrrrr rrr2rJrKcr<in_rMdeletewhere Exceptionprintr])r+rgrirkrOdelete_conditiones r-rzAnalyticDB.deleteCs  ;9: :  MM 44 0 ;d , :v 5 :td 3   $$&$ZZ\'3~~'8'8'<'>(>! >  >  > >F!%           6!%    & 2!% 0%0% 0% 0% & 0%j!% 33 3 3  3  3,< +/#CA#'&+&* (  !   ! $$ B!!  $DA#'&+&*" " !" " ! "  " ! "  $" $" "  " " H S S S S  S  S S  S Sr/r)* __future__rr&rHtypingrrrrrr r r r sqlalchemyr rrrrrrsqlalchemy.dialects.postgresqlrrrsqlalchemy.ormrrsqlalchemy.ext.declarativelangchain_core.documentsrlangchain_core.embeddingsrlangchain_core.utilsrlangchain_core.vectorstoresrrrrGrrr/r-rst" WWWOOO<<</.053#' %9"jSjS<;