tj=ddlZddlmZmZddlmZmZmZmZm Z m Z m Z m Z ddl mZddlmZddlmZddlmZddlmZmZmZmZdd lmZd ed efd Zd dd efdZe eeeedfZ GddedZ!GddedZ"GddedZ#GddedZ$GddedZ%GddedZ&GddeedZ'Gdd e'Z(Gd!d"e'Z)Gd#d$edZ*Gd%d&edZ+d'd(d)d*d+d,Z,ed-d.d/0Gd1d2eZ-dS)3N)ABCabstractmethod)AnyCallableDictListLiteralOptionalSequenceUnion) deprecated)CallbackManagerForRetrieverRun)Document) BaseRetriever) BaseModelFieldmodel_validator validator) Annotatedexcerptreturncb|s|Stjdd|dd}|S)zClean an excerpt from Kendra. Args: excerpt: The excerpt to clean. Returns: The cleaned excerpt. z\s+ z...)resubreplace)rress c/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/langchain_community/retrievers/kendra.py clean_excerptr s9  &g & & . .ub 9 9C Jitem ResultItemcd}|}|r |d|dz }t|}|r |d|dz }|S)zCombine a ResultItem title and excerpt into a single string. Args: item: the ResultItem of a Kendra search. Returns: A combined text of the title and excerpt of the given item. rzDocument Title:  zDocument Excerpt: ) get_titler get_excerpt)r"texttitlers r combined_textr*+sr D NN  E - ,5,,,,D,,..//G3 2w2222 Kr!cZeZdZUdZeed< eed< eeed< eeed<dS) Highlightz8Information that highlights the keywords in the excerpt. BeginOffset EndOffset TopAnswerTypeN) __name__ __module__ __qualname____doc__int__annotations__r boolstrr!rr,r,GsVBBLNNNJ~7 3-<selfs rget_value_textz(AdditionalResultAttribute.get_value_textqsz166r!N) r1r2r3r4r8r6r rArMr9r!rrDrDgsg&& HHH#34444 ))))%7777777r!rDceZdZUdZdZeeed< dZee ed< dZ ee eed< dZ eeed< e defdZdS) DocumentAttributeValuezValue of a document attribute.N DateValue LongValueStringListValue StringValuercv|jr|jS|jr|jS|jr|jS|jr|jSdS)zThe only defined document attribute value or None. According to Amazon Kendra, you can only provide one value for a document attribute. N)rPrQrRrSrKs rvaluezDocumentAttributeValue.valuesU > "> ! > "> !   (' '   $# #tr!)r1r2r3r4rPr r8r6rQr5rRrrSpropertyDocumentAttributeValueTyperUr9r!rrOrOvs((#Ix}###3#Ix}###+/OXd3i(/// !%K#%%% 1Xr!rOc*eZdZUdZeed< eed<dS)DocumentAttributezDocument attribute.rErHN)r1r2r3r4r8r6rOr9r!rrYrYs0 HHH# !!!!%%r!rYc.eZdZUdZeeed< eeed< eeed< gZeee ed< ee ed< e defdZ e defd Z de fd Zdeeeffd Zdefd Zefd edgefdefdZdS)r#zBase class of a result item.Id DocumentId DocumentURIDocumentAttributesScoreAttributesrcdS)zDocument title.Nr9rKs rr&zResultItem.get_titler!cdS)zDDocument excerpt or passage original content as retrieved by Kendra.Nr9rKs rr'zResultItem.get_excerptrar!ciS)zDocument additional metadata dict. This returns any extra metadata except these: * result_id * document_id * source * title * excerpt * document_attributes r9rKs rget_additional_metadataz"ResultItem.get_additional_metadatas  r!c(d|jpgDS)zDocument attributes dict.c2i|]}|j|jjSr9)rErHrU).0attrs r z;ResultItem.get_document_attributes_dict..s!WWWt$**WWWr!)r^rKs rget_document_attributes_dictz'ResultItem.get_document_attributes_dictsWW8O8USUWWWWr!c.|j |jdSdS)zDocument Score ConfidenceNScoreConfidence NOT_AVAILABLE)r_rKs rget_score_attributezResultItem.get_score_attributes   +'(9: :"?r!page_content_formatterc J||}|}||j|j|j||||dt||S)z!Converts this item to a Document.) result_id document_idsourcer)rdocument_attributesscore) page_contentmetadata) rdupdater[r\r]r&r'rjrnr)rLrorvrws rto_doczResultItem.to_docs.-d33 //11!W#*))++--'+'H'H'J'J1133   \HEEEEr!N)r1r2r3r4r r8r6r^rrYdictrr&r'rdrrWrjrnr*rrryr9r!rr#r#s~&& - #<>&7!89>>>"d^###%3^SSSSS^S     Xd38R3R.SXXXX#S####GTFF&. ~s/B&CF FFFFFFr!ceZdZUdZeed< eeed< eeed< eeed< gZee e ed< eeed< defd Z defd Z defd Z defd Zd S)QueryResultItemzQuery API result item. DocumentTitle FeedbackTokenFormatr0AdditionalAttributesDocumentExcerptrc|jjSrJ)r}r>rKs rr&zQueryResultItem.get_titles!&&r!cp|jsdS|jdsdS|jdS)Nrr)rrMrKs rget_attribute_valuez#QueryResultItem.get_attribute_valuesB( 2(+ A2,Q/>>@@ @r!c|jr+|jdjdkr|}n|jr |jj}nd}|S)Nr AnswerTextr)rrErrr>)rLrs rr'zQueryResultItem.get_excerptsX  % )!,0L@@..00GG  ! */GGGr!cd|ji}|S)Ntype)r0)rLadditional_metadatas rrdz'QueryResultItem.get_additional_metadata s%ty1""r!N)r1r2r3r4r=r6r r8rrrDr&rr'rzrdr9r!rr|r|s   %%%%C=   A SM 3-?FH(4(A#BCHHHG01111''3''''ASAAAA S    #######r!r|c\eZdZUdZeeed< eeed< defdZdefdZdS)RetrieveResultItemzRetrieve API result item.r}Contentrc|jpdSNr)r}rKs rr&zRetrieveResultItem.get_titles!'R'r!c|jpdSr)rrKs rr'zRetrieveResultItem.get_excerpts|!r!r!N) r1r2r3r4r r8r6r&r'r9r!rrrsx##C=    c]"(3(((("S""""""r!rc*eZdZUdZeeed<dS) QueryResulta`Amazon Kendra Query API` search result. It is composed of: * Relevant suggested answers: either a text excerpt or table excerpt. * Matching FAQs or questions-answer from your FAQ file. * Documents including an excerpt of each document with its title. ResultItemsN)r1r2r3r4rr|r6r9r!rrr!s/o&&&&r!rc6eZdZUdZeed< eeed<dS)RetrieveResultz`Amazon Kendra Retrieve API` search result. It is composed of: * relevant passages or text excerpts given an input query. QueryIdrN)r1r2r3r4r8r6rrr9r!rrr/s< LLL())))r!rg?g?g??)rmLOWMEDIUMHIGH VERY_HIGHz0.3.16z1.0z#langchain_aws.AmazonKendraRetriever)sinceremovalalternative_importc2eZdZUdZeed<dZeeed<dZeeed<dZ e ed<dZ ee ed<dZ eee ed <eZeegefed <eed <dZee ed <eeeed dfed<edde de fdZedede eefdefdZdedeefdZdeedeefdZ deedeefdZ!dede"deefdZ#dS) AmazonKendraRetrievera`Amazon Kendra Index` retriever. Args: index_id: Kendra index id region_name: The aws region e.g., `us-west-2`. Fallsback to AWS_DEFAULT_REGION env variable or region specified in ~/.aws/config. credentials_profile_name: The name of the profile in the ~/.aws/credentials or ~/.aws/config files, which has either access keys or role information specified. If not specified, the default credential profile or, if on an EC2 instance, credentials from IMDS will be used. top_k: No of results to return attribute_filter: Additional filtering of results based on metadata See: https://docs.aws.amazon.com/kendra/latest/APIReference document_relevance_override_configurations: Overrides relevance tuning configurations of fields/attributes set at the index level See: https://docs.aws.amazon.com/kendra/latest/APIReference page_content_formatter: generates the Document page_content allowing access to all result item attributes. By default, it uses the item's title and excerpt. client: boto3 client for Kendra user_context: Provides information about the user context See: https://docs.aws.amazon.com/kendra/latest/APIReference Example: .. code-block:: python retriever = AmazonKendraRetriever( index_id="c0806df7-e76b-4bce-9b5c-d5582f6b1a03" ) index_idN region_namecredentials_profile_nametop_kattribute_filter*document_relevance_override_configurationsroclient user_contextrr)gelemin_score_confidencerUrc8|dkrtd|d|S)Nrtop_k () cannot be negative.) ValueError)clsrUs rvalidate_top_kz$AmazonKendraRetriever.validate_top_ks* 199CuCCCDD D r!before)modevaluesc|d}||dkrtd|d|d|S ddl}|dr||d}n|}i}|dr |d|d<|jd i||d<|S#t $rt d t $r}td |d}~wwxYw) Nrrrrrr) profile_namerkendrazRCould not import boto3 python package. Please install it with `pip install boto3`.zCould not load credentials to authenticate with AWS client. Please check that credentials in the specified profile name are valid.)r)getrboto3Sessionr ImportError Exception)rrrrsession client_paramses r create_clientz#AmazonKendraRetriever.create_clientsQ 7##  CuCCCDD D ::h   +M  LLLzz455 *--V    *   s A=C"C>)C99C>queryc|j|dd|jd}|j |j|d<|j |j|d<|j |j|d<|jjdi|}t |}|j r|j S|jj di|}t |}|j S)Nri)IndexId QueryTextPageSizeAttributeFilter'DocumentRelevanceOverrideConfigurations UserContextr9) rstriprrrrrretriever parse_objrrr)rLr kendra_kwargsresponser_resultq_results r _kendra_queryz#AmazonKendraRetriever._kendra_querys}qu-     ,/3/DM+ ,  : F? C D   (+/+fd|djD}|S)NcDg|]}|jSr9)ryrorgr"rLs r z9AmazonKendraRetriever._get_top_k_docs..s8    KK3 4 4   r!)r)rLrtop_docss` r_get_top_k_docsz%AmazonKendraRetriever._get_top_k_docss=    $\tz\2   r!docsc6js|Sfd|D}|S)zr Filter out the records that have a score confidence greater than the required threshold. cg|]m}|jdQt|jdtr1t|jddjkk|nS)ruNr)rwr isinstancer8KENDRA_CONFIDENCE_MAPPINGrrs rrzEAmazonKendraRetriever._filter_by_score_confidence..s     !!'**6t}W5s;;7-11$-2H#NN,--  ---r!)r)rLr filtered_docss` r_filter_by_score_confidencez1AmazonKendraRetriever._filter_by_score_confidencesE ( K         r! run_managerc||}||}||S)zRun search on Kendra index and get top k documents Example: .. code-block:: python docs = retriever.invoke('This is my query') )rrr)rLrrr top_k_docss r_get_relevant_documentsz-AmazonKendraRetriever._get_relevant_documentss>))%00 )),77 // ;;;r!)$r1r2r3r4r8r6rr rrr5rrrrr*rorr#rrrfloatrrrr classmethodrr rrrrrrr9r!rrrEs* ''RMMM!%K#%%%.2hsm222E3NNN'+htn+++GK.d0DKKK:GHj\3%67GGG KKK#'L(4.'''#HUOUUcc5J5J5J$JKKKKYw33 _(###"4S>"c"""[$#"H$3$8J+?$$$$6HZ,@T(^X4>&<<4 < h <<<<<rs& ########                    766666CCCCCC------333333 (''''' 3 3      (#3T#Y#<= = = = = =  = = = ='*****Yg**** 7 7 7 7 7  7 7 7 7Yg>&&&&& &&&&>F>F>F>F>FCw>F>F>F>FB.#.#.#.#.#j.#.#.#b " " " " " " " "     )7         Yg          < f<f<f<f<f