tj3dZddlZddlZddlmZddlmZmZmZm Z m Z m Z m Z m Z mZddlmZddlmZddlmZddlmZdd lmZdd lmZe d eee ef Ze d eee e ee f ZGddZGddeeefZ Gdde eefZ!edddGddeZ"dS)zSagemaker InvokeEndpoint API.N)abstractmethod) AnyDictGenericIteratorListMappingOptionalTypeVarUnion) deprecated)CallbackManagerForLLMRun)LLM)pre_init) ConfigDict)enforce_stop_tokens INPUT_TYPE)bound OUTPUT_TYPEc6eZdZdZdeddfdZddZdefdZdS) LineIteratoraParse the byte stream input. The output of the model will be in the following format: b'{"outputs": [" a"]} ' b'{"outputs": [" challenging"]} ' b'{"outputs": [" problem"]} ' ... While usually each PayloadPart event from the event stream will contain a byte array with a full json, this is not guaranteed and some of the json objects may be split acrossPayloadPart events. For example: {'PayloadPart': {'Bytes': b'{"outputs": '}} {'PayloadPart': {'Bytes': b'[" problem"]} '}} This class accounts for this by concatenating bytes written via the 'write' function and then exposing a method which will return lines (ending with a ' ' character) within the buffer via the 'scan_lines' function. It maintains the position of the last read position to ensure that previous bytes are not exposed again. For more details see: https://aws.amazon.com/blogs/machine-learning/elevating-the-generative-ai-experience-introducing-streaming-support-in-amazon-sagemaker-hosting/ streamreturnNclt||_tj|_d|_dS)Nr)iter byte_iteratorioBytesIObufferread_pos)selfrs i/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/langchain_community/llms/sagemaker_endpoint.py__init__zLineIterator.__init__2s(!&\\jll  c|S)Nr!s r"__iter__zLineIterator.__iter__7s r$c8 |j|j|j}|r@|dt dkr'|xjt |z c_|ddS t |j}n8#t$r+|j|j j krYĂwxYwd|vr|jdtj |j |dd)NT PayloadPartrBytes)rseekr readlineordlennextr StopIteration getbuffernbytesrSEEK_ENDwrite)r!linechunks r"__next__zLineIterator.__next__:s = K  T] + + +;''))D !RCII-- T* CRCy  T/00    =4;#8#8#:#:#AAAH E)) K  Q , , , K  eM27; < < rAr str__annotations__rBrrrbytesrFrrJr&r$r"r@r@Ns"#/L(3-...<)GXc])))C j     ^  u    ^   r$r@ceZdZdZdS)LLMContentHandlerzContent handler for LLM class.N)r;r<r=r>r&r$r"rOrO|s((((r$rOz0.3.16z1.0z$langchain_aws.llms.SagemakerEndpoint)sinceremovalalternative_importc ~eZdZUdZ dZeed< dZeed< dZ eed< dZ e eed< e ed< d Z eed < dZe eed < dZe eed < ed ZededefdZedeeeffdZedefdZ ddede eede ededef dZdS)SagemakerEndpointaSagemaker Inference Endpoint models. To use, you must supply the endpoint name from your deployed Sagemaker model & the region where it is deployed. To authenticate, the AWS client uses the following methods to automatically load credentials: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html If a specific credential profile should be used, you must pass the name of the profile from the ~/.aws/credentials file that is to be used. Make sure the credentials / roles used have the required policies to access the Sagemaker endpoint. See: https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html Nclient endpoint_name region_namecredentials_profile_namecontent_handlerF streamingrDendpoint_kwargsforbid)extravaluesrcv|d|S ddl} |d||d}n|}|d|d|d<n"#t$r}t d |d}~wwxYwn#t $rt d wxYw|S) z.Dont do anything if client provided externallyrUNrrY) profile_namezsagemaker-runtimerX)rXzCould not load credentials to authenticate with AWS client. Please check that credentials in the specified profile name are valid.zRCould not import boto3 python package. Please install it with `pip install boto3`.)getboto3SessionrU Exception ValueError ImportError)clsr_rcsessiones r"validate_environmentz&SagemakerEndpoint.validate_environments ::h   +MX  LLL 45A#mm%+,F%G,GG $mmooG#*>>'VM5J$2$$x      .   !   >    s/BAA98B9 BBBBB6c2|jpi}id|jid|iS)zGet the identifying parameters.rWrD)rDrW)r! _model_kwargss r"_identifying_paramsz%SagemakerEndpoint._identifying_params's8)/R   23 }-  r$cdS)zReturn type of llm.sagemaker_endpointr&r's r" _llm_typezSagemakerEndpoint._llm_type0s $#r$rCstop run_managerkwargsc |jpi}i||}|jpi}|j||}|jj}|jj} |jr|r |jjd |j ||jjd|} t| d} d} | D]]} tj | } | dd}|t||}| |z } ||^| S#t $r}t#d|d}~wwxYw |jjd |j ||| d|}n$#t $r}t#d |d}~wwxYw|j|d}|t||}|S) afCall out to Sagemaker inference endpoint. Args: prompt: The prompt to pass into the model. stop: Optional list of stop words to use when generating. Returns: The string generated by the model. Example: .. code-block:: python response = se("Tell me a joke.") ) EndpointNameBody ContentTyperwrVoutputsrNz.Error raised by streaming inference endpoint: )rvrwrxAcceptz$Error raised by inference endpoint: r&)rDr\rZrFrArBr[rU$invoke_endpoint_with_response_streamrWrjsonloadsrbron_llm_new_tokenrerfinvoke_endpointrJ)r!rCrrrsrtrm_endpoint_kwargsbodyrArBrespiteratorcurrent_completionr8 resp_outputrjresponsetexts r"_callzSagemakerEndpoint._call5s**)/R 3=3F3 /52#33FMJJ+8 &. >' k'  WGt{G!%!3 $ 4 A'  (V 55*,"$>>D:d++D"&((9"5"5a"8K'&9+t&L&L &+5&00====)) W W W !URS!U!UVVV W M64;6!%!3 ,"  '  M M M !K!K!KLLL M'88&9IJJD+466Ks1BC44 D>DDD66 EEE)NN)r;r<r=r>rUrrLrWrKrXrYr rOr[boolrDrr\r model_configrrkpropertyr rnrqrrrr&r$r"rTrTs "/`FC,M3,KP.2hsm222'&&& It(&$(L(4.'''1&*OXd^*** :L"$"4"""X"H WS#X%6   X $3$$$X$%):> DDDtCy!D67 D  D DDDDDDr$rT)#r>rr|abcrtypingrrrrrr r r r langchain_core._api.deprecationr langchain_core.callbacksr#langchain_core.language_models.llmsrlangchain_core.utilsrpydanticrlangchain_community.llms.utilsrrKrfloatrrr@rOrTr&r$r"rs ## XXXXXXXXXXXXXXXXXXXXXX666666======333333))))))>>>>>> W\sDI~)> ? ? ? gm5d4;6G1Q+RSSS 7=7=7=7=7=7=7=7=t+ + + + + [!89+ + + \)))))*384)))   = ttttttt  tttr$