dhpSSKrSSKJr SSKJrJrJrJrJrJ r J r J r J r J r JrJr SSKrSSKJr SSKJr SSKJr SSKJrJr SSKJrJrJrJrJrJ r J!r!J"r" SS K#J$r$J%r% SS K&J'r' SS K(J)r)J*r*J+r+J,r, SS K-J.r.J/r/J0r0 SS K1J2r2J3r3J4r4 SSK5J6r6 SSK7J8r8J9r9 SSK:J;r; SSKJ?r?J@r@JArA SSKJBrB S\S\\C\44SjrDS\\S\\\C\44SjrES\S\F4SjrG\"SSSS9"SS \55rH\"SSS!S9"S"S#\55rIg)$N) itemgetter) AnyCallableDictIteratorListLiteralOptionalSequenceTupleTypeUnioncast) deprecated)CallbackManagerForLLMRun)LanguageModelInput) BaseChatModelgenerate_from_stream) AIMessageAIMessageChunk BaseMessageBaseMessageChunk ChatMessage HumanMessage SystemMessage ToolMessage)JsonOutputParserPydanticOutputParser)OutputParserLike)JsonOutputKeyToolsParserPydanticToolsParsermake_invalid_tool_callparse_tool_call)ChatGenerationChatGenerationChunk ChatResult)Runnable RunnableMapRunnablePassthrough)BaseTool)convert_to_secret_strget_from_dict_or_env)convert_to_openai_tool)is_basemodel_subclass) BaseModelField SecretStr)Responsemessagereturnc(0n[U[5(aURURS.nU$[U[5(aSURS.nU$[U[ 5(aSURS.nU$[U[ 5(aASURS.nSUR;a URSUS'USS:XaSUS'U$[U[5(aS URURS .nU$[S U35e) z convert a BaseMessage to a dictionary with Role / content Args: message: BaseMessage Returns: messages_dict: role / content dict )rolecontentsystemuser assistant tool_callsr7Ntool)r6r7 tool_call_idGot unknown type ) isinstancerr6r7rrradditional_kwargsrr> TypeError)r3 message_dicts a/var/www/html/shao/venv/lib/python3.13/site-packages/langchain_community/chat_models/sambanova.py_convert_message_to_dictrE;s$&L';'' ' I & % G] + + (W__E " ! G\ * * &7??C   GY ' ' +H 744 4)0)B)B<)PL &I&",*. Y'  G[ ) )#00  +G9566messagescDUVs/sHn[U5PM nnU$s snf)z Convert a list of BaseMessages to a list of dictionaries with Role / content Args: messages: list of BaseMessages Returns: messages_dicts: list of role / content dicts )rE)rGm message_dictss rD_create_message_dictsrK]s+;CC(Q-a0(MC DsobjcF[U[5=(a [U5$N)r@typer.)rLs rD_is_pydantic_classrPks c4 ?%:3%??rFz0.3.16z1.0z&langchain_sambanova.ChatSambaNovaCloud)sinceremovalalternative_importc^\rSrSr%Sr\"SS9r\\S'\"\ "S5S9r \ \S'\"SS9r \\S'\"S S9r \ \S '\"S S9r\\S '\"S S9r\\S'\"SS9r\\\S'\"SS9r\\\S'\"SS0S9r\\\4\S'\"0S9r\\\4\S'"SS5r\S\ 4Sj5r\S\\\44Sj5r\S\\\44Sj5r\S\4Sj5rS\SS4U4Sjjr SS S.S \!\"\\\4\#\\$S!\4\%4S"\\"\\\4\ \4S#\\ S\S\&\'\(44 U4S$jjjr)S7S%S S&.S'\\"\\\4\#\*4S(\+S)S*\ S\S\&\'\"\\\4\*444 S+jjjr,S8S,\-\\\4S-\\-\S \ S\S\.4 S.jjr/S/\.S\04S0jr1S/\.S\2\34S1jr4S9S2\-\(S-\\-\S3\\5S\S\64 S4jjr7S9S2\-\(S-\\-\S3\\5S\S\2\84 S5jjr9S6r:U=r;$):ChatSambaNovaCloudou SambaNova Cloud chat model. Setup: To use, you should have the environment variables: `SAMBANOVA_URL` set with your SambaNova Cloud URL. `SAMBANOVA_API_KEY` set with your SambaNova Cloud API Key. http://cloud.sambanova.ai/ Example: .. code-block:: python ChatSambaNovaCloud( sambanova_url = SambaNova cloud endpoint URL, sambanova_api_key = set with your SambaNova cloud API key, model = model name, max_tokens = max number of tokens to generate, temperature = model temperature, top_p = model top p, top_k = model top k, stream_options = include usage to get generation metrics ) Key init args — completion params: model: str The name of the model to use, e.g., Meta-Llama-3-70B-Instruct. streaming: bool Whether to use streaming handler when using non streaming methods max_tokens: int max tokens to generate temperature: float model temperature top_p: float model top p top_k: int model top k stream_options: dict stream options, include usage to get generation metrics Key init args — client params: sambanova_url: str SambaNova Cloud Url sambanova_api_key: str SambaNova Cloud api key Instantiate: .. code-block:: python from langchain_community.chat_models import ChatSambaNovaCloud chat = ChatSambaNovaCloud( sambanova_url = SambaNova cloud endpoint URL, sambanova_api_key = set with your SambaNova cloud API key, model = model name, max_tokens = max number of tokens to generate, temperature = model temperature, top_p = model top p, top_k = model top k, stream_options = include usage to get generation metrics ) Invoke: .. code-block:: python messages = [ SystemMessage(content="your are an AI assistant."), HumanMessage(content="tell me a joke."), ] response = chat.invoke(messages) Stream: .. code-block:: python for chunk in chat.stream(messages): print(chunk.content, end="", flush=True) Async: .. code-block:: python response = chat.ainvoke(messages) await response Tool calling: .. code-block:: python from pydantic import BaseModel, Field class GetWeather(BaseModel): '''Get the current weather in a given location''' location: str = Field( ..., description="The city and state, e.g. Los Angeles, CA" ) llm_with_tools = llm.bind_tools([GetWeather, GetPopulation]) ai_msg = llm_with_tools.invoke("Should I bring my umbrella today in LA?") ai_msg.tool_calls .. code-block:: none [ { 'name': 'GetWeather', 'args': {'location': 'Los Angeles, CA'}, 'id': 'call_adf61180ea2b4d228a' } ] Structured output: .. code-block:: python from typing import Optional from pydantic import BaseModel, Field class Joke(BaseModel): '''Joke to tell user.''' setup: str = Field(description="The setup of the joke") punchline: str = Field(description="The punchline to the joke") structured_model = llm.with_structured_output(Joke) structured_model.invoke("Tell me a joke about cats") .. code-block:: python Joke(setup="Why did the cat join a band?", punchline="Because it wanted to be the purr-cussionist!") See `ChatSambanovaCloud.with_structured_output()` for more. Token usage: .. code-block:: python response = chat.invoke(messages) print(response.response_metadata["usage"]["prompt_tokens"] print(response.response_metadata["usage"]["total_tokens"] Response metadata .. code-block:: python response = chat.invoke(messages) print(response.response_metadata) r<default sambanova_urlsambanova_api_keyzMeta-Llama-3.1-8B-InstructmodelF streaming max_tokensffffff? temperatureNtop_ptop_k include_usageTstream_optionsadditional_headersc\rSrSrSrSrg)ChatSambaNovaCloud.Configi$TN__name__ __module__ __qualname____firstlineno__populate_by_name__static_attributes__rhrFrDConfigrg$rFrpr4cgz9Return whether this model can be serialized by Langchain.Frhclss rDis_lc_serializable%ChatSambaNovaCloud.is_lc_serializable'rFc SS0$)NrZrhselfs rD lc_secretsChatSambaNovaCloud.lc_secrets,s#%899rFcURURURURURUR UR S.$)Return a dictionary of identifying parameters. This information is used by the LangChain callback system, which is used for tracing purposes make it possible to monitor LLMs. r[r\r^r`rarbrdrrzs rD_identifying_params&ChatSambaNovaCloud._identifying_params0sCZZ//++ZZZZ"11  rFcg)7Get the type of language model used by this chat model.zsambanovacloud-chatmodelrhrzs rD _llm_typeChatSambaNovaCloud._llm_typeAs*rFkwargsc v>[USSSS9US'[[USS55US'[TU] "S0UD6 g) 'init and validate environment variablesrY SAMBANOVA_URLz,https://api.sambanova.ai/v1/chat/completionsrWrZSAMBANOVA_API_KEYNrh)r,r+super__init__r{r __class__s rDrChatSambaNovaCloud.__init__FsP"6   B #  '< )<>Q R' "# "6"rF tool_choiceparallel_tool_callstools.rrc ^>UVs/sHn[U5PM nnU(ak[U[5(a US;aSnOO[U[5(a U(aSnO0[U[5(a [ S5e[ SU35eSnX$S'X4S'[ TU]"S SU0UD6$s snf zBind tool-like objects to this chat model tool_choice: does not currently support "any", choice like should be one of ["auto", "none", "required"] )autononerequiredrrz7tool_choice must be one of ['auto', 'none', 'required']z;Unrecognized tool_choice type. Expected str, boolReceived: rrrrhr-r@strbooldict ValueErrorrbindr{rrrrr=formatted_toolsrs rD bind_toolsChatSambaNovaCloud.bind_toolsSEJJED1$7EJ +s++&BB"(KK..",KK.. M!!!, / !K +}(;$%w| AnswerWithJustification( # answer='They weigh the same', # justification='A pound is a unit of weight or mass, so a pound of bricks and a pound of feathers both weigh the same.' # ) Example: schema=Pydantic class, method="function_calling", include_raw=True: .. code-block:: python from langchain_community.chat_models import ChatSambaNovaCloud from pydantic import BaseModel class AnswerWithJustification(BaseModel): '''An answer to the user question along with justification for the answer.''' answer: str justification: str llm = ChatSambaNovaCloud(model="Meta-Llama-3.1-70B-Instruct", temperature=0) structured_llm = llm.with_structured_output( AnswerWithJustification, include_raw=True ) structured_llm.invoke( "What weighs more a pound of bricks or a pound of feathers" ) # -> { # 'raw': AIMessage(content='', additional_kwargs={'tool_calls': [{'function': {'arguments': '{"answer": "They weigh the same.", "justification": "A pound is a unit of weight or mass, so one pound of bricks and one pound of feathers both weigh the same amount."}', 'name': 'AnswerWithJustification'}, 'id': 'call_17a431fc6a4240e1bd', 'type': 'function'}]}, response_metadata={'finish_reason': 'tool_calls', 'usage': {'acceptance_rate': 5, 'completion_tokens': 53, 'completion_tokens_after_first_per_sec': 343.7964936837758, 'completion_tokens_after_first_per_sec_first_ten': 439.1205661878638, 'completion_tokens_per_sec': 162.8511306784833, 'end_time': 1731527851.0698032, 'is_last_response': True, 'prompt_tokens': 213, 'start_time': 1731527850.7137961, 'time_to_first_token': 0.20475482940673828, 'total_latency': 0.32545061111450196, 'total_tokens': 266, 'total_tokens_per_sec': 817.3283162354066}, 'model_name': 'Meta-Llama-3.1-70B-Instruct', 'system_fingerprint': 'fastcoe', 'created': 1731527850}, id='95667eaf-447f-4b53-bb6e-b6e1094ded88', tool_calls=[{'name': 'AnswerWithJustification', 'args': {'answer': 'They weigh the same.', 'justification': 'A pound is a unit of weight or mass, so one pound of bricks and one pound of feathers both weigh the same amount.'}, 'id': 'call_17a431fc6a4240e1bd', 'type': 'tool_call'}]), # 'parsed': AnswerWithJustification(answer='They weigh the same.', justification='A pound is a unit of weight or mass, so one pound of bricks and one pound of feathers both weigh the same amount.'), # 'parsing_error': None # } Example: schema=TypedDict class, method="function_calling", include_raw=False: .. code-block:: python # IMPORTANT: If you are using Python <=3.8, you need to import Annotated # from typing_extensions, not from typing. from typing_extensions import Annotated, TypedDict from langchain_community.chat_models import ChatSambaNovaCloud class AnswerWithJustification(TypedDict): '''An answer to the user question along with justification for the answer.''' answer: str justification: Annotated[ Optional[str], None, "A justification for the answer." ] llm = ChatSambaNovaCloud(model="Meta-Llama-3.1-70B-Instruct", temperature=0) structured_llm = llm.with_structured_output(AnswerWithJustification) structured_llm.invoke( "What weighs more a pound of bricks or a pound of feathers" ) # -> { # 'answer': 'They weigh the same', # 'justification': 'A pound is a unit of weight or mass, so one pound of bricks and one pound of feathers both weigh the same amount.' # } Example: schema=OpenAI function schema, method="function_calling", include_raw=False: .. code-block:: python from langchain_community.chat_models import ChatSambaNovaCloud oai_schema = { 'name': 'AnswerWithJustification', 'description': 'An answer to the user question along with justification for the answer.', 'parameters': { 'type': 'object', 'properties': { 'answer': {'type': 'string'}, 'justification': {'description': 'A justification for the answer.', 'type': 'string'} }, 'required': ['answer'] } } llm = ChatSambaNovaCloud(model="Meta-Llama-3.1-70B-Instruct", temperature=0) structured_llm = llm.with_structured_output(oai_schema) structured_llm.invoke( "What weighs more a pound of bricks or a pound of feathers" ) # -> { # 'answer': 'They weigh the same', # 'justification': 'A pound is a unit of weight or mass, so one pound of bricks and one pound of feathers both weigh the same amount.' # } Example: schema=Pydantic class, method="json_mode", include_raw=True: .. code-block:: from langchain_community.chat_models import ChatSambaNovaCloud from pydantic import BaseModel class AnswerWithJustification(BaseModel): answer: str justification: str llm = ChatSambaNovaCloud(model="Meta-Llama-3.1-70B-Instruct", temperature=0) structured_llm = llm.with_structured_output( AnswerWithJustification, method="json_mode", include_raw=True ) structured_llm.invoke( "Answer the following question. " "Make sure to return a JSON blob with keys 'answer' and 'justification'. " "What's heavier a pound of bricks or a pound of feathers?" ) # -> { # 'raw': AIMessage(content='{ "answer": "They are the same weight", "justification": "A pound is a unit of weight or mass, so a pound of bricks and a pound of feathers both weigh the same amount, one pound. The difference is in their density and volume. A pound of feathers would take up more space than a pound of bricks due to the difference in their densities." }', additional_kwargs={}, response_metadata={'finish_reason': 'stop', 'usage': {'acceptance_rate': 5.3125, 'completion_tokens': 79, 'completion_tokens_after_first_per_sec': 292.65701089829776, 'completion_tokens_after_first_per_sec_first_ten': 346.43324678555325, 'completion_tokens_per_sec': 200.012158915008, 'end_time': 1731528071.1708555, 'is_last_response': True, 'prompt_tokens': 70, 'start_time': 1731528070.737394, 'time_to_first_token': 0.16693782806396484, 'total_latency': 0.3949759876026827, 'total_tokens': 149, 'total_tokens_per_sec': 377.2381225105847}, 'model_name': 'Meta-Llama-3.1-70B-Instruct', 'system_fingerprint': 'fastcoe', 'created': 1731528070}, id='83208297-3eb9-4021-a856-ca78a15758df'), # 'parsed': AnswerWithJustification(answer='They are the same weight', justification='A pound is a unit of weight or mass, so a pound of bricks and a pound of feathers both weigh the same amount, one pound. The difference is in their density and volume. A pound of feathers would take up more space than a pound of bricks due to the difference in their densities.'), # 'parsing_error': None # } Example: schema=None, method="json_mode", include_raw=True: .. code-block:: from langchain_community.chat_models import ChatSambaNovaCloud llm = ChatSambaNovaCloud(model="Meta-Llama-3.1-70B-Instruct", temperature=0) structured_llm = llm.with_structured_output(method="json_mode", include_raw=True) structured_llm.invoke( "Answer the following question. " "Make sure to return a JSON blob with keys 'answer' and 'justification'. " "What's heavier a pound of bricks or a pound of feathers?" ) # -> { # 'raw': AIMessage(content='{ "answer": "They are the same weight", "justification": "A pound is a unit of weight or mass, so a pound of bricks and a pound of feathers both weigh the same amount, one pound. The difference is in their density and volume. A pound of feathers would take up more space than a pound of bricks due to the difference in their densities." }', additional_kwargs={}, response_metadata={'finish_reason': 'stop', 'usage': {'acceptance_rate': 4.722222222222222, 'completion_tokens': 79, 'completion_tokens_after_first_per_sec': 357.1315485254867, 'completion_tokens_after_first_per_sec_first_ten': 416.83279609305305, 'completion_tokens_per_sec': 240.92819585198137, 'end_time': 1731528164.8474727, 'is_last_response': True, 'prompt_tokens': 70, 'start_time': 1731528164.4906917, 'time_to_first_token': 0.13837409019470215, 'total_latency': 0.3278985247892492, 'total_tokens': 149, 'total_tokens_per_sec': 454.4088757208256}, 'model_name': 'Meta-Llama-3.1-70B-Instruct', 'system_fingerprint': 'fastcoe', 'created': 1731528164}, id='15261eaf-8a25-42ef-8ed5-f63d8bf5b1b0'), # 'parsed': { # 'answer': 'They are the same weight', # 'justification': 'A pound is a unit of weight or mass, so a pound of bricks and a pound of feathers both weigh the same amount, one pound. The difference is in their density and volume. A pound of feathers would take up more space than a pound of bricks due to the difference in their densities.'}, # }, # 'parsing_error': None # } Example: schema=None, method="json_schema", include_raw=True: .. code-block:: from langchain_community.chat_models import ChatSambaNovaCloud class AnswerWithJustification(BaseModel): answer: str justification: str llm = ChatSambaNovaCloud(model="Meta-Llama-3.1-70B-Instruct", temperature=0) structured_llm = llm.with_structured_output(AnswerWithJustification, method="json_schema", include_raw=True) structured_llm.invoke( "Answer the following question. " "Make sure to return a JSON blob with keys 'answer' and 'justification'. " "What's heavier a pound of bricks or a pound of feathers?" ) # -> { # 'raw': AIMessage(content='{ "answer": "They are the same weight", "justification": "A pound is a unit of weight or mass, so a pound of bricks and a pound of feathers both weigh the same amount, one pound. The difference is in their density and volume. A pound of feathers would take up more space than a pound of bricks due to the difference in their densities." }', additional_kwargs={}, response_metadata={'finish_reason': 'stop', 'usage': {'acceptance_rate': 5.3125, 'completion_tokens': 79, 'completion_tokens_after_first_per_sec': 292.65701089829776, 'completion_tokens_after_first_per_sec_first_ten': 346.43324678555325, 'completion_tokens_per_sec': 200.012158915008, 'end_time': 1731528071.1708555, 'is_last_response': True, 'prompt_tokens': 70, 'start_time': 1731528070.737394, 'time_to_first_token': 0.16693782806396484, 'total_latency': 0.3949759876026827, 'total_tokens': 149, 'total_tokens_per_sec': 377.2381225105847}, 'model_name': 'Meta-Llama-3.1-70B-Instruct', 'system_fingerprint': 'fastcoe', 'created': 1731528070}, id='83208297-3eb9-4021-a856-ca78a15758df'), # 'parsed': AnswerWithJustification(answer='They are the same weight', justification='A pound is a unit of weight or mass, so a pound of bricks and a pound of feathers both weigh the same amount, one pound. The difference is in their density and volume. A pound of feathers would take up more space than a pound of bricks due to the difference in their densities.'), # 'parsing_error': None # } Received unsupported arguments rzL`schema` must be specified when method is `function_calling`. Received None.functionnamerTrfirst_tool_onlykey_namerrpydantic_objectrzI`schema` must be specified when method is not `json_mode`. Received None.z\Unrecognized method argument. Expected one of `function_calling` or `json_mode`. Received: ``rawcgrNrh_s rD;ChatSambaNovaCloud.with_structured_output..RVrFparsed parsing_errorcgrNrhrs rDrrdrFrr exception_keyrrrPr-rr!r rr r/rrr)assignrwith_fallbacksr( r{rrrris_pydantic_schema tool_namellm output_parser parser_assign parser_noneparser_with_fallbacks rDwith_structured_output)ChatSambaNovaCloud.with_structured_outputzsn >vhGH H/7 ' '~ %/v6zB6JI//6( /BC!7J!($(8 !9&! { "C"d9ov6 4V L 0 2 } $~ %C "d9ov6 4V L 0 2 ++1(!5  /66!%(=8M.44NKK#0#?#? _$@$ 3'*>> >& &rFmessages_dictsstopc JU(aKUURUURURURURSUR S. UEnO>UURUURURURURS.UEn[ R"5nURURSURR53SS.UREUUS9nURS:wa'[S URS 3URS 35eU$) z Performs a post request to the LLM API. Args: messages_dicts: List of role / content dicts to use as input. stop: list of stop tokens streaming: wether to do a streaming call Returns: An iterator of response dicts. T rGr^rr[r`rarbstreamrd)rGr^rr[r`rarbBearer application/json Authorizationz Content-Typeheadersjsonr1Sambanova /complete call failed with status code .)r^r[r`rarbrdrequestsSessionpostrYrZget_secret_valuere status_code RuntimeErrortext)r{rrr\rdata http_sessionresponses rD_handle_request"ChatSambaNovaCloud._handle_requests=$ *"oo#//"&"5"5  D+"oo#//  D '') $$   #*4+A+A+R+R+T*U!V 2)) %     3 &C''(+==/#  rFrc DUR5nURS5(a[SURS3US35eUSSS RS S 5nUcS n0n/n/nUSSS RS 5nU(aqXS 'UHgn [ U S S[5(a/[R"U S RS055U S S'UR[U SS95 Mi [UUUUUSSSURS5USUSUSS.USS9n U $![a n[SUSUR 35eSnAff=f![a0nUR[U [U555 SnAGMSnAff=f) Process a non streaming response from the api Args: response: A request Response object Returns generation: an AIMessage with model generation errorrr;Sambanova /complete call failed couldn't get JSON response response: Nchoicesrr3r7r<r;r argumentsT return_id finish_reasonusager[system_fingerprintcreatedrr model_namerridr7rAr;invalid_tool_callsresponse_metadatar)rgetrr Exceptionrr@rdumpsappendr#r"rr) r{r response_dicter7rAr;rraw_tool_calls raw_tool_callr3s rD_process_response$ChatSambaNovaCloud._process_responses $MMOM  ))"G++,A/$oQ'* *1-i8<M*-k:%%omt&TU "0/!1!.y!9!!<_!M&**73+G4&34H&I(3 T"  O MaSX]]O-  *!&--.}c!fEs0AD8!E%8 E"EE"% F/$FFc## SSKnURU5nUR5GHonURS:Xa&[ SUR SURS35eURS:wGa#[UR[5(a![R"UR5nO&[ SUR SURS35eURS5(a&[ SUR SURS35e[US 5S:a5US SRS 5nUS SS S nUS n[Xx0S9n O3SnUS nWURS5USUSUSS.n [UUU 0S9n U v GMoGMr g![a [S5ef=f![a n [ SU SUR35eSn A ff=f7f) Process a streaming response from the api Args: response: An iterable request Response object Yields: generation: an AIMessageChunk with model partial generation rNTcould not import sseclient libraryPlease install it with `pip install sseclient-py`. error_eventrr[DONE]rrrdeltar7r)r7rrAr<rr[rrrr7rrrA3Error getting content chunk raw streamed response: data: ) sseclient ImportError SSEClienteventseventrrrr@rrloadsrlenrr) r{rrclientrrrr7rchunkmetadatar s rD_process_stream_response+ChatSambaNovaCloud._process_stream_response*s#  $$X.]]_E{{m+"G++,Azzl!% / ::)!%**c22#zz%**5*O'334A$zzl!- xx((*O'334A$zzl!- 4 ?+a/(,Y(:(>(>(O "&y/!"4W"=i"H!$Z .$+b!#%!$Z-:%)XXg%6*.w-267K2L'+I $!/$+!.6.0 !  KM*% E  v "I!UZZL* sAG$FAG$$D1F7 G$F44G$7 G!GG!!G$rG run_managerc UR(a'UR"U4X#S.UD6nU(a [U5$[U5nUR"Xb4SS0UD6nUR U5n[ USURS0S9n [U /S9$)as Call SambaNovaCloud models. Args: messages: the prompt composed of a list of messages. stop: a list of strings on which the model should stop generating. If generation stops due to a stop token, the stop token itself SHOULD BE INCLUDED as part of the output. This is not enforced across models right now, but it's a good practice to follow since it makes it much easier to parse the output of the model downstream and understand why generation stopped. run_manager: A run manager with callbacks for the LLM. Returns: result: ChatResult with model generation rr$r\Fr)r3generation_info generations) r\_streamrrKrr r$rr&) r{rGrr$r stream_iterrrr3 generations rD _generateChatSambaNovaCloud._generateys. >>,,#@FK+K88.x8''XXQWX((2#!:!:?!K zl33rFc+# [U5nUR"XR4SS0UD6nURU5H1n[US9nU(aUR UR US9 Uv M3 g7f)a Stream the output of the SambaNovaCloud chat model. Args: messages: the prompt composed of a list of messages. stop: a list of strings on which the model should stop generating. If generation stops due to a stop token, the stop token itself SHOULD BE INCLUDED as part of the output. This is not enforced across models right now, but it's a good practice to follow since it makes it much easier to parse the output of the model downstream and understand why generation stopped. run_manager: A run manager with callbacks for the LLM. Yields: chunk: ChatGenerationChunk with model partial generation r\Tr3r N)rKrr"r%on_llm_new_tokenr) r{rGrr$rrrai_message_chunkr s rDr*ChatSambaNovaCloud._streamsm./x8''WWPVW $ = =h G '0@AE,,UZZu,EK !HsA(A*rhrNNFNN)? %= &d^ %=  %= $k1 2%=%=RDHv' !v'tCH~tI>?@v' : v'v'v' $eDcNI,E&FF Gv'v %) ;T#s(^,;tCy!; ;  ;  ;z9(9y9vM M " #Md%):> &4{#&4tCy!&467 &4  &4  &4V%):> {#tCy!67    % & rFrUz#langchain_sambanova.ChatSambaStudioc^\rSrSr%Sr\"SS9r\\S'\"\ "S5S9r \ \S'\"SSS9r \\S '\"SSS9r \\S '\"S S9r \\\S '\"S S9r\\S'\"SS9r\\S'\"SS9r\\\S'\"S S9r\\\S'\"S S9r\\\S'\"S S9r\\\S'\"SS9r\\\S'\"SS0S9r\\\4\S'\"SSSSS.S9r\\\4\S'S r\\\\4\S'\"0S9r\\\4\S '"S!S"5r\ S#\4S$j5r!\"S#\\\44S%j5r#\"S#\\\44S&j5r$\"S#\4S'j5r%S(\S#S 4U4S)jjr&S S S*.S+\'\(\\\4\)\\*S,\4\+4S-\\(\\\4\\4S.\\S(\S#\,\-\.44 U4S/jjjr/SFS0S S1.S2\\(\\\4\)\04S3\1S4S5\S(\S#\,\-\(\\\4\0444 S6jjjr2S7\.S#\4S8jr3S9\4\.S(\S#\4S:jr5S;\S#\6\\44S<jr7SGS9\4\.S=\\4\S\\S(\S#\84 S>jjr9S?\8S#\:4S@jr;S?\8S#\<\=4SAjr>SHS9\4\.S=\\4\SB\\?S(\S#\@4 SCjjrASHS9\4\.S=\\4\SB\\?S(\S#\<\B4 SDjjrCSErDU=rE$)IChatSambaStudioiu SambaStudio chat model. Setup: To use, you should have the environment variables: `SAMBASTUDIO_URL` set with your SambaStudio deployed endpoint URL. `SAMBASTUDIO_API_KEY` set with your SambaStudio deployed endpoint Key. https://docs.sambanova.ai/sambastudio/latest/index.html Example: .. code-block:: python ChatSambaStudio( sambastudio_url = set with your SambaStudio deployed endpoint URL, sambastudio_api_key = set with your SambaStudio deployed endpoint Key. model = model or expert name (set for Bundle endpoints), max_tokens = max number of tokens to generate, temperature = model temperature, top_p = model top p, top_k = model top k, do_sample = wether to do sample process_prompt = wether to process prompt (set for Bundle generic v1 and v2 endpoints) stream_options = include usage to get generation metrics special_tokens = start, start_role, end_role, end special tokens (set for Bundle generic v1 and v2 endpoints when process prompt set to false or for StandAlone v1 and v2 endpoints) model_kwargs: Optional = Extra Key word arguments to pass to the model. ) Key init args — completion params: model: str The name of the model to use, e.g., Meta-Llama-3-70B-Instruct-4096 (set for Bundle endpoints). streaming: bool Whether to use streaming max_tokens: inthandler when using non streaming methods max tokens to generate temperature: float model temperature top_p: float model top p top_k: int model top k do_sample: bool wether to do sample process_prompt: wether to process prompt (set for Bundle generic v1 and v2 endpoints) stream_options: dict stream options, include usage to get generation metrics special_tokens: dict start, start_role, end_role and end special tokens (set for Bundle generic v1 and v2 endpoints when process prompt set to false or for StandAlone v1 and v2 endpoints) default to llama3 special tokens model_kwargs: dict Extra Key word arguments to pass to the model. Key init args — client params: sambastudio_url: str SambaStudio endpoint Url sambastudio_api_key: str SambaStudio endpoint api key Instantiate: .. code-block:: python from langchain_community.chat_models import ChatSambaStudio chat = ChatSambaStudio=( sambastudio_url = set with your SambaStudio deployed endpoint URL, sambastudio_api_key = set with your SambaStudio deployed endpoint Key. model = model or expert name (set for Bundle endpoints), max_tokens = max number of tokens to generate, temperature = model temperature, top_p = model top p, top_k = model top k, do_sample = wether to do sample process_prompt = wether to process prompt (set for Bundle generic v1 and v2 endpoints) stream_options = include usage to get generation metrics special_tokens = start, start_role, end_role, and special tokens (set for Bundle generic v1 and v2 endpoints when process prompt set to false or for StandAlone v1 and v2 endpoints) model_kwargs: Optional = Extra Key word arguments to pass to the model. ) Invoke: .. code-block:: python messages = [ SystemMessage(content="your are an AI assistant."), HumanMessage(content="tell me a joke."), ] response = chat.invoke(messages) Stream: .. code-block:: python for chunk in chat.stream(messages): print(chunk.content, end="", flush=True) Async: .. code-block:: python response = chat.ainvoke(messages) await response Tool calling: .. code-block:: python from pydantic import BaseModel, Field class GetWeather(BaseModel): '''Get the current weather in a given location''' location: str = Field( ..., description="The city and state, e.g. Los Angeles, CA" ) llm_with_tools = llm.bind_tools([GetWeather, GetPopulation]) ai_msg = llm_with_tools.invoke("Should I bring my umbrella today in LA?") ai_msg.tool_calls .. code-block:: python [ { 'name': 'GetWeather', 'args': {'location': 'Los Angeles, CA'}, 'id': 'call_adf61180ea2b4d228a' } ] Structured output: .. code-block:: python from typing import Optional from pydantic import BaseModel, Field class Joke(BaseModel): '''Joke to tell user.''' setup: str = Field(description="The setup of the joke") punchline: str = Field(description="The punchline to the joke") structured_model = llm.with_structured_output(Joke) structured_model.invoke("Tell me a joke about cats") .. code-block:: python Joke(setup="Why did the cat join a band?", punchline="Because it wanted to be the purr-cussionist!") See `ChatSambaStudio.with_structured_output()` for more. Token usage: .. code-block:: python response = chat.invoke(messages) print(response.response_metadata["usage"]["prompt_tokens"] print(response.response_metadata["usage"]["total_tokens"] Response metadata .. code-block:: python response = chat.invoke(messages) print(response.response_metadata) r<rWsambastudio_urlsambastudio_api_keyT)rXexcludebase_url streaming_urlNr[Fr\r]r^r_r`rarb do_sampleprocess_promptrcrdz<|begin_of_text|>z;<|begin_of_text|><|start_header_id|>{role}<|end_header_id|>z <|eot_id|>z.<|start_header_id|>assistant<|end_header_id|> )start start_roleend_roleendspecial_tokens model_kwargsrec\rSrSrSrSrg)ChatSambaStudio.ConfigiTrhNrirhrFrDrprOrqrFrpr4cgrsrhrts rDrv"ChatSambaStudio.is_lc_serializablerxrFc SSS.$)NrArB)rArBrhrzs rDr|ChatSambaStudio.lc_secretss 1#8  rFc URURURURURUR UR URURURURS. $)r r[r\r^r`rarbrFrGrdrLrMrUrzs rDr#ChatSambaStudio._identifying_paramsseZZ//++ZZZZ"11"11"11 --  rFcg)rzsambastudio-chatmodelrhrzs rDrChatSambaStudio._llm_types'rFrc >[USS5US'[[USS55US'URUS5uUS'US'[TU]"S0UD6 g) rrASAMBASTUDIO_URLrBSAMBASTUDIO_API_KEYrDrENrh)r,r+_get_sambastudio_urlsrrrs rDrChatSambaStudio.__init__sx$8 %'8%  !)> )>@U V) $%7;6P6P $ %7 3zF?3 "6"rFrr.rrc ^>UVs/sHn[U5PM nnU(ak[U[5(a US;aSnOO[U[5(a U(aSnO0[U[5(a [ S5e[ SU35eSnX$S'X4S'[ TU]"S SU0UD6$s snfrrrs rDrChatSambaStudio.bind_toolsrrrrrrrrc U(a[SU35e[U5nUS:XaNUc [S5e[U5SSnURU/US9nU(a [ U/SS9nO[ USS 9nOUS :Xa5UnU(a![ [[U5n[US 9nOc[5nOXUS :XaCUc [S 5eUnU(a![ [[U5n[US 9nO[5nO[SUS35eU(aT[R"[S5U-SS9n [R"SS9n U RU /SS9n [US9U -$Xx-$)a?Model wrapper that returns outputs formatted to match the given schema. Args: schema: The output schema. Can be passed in as: - an OpenAI function/tool schema, - a JSON Schema, - a TypedDict class, - or a Pydantic class. If `schema` is a Pydantic class then the model output will be a Pydantic instance of that class, and the model-generated fields will be validated by the Pydantic class. Otherwise the model output will be a dict and will not be validated. See :meth:`langchain_core.utils.function_calling.convert_to_openai_tool` for more on how to properly specify types and descriptions of schema fields when specifying a Pydantic or TypedDict class. method: The method for steering model generation, either "function_calling" "json_mode" or "json_schema". If "function_calling" then the schema will be converted to an OpenAI function and the returned model will make use of the function-calling API. If "json_mode" or "json_schema" then OpenAI's JSON mode will be used. Note that if using "json_mode" or "json_schema" then you must include instructions for formatting the output into the desired schema into the model call. include_raw: If False then only the parsed structured output is returned. If an error occurs during model output parsing it will be raised. If True then both the raw model response (a BaseMessage) and the parsed model response will be returned. If an error occurs during output parsing it will be caught and returned as well. The final output is always a dict with keys "raw", "parsed", and "parsing_error". Returns: A Runnable that takes same inputs as a :class:`langchain_core.language_models.chat.BaseChatModel`. If `include_raw` is False and `schema` is a Pydantic class, Runnable outputs an instance of `schema` (i.e., a Pydantic object). Otherwise, if `include_raw` is False then Runnable outputs a dict. If `include_raw` is True, then Runnable outputs a dict with keys: - `"raw"`: BaseMessage - `"parsed"`: None if there was a parsing error, otherwise the type depends on the `schema` as described above. - `"parsing_error"`: Optional[BaseException] Example: schema=Pydantic class, method="function_calling", include_raw=False: .. code-block:: python from typing import Optional from langchain_community.chat_models import ChatSambaStudio from pydantic import BaseModel, Field class AnswerWithJustification(BaseModel): '''An answer to the user question along with justification for the answer.''' answer: str justification: str = Field( description="A justification for the answer." ) llm = ChatSambaStudio(model="Meta-Llama-3.1-70B-Instruct", temperature=0) structured_llm = llm.with_structured_output(AnswerWithJustification) structured_llm.invoke( "What weighs more a pound of bricks or a pound of feathers" ) # -> AnswerWithJustification( # answer='They weigh the same', # justification='A pound is a unit of weight or mass, so a pound of bricks and a pound of feathers both weigh the same.' # ) Example: schema=Pydantic class, method="function_calling", include_raw=True: .. code-block:: python from langchain_community.chat_models import ChatSambaStudio from pydantic import BaseModel class AnswerWithJustification(BaseModel): '''An answer to the user question along with justification for the answer.''' answer: str justification: str llm = ChatSambaStudio(model="Meta-Llama-3.1-70B-Instruct", temperature=0) structured_llm = llm.with_structured_output( AnswerWithJustification, include_raw=True ) structured_llm.invoke( "What weighs more a pound of bricks or a pound of feathers" ) # -> { # 'raw': AIMessage(content='', additional_kwargs={'tool_calls': [{'function': {'arguments': '{"answer": "They weigh the same.", "justification": "A pound is a unit of weight or mass, so one pound of bricks and one pound of feathers both weigh the same amount."}', 'name': 'AnswerWithJustification'}, 'id': 'call_17a431fc6a4240e1bd', 'type': 'function'}]}, response_metadata={'finish_reason': 'tool_calls', 'usage': {'acceptance_rate': 5, 'completion_tokens': 53, 'completion_tokens_after_first_per_sec': 343.7964936837758, 'completion_tokens_after_first_per_sec_first_ten': 439.1205661878638, 'completion_tokens_per_sec': 162.8511306784833, 'end_time': 1731527851.0698032, 'is_last_response': True, 'prompt_tokens': 213, 'start_time': 1731527850.7137961, 'time_to_first_token': 0.20475482940673828, 'total_latency': 0.32545061111450196, 'total_tokens': 266, 'total_tokens_per_sec': 817.3283162354066}, 'model_name': 'Meta-Llama-3.1-70B-Instruct', 'system_fingerprint': 'fastcoe', 'created': 1731527850}, id='95667eaf-447f-4b53-bb6e-b6e1094ded88', tool_calls=[{'name': 'AnswerWithJustification', 'args': {'answer': 'They weigh the same.', 'justification': 'A pound is a unit of weight or mass, so one pound of bricks and one pound of feathers both weigh the same amount.'}, 'id': 'call_17a431fc6a4240e1bd', 'type': 'tool_call'}]), # 'parsed': AnswerWithJustification(answer='They weigh the same.', justification='A pound is a unit of weight or mass, so one pound of bricks and one pound of feathers both weigh the same amount.'), # 'parsing_error': None # } Example: schema=TypedDict class, method="function_calling", include_raw=False: .. code-block:: python # IMPORTANT: If you are using Python <=3.8, you need to import Annotated # from typing_extensions, not from typing. from typing_extensions import Annotated, TypedDict from langchain_community.chat_models import ChatSambaStudio class AnswerWithJustification(TypedDict): '''An answer to the user question along with justification for the answer.''' answer: str justification: Annotated[ Optional[str], None, "A justification for the answer." ] llm = ChatSambaStudio(model="Meta-Llama-3.1-70B-Instruct", temperature=0) structured_llm = llm.with_structured_output(AnswerWithJustification) structured_llm.invoke( "What weighs more a pound of bricks or a pound of feathers" ) # -> { # 'answer': 'They weigh the same', # 'justification': 'A pound is a unit of weight or mass, so one pound of bricks and one pound of feathers both weigh the same amount.' # } Example: schema=OpenAI function schema, method="function_calling", include_raw=False: .. code-block:: python from langchain_community.chat_models import ChatSambaStudio oai_schema = { 'name': 'AnswerWithJustification', 'description': 'An answer to the user question along with justification for the answer.', 'parameters': { 'type': 'object', 'properties': { 'answer': {'type': 'string'}, 'justification': {'description': 'A justification for the answer.', 'type': 'string'} }, 'required': ['answer'] } } llm = ChatSambaStudio(model="Meta-Llama-3.1-70B-Instruct", temperature=0) structured_llm = llm.with_structured_output(oai_schema) structured_llm.invoke( "What weighs more a pound of bricks or a pound of feathers" ) # -> { # 'answer': 'They weigh the same', # 'justification': 'A pound is a unit of weight or mass, so one pound of bricks and one pound of feathers both weigh the same amount.' # } Example: schema=Pydantic class, method="json_mode", include_raw=True: .. code-block:: from langchain_community.chat_models import ChatSambaStudio from pydantic import BaseModel class AnswerWithJustification(BaseModel): answer: str justification: str llm = ChatSambaStudio(model="Meta-Llama-3.1-70B-Instruct", temperature=0) structured_llm = llm.with_structured_output( AnswerWithJustification, method="json_mode", include_raw=True ) structured_llm.invoke( "Answer the following question. " "Make sure to return a JSON blob with keys 'answer' and 'justification'. " "What's heavier a pound of bricks or a pound of feathers?" ) # -> { # 'raw': AIMessage(content='{ "answer": "They are the same weight", "justification": "A pound is a unit of weight or mass, so a pound of bricks and a pound of feathers both weigh the same amount, one pound. The difference is in their density and volume. A pound of feathers would take up more space than a pound of bricks due to the difference in their densities." }', additional_kwargs={}, response_metadata={'finish_reason': 'stop', 'usage': {'acceptance_rate': 5.3125, 'completion_tokens': 79, 'completion_tokens_after_first_per_sec': 292.65701089829776, 'completion_tokens_after_first_per_sec_first_ten': 346.43324678555325, 'completion_tokens_per_sec': 200.012158915008, 'end_time': 1731528071.1708555, 'is_last_response': True, 'prompt_tokens': 70, 'start_time': 1731528070.737394, 'time_to_first_token': 0.16693782806396484, 'total_latency': 0.3949759876026827, 'total_tokens': 149, 'total_tokens_per_sec': 377.2381225105847}, 'model_name': 'Meta-Llama-3.1-70B-Instruct', 'system_fingerprint': 'fastcoe', 'created': 1731528070}, id='83208297-3eb9-4021-a856-ca78a15758df'), # 'parsed': AnswerWithJustification(answer='They are the same weight', justification='A pound is a unit of weight or mass, so a pound of bricks and a pound of feathers both weigh the same amount, one pound. The difference is in their density and volume. A pound of feathers would take up more space than a pound of bricks due to the difference in their densities.'), # 'parsing_error': None # } Example: schema=None, method="json_mode", include_raw=True: .. code-block:: from langchain_community.chat_models import ChatSambaStudio llm = ChatSambaStudio(model="Meta-Llama-3.1-70B-Instruct", temperature=0) structured_llm = llm.with_structured_output(method="json_mode", include_raw=True) structured_llm.invoke( "Answer the following question. " "Make sure to return a JSON blob with keys 'answer' and 'justification'. " "What's heavier a pound of bricks or a pound of feathers?" ) # -> { # 'raw': AIMessage(content='{ "answer": "They are the same weight", "justification": "A pound is a unit of weight or mass, so a pound of bricks and a pound of feathers both weigh the same amount, one pound. The difference is in their density and volume. A pound of feathers would take up more space than a pound of bricks due to the difference in their densities." }', additional_kwargs={}, response_metadata={'finish_reason': 'stop', 'usage': {'acceptance_rate': 4.722222222222222, 'completion_tokens': 79, 'completion_tokens_after_first_per_sec': 357.1315485254867, 'completion_tokens_after_first_per_sec_first_ten': 416.83279609305305, 'completion_tokens_per_sec': 240.92819585198137, 'end_time': 1731528164.8474727, 'is_last_response': True, 'prompt_tokens': 70, 'start_time': 1731528164.4906917, 'time_to_first_token': 0.13837409019470215, 'total_latency': 0.3278985247892492, 'total_tokens': 149, 'total_tokens_per_sec': 454.4088757208256}, 'model_name': 'Meta-Llama-3.1-70B-Instruct', 'system_fingerprint': 'fastcoe', 'created': 1731528164}, id='15261eaf-8a25-42ef-8ed5-f63d8bf5b1b0'), # 'parsed': { # 'answer': 'They are the same weight', # 'justification': 'A pound is a unit of weight or mass, so a pound of bricks and a pound of feathers both weigh the same amount, one pound. The difference is in their density and volume. A pound of feathers would take up more space than a pound of bricks due to the difference in their densities.'}, # }, # 'parsing_error': None # } Example: schema=None, method="json_schema", include_raw=True: .. code-block:: from langchain_community.chat_models import ChatSambaStudio class AnswerWithJustification(BaseModel): answer: str justification: str llm = ChatSambaStudio(model="Meta-Llama-3.1-70B-Instruct", temperature=0) structured_llm = llm.with_structured_output(AnswerWithJustification, method="json_schema", include_raw=True) structured_llm.invoke( "Answer the following question. " "Make sure to return a JSON blob with keys 'answer' and 'justification'. " "What's heavier a pound of bricks or a pound of feathers?" ) # -> { # 'raw': AIMessage(content='{ "answer": "They are the same weight", "justification": "A pound is a unit of weight or mass, so a pound of bricks and a pound of feathers both weigh the same amount, one pound. The difference is in their density and volume. A pound of feathers would take up more space than a pound of bricks due to the difference in their densities." }', additional_kwargs={}, response_metadata={'finish_reason': 'stop', 'usage': {'acceptance_rate': 5.3125, 'completion_tokens': 79, 'completion_tokens_after_first_per_sec': 292.65701089829776, 'completion_tokens_after_first_per_sec_first_ten': 346.43324678555325, 'completion_tokens_per_sec': 200.012158915008, 'end_time': 1731528071.1708555, 'is_last_response': True, 'prompt_tokens': 70, 'start_time': 1731528070.737394, 'time_to_first_token': 0.16693782806396484, 'total_latency': 0.3949759876026827, 'total_tokens': 149, 'total_tokens_per_sec': 377.2381225105847}, 'model_name': 'Meta-Llama-3.1-70B-Instruct', 'system_fingerprint': 'fastcoe', 'created': 1731528070}, id='83208297-3eb9-4021-a856-ca78a15758df'), # 'parsed': AnswerWithJustification(answer='They are the same weight', justification='A pound is a unit of weight or mass, so a pound of bricks and a pound of feathers both weigh the same amount, one pound. The difference is in their density and volume. A pound of feathers would take up more space than a pound of bricks due to the difference in their densities.'), # 'parsing_error': None # } rrzJschema must be specified when method is 'function_calling'. Received None.rrrTrrrrrzGschema must be specified when method is not 'json_mode'. Received None.z\Unrecognized method argument. Expected one of 'function_calling' or 'json_mode'. Received: ''rcgrNrhrs rDr8ChatSambaStudio.with_structured_output..9rrFrcgrNrhrs rDrrc;rrFrrrrrrs rDr&ChatSambaStudio.with_structured_output sp >vhGH H/7 ' '~ %/v6zB6JI//6( /BC!7J!($(8 !9&! { "C"d9ov6 4V L 0 2 } $~ %C "d9ov6 4V L 0 2 ++1(!5  /66!%(=8M.44NKK#0#?#? _$@$ 3'*>> >& &rFr3c,[U[5(aSnU$[U[5(aSnU$[U[5(aSnU$[U[5(aSnU$[U[ 5(aUR nU$[SU35e)z Get the role of LangChain BaseMessage Args: message: LangChain BaseMessage Returns: str: Role of the LangChain BaseMessage r8r9r:r=r?)r@rrrrrr6rB)r{r3r6s rD _get_roleChatSambaStudio._get_roleCs g} - -D  . .D  + +D  - -D   - -<rGrztool calling not supported in API Generic V2 without process_prompt, switch to OpenAI compatible API or Generic V2 API with process_prompt=TruerHrI)r6 rJrK)rGr@rrrgr7rArr>rrrkeysNotImplementedErrorrLformat)r{rGr messages_dictr3rCmessages_strings rD_messages_to_string#ChatSambaStudio._messages_to_string[s    #?--M $gy11&-jj $w 7#*??$L $w'@'@@5<5N5N(6 \2( 2b86:L355&-jj $w 7#*??(/(<(< $L'.jj $w 7#*??$L j)00>;$>#jj7O$&++-')A #11':O##6#6|#D#K#K0$L$ Qw&7q#99#6#6z#BB $ 2259 9OrFurlcSU;aUnUnX#4$SU;aURSS5nUnX#4$UnSU;a#SRURS55nX#4$[S5e)z Get streaming and non streaming URLs from the given URL Args: url: string with sambastudio base or streaming endpoint url Returns: base_url: string with url to do non streaming calls streaming_url: string with url to do streaming calls chat/completionsrzstream/r<genericzgeneric/streamUnsupported URL)replacejoinsplitr)r{rtrD stream_urls rDr\%ChatSambaStudio._get_sambastudio_urlss  $HJ##3;;y"5 ## #!1!6!6syy7K!LJ##%%677rFrc :SUR;a[U5nUURUURURUR UR UURS. UEnUR5VVs0sH upxUcM Xx_M nnnSURR53SS.UREn GODSUR;aSUR"U40UD6S./n URURURURUR UR URS .n URb0U EUREn U R5VVs0sH upxUcM Xx_M n nnXS .nS URR50UREn GOZS UR;Ga0S UR!5;a [#S5eURURURURUR UR URS .UEn URb0U EUREn U R5VVs0sH,upxUcM U[%U5R&[)U5S._M. n nnU(aURU5U S.nOURU5/U S.nS URR50UREn O[+SURS35e[,R."5n U(aU R1UR2XSS9n OU R1UR4XSS9n U R6S:wa&[9SU R6SU R:S35eU $s snnfs snnfs snnf)z Performs a post request to the LLM API. Args: messages_dicts: List of role / content dicts to use as input. stop: list of stop tokens streaming: wether to do a streaming call Returns: A request Response object rvrrrrapi/v2/predict/genericitem0)rvalue) select_expertrGmax_tokens_to_generater`rarbrF)itemsparamskeyapi/predict/genericrz_tool calling not supported in API Generic V1, switch to OpenAI compatible API or Generic V2 API)rOr)instancer) instancesrrx9only openai, generic v1 and generic v2 APIs are supportedTrFrrr)rArKr^r[r`rarbrdrrBrrerrrGrFrMrmrnrOrjrrrrrrErDrrr)r{rGrr\rrrrrrrrrrs rDrChatSambaStudio._handle_requests( !5!5 528?"A 2)) G&)=)= =)A)A()Uf)UVE"&"&"5"5*.//#//!^^&F  ,8F8d&7&7839<<>W>ZSUjcj>FW"5Dt//@@B))G #d&:&: :&++-')H "&"&"5"5*.//#//!^^  F  ,8F8d&7&78#),,."0JCId5k22SZHH"0   $ 8 8 B$ #'":":8"D!E$ t//@@B))G !$"6"6!7KL   '') #((""Gt)H$(( w%)H   3 &C''(==/$  wT0X4s$9N N #N/NN$&Nrc vUR5n0n/n/nSUR;aUSSSR SS 5nUcS nUS nUSSS UR S 5US USUSS.n USSSR S5n U (aqXS'U Hgn [ U SS[5(a/[R"U SR S055U SS'UR[U SS95 Mi OSUR;aUSSSSnUSSS nUSSn USSSR S5n U (aqXS'U Hgn [ U SS[5(a/[R"U SR S055U SS'UR[U SS95 Mi O9SUR;aUSSSnSnUn O[SURS35e[UUUUU US9$![a n[SUSUR35eSnAff=f![a0nUR[U [U555 SnAGMSnAff=f![a0nUR[U [U555 SnAGMRSnAff=f)rrrNrvrrr3r7r<rrrr[rrrr;rrTrrrr completionr predictionsrxrr)rrrrrArr@rrrr#r"rrr) r{rr r rAr;rr7rrr r s rDr !ChatSambaStudio._process_response;s: $MMOM-/  !5!5 5#I.q1)<@@BOGt$B!.y!9!!<_!M&**73+G4&34H&I(3 ! +95a8CGG UN2@,/%3M!- ";K"H$OOAE)*599+rJB j1+>"))+MTJ &4&)=)= =#G,Q/8FGw'*40B -g 6q 9 *73A6w?CCLQN2@,/%3M!- ";K"H$OOAE)*599+rJB j1+>"))+MTJ &4#d&:&: :#M215lCGB - !$"6"6!7KL  /!1/   U MaSX]]O-  F%*112=#a&I,%*112=#a&IsGH+I/I> H>H99H> I; $I66I;> J8$J33J8c# # SSKnSUR;GaSnURU5nUR 5GHnUR S:Xa&[ SURSURS35eURS :wGaR[UR[5(a![R"UR5nO&[ SURSURS35eURS 5(a&[ SURSURS35e[US 5S:a-US SRS 5nUS SS SnUSn0n O'SnUSnUURS5USUSUSS.n URS5b'SnUSnUURS5USUSUSS.n [UUU 0S9v GMGM gSUR;GaUR#5GHn [R"U 5nUSSSSSnUSSSSnUSSSSS(Ga:USSSSRS5USSSSRS5USSSSRS 5USSSSRS!5USSSSRS"5USSSSRS#5USSSSRS$5USSSSRS%5USSSSRS&5USSSSRS'5USSSSRS(5S). S*.n O0n [UUU 0S9v GM gS,UR;Ga~UR#5GHhn [R"U 5nUSS-SSnSnUSS-SS(GaUSS-SRS5USS-SRS5USS-SRS 5USS-SRS!5USS-SRS"5USS-SRS#5USS-SRS$5USS-SRS%5USS-SRS&5USS-SRS'5USS-SRS(5S). S*.n O0n [UUU 0S9v GMk g[%S.URS/35e![a [S5ef=f![ a n [ SU SUR35eSn A ff=f![ an [ SU S+U 35eSn A ff=f![ an [ SU S+U 35eSn A ff=f7f)0rrNrrvr<rrrrrrrrr7rrr[rrrrrrrresultrr stream_tokenis_last_response stop_reasonpromptprompt_tokens_countcompletion_tokens_counttotal_tokens_count start_timeend_timemodel_execution_timetime_to_first_tokenthroughput_after_first_tokenbatch_size_used) rrrrrrrrr)rrrzline: r responsesrxr)rrrArrrrrrr@rrrrrrr iter_linesr) r{rrrrrrr7rr!r lines rDr"(ChatSambaStudio._process_stream_responsesD   !5!5 5M((2F;;-/&K#//0 ::,a) 5zzX-%ejj#66#'::ejj#9D"."S#+#7#7"8#(::,a!1#  88G,,"."S#+#7#7"8#(::,a!1# tI/!3,0OA,>,B,B?,SM&*9oa&8&A)&LG!%dB')H&(G!%dB1>)-'):.27m6:;O6P+/ ? (H 88G,8&(G!%dB1>)-'):.27m6:;O6P+/ ? (H-$+!.6.0 Q.)~&)=)= = ++-5::d+D"8nW5a8A.QGh03D9BH~g.q1':;MNN-1(^G-DQ-G-P-T-T -.'+8nW&=a&@&I&M&Mh&W7;H~g7Nq7Q$+8""%#&;"<;?>';RST;U$+<""%#&?"@6:8nW6Ma6P$+7""%#&:";.28nW.Ea.H.Q.U.U$0/"-1N7,CA,Fw,O,S,S$.-"9=Xw8OPQ8R$+9""%#&<"=7;H~g7Nq7Q$+8""%#&;"<@DXw@W$%A"")A++.3/M+N37>'3J13M$+4""%#&7"87& "$H$&( '*2*, W.r#d&:&: : ++-5::d+D"8n[9!<^LGBH~k2156HII-1(^K-H-K-O-O -.'+8n[&A!&D&H&H&R7;H~k7R$%8""%#&;"<;?>+;V$%<""%#&?"@6:8n[6Q$%7""%#&:";.28n[.I!.L.P.P$0/"-1N;,G,J,N,N$.-"9=X{8S$%9""%#&<"=7;H~k7R$%8""%#&;"<@DX$/A""#A%%(S)G%H37>+3Nq3Q3U3U$54"3& "$H$&( '*2*, W.r!$"6"6!7KL { E  D!&MaS  .t!&MaS (t!&MaS (sWU A.W7E U#.WFV+WE#V3+W U  W# V -VV  W V0V++V00W3 W=WWWr$c UR(a'UR"U4X#S.UD6nU(a [U5$UR"X4SS0UD6nUR U5n[ US9n[ U/S9$)ap Call SambaStudio models. Args: messages: the prompt composed of a list of messages. stop: a list of strings on which the model should stop generating. If generation stops due to a stop token, the stop token itself SHOULD BE INCLUDED as part of the output. This is not enforced across models right now, but it's a good practice to follow since it makes it much easier to parse the output of the model downstream and understand why generation stopped. run_manager: A run manager with callbacks for the LLM. Returns: result: ChatResult with model generation r&r\Fr0r()r\r*rrr r$r&) r{rGrr$rr+rr3r,s rDr-ChatSambaStudio._generatems}. >>,,#@FK+K88''R%R6R((2#G4 zl33rFc+# UR"X4SS0UD6nURU5H1n[US9nU(aURURUS9 Uv M3 g7f)a Stream the output of the SambaStudio model. Args: messages: the prompt composed of a list of messages. stop: a list of strings on which the model should stop generating. If generation stops due to a stop token, the stop token itself SHOULD BE INCLUDED as part of the output. This is not enforced across models right now, but it's a good practice to follow since it makes it much easier to parse the output of the model downstream and understand why generation stopped. run_manager: A run manager with callbacks for the LLM. Yields: chunk: ChatGenerationChunk with model partial generation r\Tr0r1N)rr"r%r2r)r{rGrr$rrr3r s rDr*ChatSambaStudio._streamsb.''Q$Q&Q $ = =h G '0@AE,,UZZu,EK !HsAArhrNr5r6)Frjrkrlrmr7r0rArr8r1rBrDrEr[r r\rr^r9r`r:rarbrFrGrdrrrLrMrerpr;rvr<r|rrrr rr rr*r'rrrr/r rrgrrrr r\r2rrr rrr"rr&r-r%r*ror=r>s@rDr@r@s iV!,OS,%*9R=%AA"d3Hc3'r48M38# .E8C=.GE*It*KD)J) #(#5K%5"40E8E?0 .E8C=. %d 3Ix~3 %*4%8NHTN8I%*OT3J%KNDcNKA%*(W$D  &NDcN( .2L(4S>*12).r):S#X:/  4 DcN   T#s(^  ('3'' # # #$CG.3 %=d38nd3i#s(9KXUVW%=eDcND#$=>? %= &d^ %=  %= $k1 2%=%=RDHw' !w'tCH~tI>?@w' : w'w'w' $eDcNI,E&FF Gw'r 0ED,=EEQTEN$$sCx$:%)$) }{#}tCy!}D> }  }  }~_ (_ y_ BO O " #Oh%):> 4{# 4tCy! 467 4  4  4J%):> {#tCy!67    % & rFr@)Jroperatorrtypingrrrrrr r r r r rrrlangchain_core._api.deprecationrlangchain_core.callbacksrlangchain_core.language_modelsr*langchain_core.language_models.chat_modelsrrlangchain_core.messagesrrrrrrrrlangchain_core.output_parsersrr"langchain_core.output_parsers.baser*langchain_core.output_parsers.openai_toolsr r!r"r#langchain_core.outputsr$r%r&langchain_core.runnablesr'r(r)langchain_core.toolsr*langchain_core.utilsr+r,%langchain_core.utils.function_callingr-langchain_core.utils.pydanticr.pydanticr/r0r1r2rrErKrrPrUr@rhrFrDrs.     6>   @ SROO)LH?00kd38nD D$5 $tCH~:N @C@D@  ? J J   J Z  < eme  erF