dh(SSKJr SSKrSSKJr SSKJrJrJrJ r J r SSK r SSK J r SSKJr SSKJr SSKJr SS KJr "S S \5rg) ) annotationsN)StringIO)AnyDictIteratorListOptional)CallbackManagerForLLMRun)LLM)GenerationChunk)get_pydantic_field_names) ConfigDictc\rSrSr%SrSrS\S'SrS\S'S rS \S 'S r S \S'Sr S\S'Sr S \S'Sr S\S'Sr S\S'S rS \S'SrS \S'SrS\S'SrS\S'SrS\S'S rS \S!'S"rS \S#'S$rS\S%'S$rS\S&'SrS \S''S(rS\S)'S*rS\S+'\"S,S-9r\S6S.j5r\S7S/j5r\S8S0j5rS9S:S1jjr S;SS4jr#S5r$g)? LlamafileaLlamafile lets you distribute and run large language models with a single file. To get started, see: https://github.com/Mozilla-Ocho/llamafile To use this class, you will need to first: 1. Download a llamafile. 2. Make the downloaded file executable: `chmod +x path/to/model.llamafile` 3. Start the llamafile in server mode: `./path/to/model.llamafile --server --nobrowser` Example: .. code-block:: python from langchain_community.llms import Llamafile llm = Llamafile() llm.invoke("Tell me a joke.") zhttp://localhost:8080strbase_urlNz Optional[int]request_timeoutFbool streamingintseedg?float temperature(top_kgffffff?top_pg?min_p n_predictrn_keepg?tfs_z typical_pg?repeat_penalty@ repeat_last_nT penalize_nlgpresence_penaltyfrequency_penaltymirostatg@ mirostat_taug? mirostat_etaforbid)extracg)N llamafile)selfs Z/var/www/html/shao/venv/lib/python3.13/site-packages/langchain_community/llms/llamafile.py _llm_typeLlamafile._llm_typexscn/SQn[UR5Vs/sH o"U;dM UPM nnU$s snf)N) rcachecallback_manager callbacksmetadatanamerrtagsverbosecustom_get_token_ids)r __class__)r2 ignore_keyskattrss r3_param_fieldnamesLlamafile._param_fieldnames|sB  0? ?!KCWA?    s22cJ0nURHn[X5X'M U$N)rDgetattr)r2params fieldnames r3_default_paramsLlamafile._default_paramss)//I ' 8F 0 r6c URnUR5HupEXC;dM XSU'M Ub[U5S:aXS'UR(aSUS'U$)NrstopTstream)rKitemslenr)r2rNkwargsrIrBvs r3_get_parametersLlamafile._get_parameterss^%% LLNDA{q #  D A !6N >>#F8  r6c UR(a\[5nUR"U4X#S.UD6HnURUR5 M UR 5nSSS5 U$UR "SSU0UD6nSU0UEn [R"URS3SS0U SURS 9n U R5 S U lU R5S nU$!,(df  W$=f![RRa. [RRS URS 35ef=f)aXRequest prompt completion from the llamafile server and return the output. Args: prompt: The prompt to use for generation. stop: A list of strings to stop generation when encountered. run_manager: **kwargs: Any additional options to pass as part of the generation request. Returns: The string generated by the model. )rN run_managerNrNprompt /completion Content-Typeapplication/jsonFurlheadersjsonrOtimeoutTCould not connect to Llamafile server. Please make sure that a server is running at .zutf-8contentr1)rr_streamwritetextgetvaluerTrequestspostrr exceptionsConnectionErrorraise_for_statusencodingr_) r2rXrNrWrRbuffchunkrfrIpayloadresponses r3_callLlamafile._callsO, >>t!\\!%BHEJJuzz* }} K))>t>v>F262G #====/5&(:!  00  % % ' 'H ==?9-DKGK &&66 ))99337==/D sAC& 1C8& C58A Ec+# UR"SSU0UD6nSU;aSUS'SU0UEn[R"URS3SS0USURS9nS UlURSS 9HAnURU5n [U S 9n U(aURU RS9 U v MC g![R R a. [R R S URS 35ef=f7f)a"Yields results objects as they are generated in real time. It also calls the callback manager's on_llm_new_token event with similar parameters to the OpenAI LLM class method of the same name. Args: prompt: The prompts to pass into the model. stop: Optional list of stop words to use when generating. run_manager: **kwargs: Any additional options to pass as part of the generation request. Returns: A generator representing the stream of tokens being generated. Yields: Dictionary-like objects each containing a token Example: .. code-block:: python from langchain_community.llms import Llamafile llm = Llamafile( temperature = 0.0 ) for chunk in llm.stream("Ask 'Hi, how are you?' like a pirate:'", stop=["'"," "]): result = chunk["choices"][0] print(result["text"], end='', flush=True) rNrOTrXrYrZr[r\rarbutf8)decode_unicode)rf)tokenNr1) rTrhrirrrjrkrm iter_lines_get_chunk_contentr on_llm_new_tokenrf) r2rXrNrWrRrIrprq raw_chunkrcros r3rdLlamafile._streams#L%%:4:6: 6 !#F8 V.v. }}}}o[1"$6,,H#!,,D,AI--i8G#1E,,5::,>K B""22 %%55//3}}oQ@  s&D1B6AD6A DDcURS5(a,URS5n[R"U5nUS$U$)zWhen streaming is turned on, llamafile server returns lines like: 'data: {"content":" They","multimodal":true,"slot_id":0,"stop":false}' Here, we convert this to a dict and return the value of the 'content' field zdata:zdata: rc) startswithlstripr_loads)r2rocleaneddatas r3ryLlamafile._get_chunk_content1s@   G $ $ll8,G::g&D ? "Lr6r1)returnr)rz List[str])rDict[str, Any]rG)rNOptional[List[str]]rRrrr)NN) rXrrNrrW"Optional[CallbackManagerForLLMRun]rRrrr) rXrrNrrWrrRrrzIterator[GenerationChunk])rorrr)%__name__ __module__ __qualname____firstlineno____doc__r__annotations__rrrrrrrr r!r"r#r$r&r'r(r)r*r+r,r model_configpropertyr4rDrKrTrrrdry__static_attributes__r1r6r3rrs0*,Hc+;%)O])%ItJ D#N#K#E3OE57E5@Is< FCO E5NIu  NEM3$K"e!A"u"AHcL%GL%FL.+/':= .%):> ::":8 :  : :~%):> DD"D8 D  D # DLr6r) __future__rr_iortypingrrrrr rh langchain_core.callbacks.managerr #langchain_core.language_models.llmsr langchain_core.outputsr langchain_core.utilsr pydanticrrr1r6r3rs4" 66E329ppr6