dhg$SSKJr SSKJrJrJrJr SSKJr SSK J r SSK J r SSK JrJr SSKJr "SS \\5r"S S \5r"S S \ 5rg))Enum)AnyIteratorListOptional)CallbackManagerForLLMRun)LLM)GenerationChunk) BaseModel ConfigDict)enforce_stop_tokensc \rSrSrSrSrSrSrg)Device z,The device to use for inference, cuda or cpucudacpuN)__name__ __module__ __qualname____firstlineno____doc__rr__static_attributes__r^/var/www/html/shao/venv/lib/python3.13/site-packages/langchain_community/llms/titan_takeoff.pyrr s6 D Crrc\rSrSr%Sr\"SS9r\\S'\ Rr \ \S'Sr \\S'S r \\\S 'S r\\S 'S r\\S'Srg ) ReaderConfigzAConfiguration for the reader to be deployed in Titan Takeoff API.r)protected_namespaces model_namedeviceprimaryconsumer_groupNtensor_parallelimax_seq_lengthmax_batch_size)rrrrrr model_configstr__annotations__rrr!r#r$rintr%r'rrrrrrseKLO&[[FF 6#NC#5%)OXc])INCKNC@rrc 8^\rSrSr%SrSr\\S'Sr\ \S'Sr \ \S'S r \ \S 'S r \\S 'SSSS /4S\S\ S\ S \ S \\4 U4Sjjjr\S\4Sj5rSS\S\\\S\\S\S\4 SjjrSS\S\\\S\\S\S\\4 SjjrSrU=r$) TitanTakeoff-aTitan Takeoff API LLMs. Titan Takeoff is a wrapper to interface with Takeoff Inference API for generative text to text language models. You can use this wrapper to send requests to a generative language model and to deploy readers with Takeoff. Examples: This is an example how to deploy a generative language model and send requests. .. code-block:: python # Import the TitanTakeoff class from community package import time from langchain_community.llms import TitanTakeoff # Specify the embedding reader you'd like to deploy reader_1 = { "model_name": "TheBloke/Llama-2-7b-Chat-AWQ", "device": "cuda", "tensor_parallel": 1, "consumer_group": "llama" } # For every reader you pass into models arg Takeoff will spin # up a reader according to the specs you provide. If you don't # specify the arg no models are spun up and it assumes you have # already done this separately. llm = TitanTakeoff(models=[reader_1]) # Wait for the reader to be deployed, time needed depends on the # model size and your internet speed time.sleep(60) # Returns the query, ie a List[float], sent to `llama` consumer group # where we just spun up the Llama 7B model print(embed.invoke( "Where can I see football?", consumer_group="llama" )) # You can also send generation parameters to the model, any of the # following can be passed in as kwargs: # https://docs.titanml.co/docs/next/apis/Takeoff%20inference_REST_API/generate#request # for instance: print(embed.invoke( "Where can I see football?", consumer_group="llama", max_new_tokens=100 )) zhttp://localhostbase_urli porti mgmt_portF streamingNclientmodelsc>[TU]XX4S9 SSKJn U"UR UR URS9UlUHnURRU5 M g![a [ S5ef=f)aInitialize the Titan Takeoff language wrapper. Args: base_url (str, optional): The base URL where the Takeoff Inference Server is listening. Defaults to `http://localhost`. port (int, optional): What port is Takeoff Inference API listening on. Defaults to 3000. mgmt_port (int, optional): What port is Takeoff Management API listening on. Defaults to 3001. streaming (bool, optional): Whether you want to by default use the generate_stream endpoint over generate to stream responses. Defaults to False. In reality, this is not significantly different as the streamed response is buffered and returned similar to the non-streamed response, but the run manager is applied per token generated. models (List[ReaderConfig], optional): Any readers you'd like to spin up on. Defaults to []. Raises: ImportError: If you haven't installed takeoff-client, you will get an ImportError. To remedy run `pip install 'takeoff-client==0.4.0'` )r/r0r1r2r) TakeoffClientzjtakeoff-client is required for TitanTakeoff. Please install it with `pip install 'takeoff-client>=0.4.0'`.)r0r1N) super__init__takeoff_clientr6 ImportErrorr/r0r1r3 create_reader) selfr/r0r1r2r4r6model __class__s rr8TitanTakeoff.__init__os< I    4 $ MM T^^  E KK % %e , P  s A''A=returncg)zReturn type of llm. titan_takeoffr)r<s r _llm_typeTitanTakeoff._llm_typesrpromptstop run_managerkwargsc UR(a*SnURUUUS9HnXVR- nM U$URR"U40UD6nUSnUb [ X5nU$)aECall out to Titan Takeoff (Pro) generate endpoint. Args: prompt: The prompt to pass into the model. stop: Optional list of stop words to use when generating. run_manager: Optional callback manager to use when streaming. Returns: The string generated by the model. Example: .. code-block:: python model = TitanTakeoff() prompt = "What is the capital of the United Kingdom?" # Use of model(prompt), ie `__call__` was deprecated in LangChain 0.1.7, # use model.invoke(prompt) instead. response = model.invoke(prompt) )rErFrGtext)r2_streamrKr3generater ) r<rErFrGrH text_outputchunkresponserKs r_callTitanTakeoff._calls: >>K'& zz)   ;;''9&9  &t2D rc+:# URR"U40UD6nSnUHnXgR- nSU;dMURS5(aSn[ UR SS55S:Xa%UR SS5upUR S5nU(dM~[US9n SnU(aURU RS9 U v M U(a>[URSS5S9n U(aURU RS9 U v g g 7f) a:Call out to Titan Takeoff (Pro) stream endpoint. Args: prompt: The prompt to pass into the model. stop: Optional list of stop words to use when generating. run_manager: Optional callback manager to use when streaming. Yields: A dictionary like object containing a string token. Example: .. code-block:: python model = TitanTakeoff() prompt = "What is the capital of the United Kingdom?" response = model.stream(prompt) # OR model = TitanTakeoff(streaming=True) response = model.invoke(prompt) rJzdata: )rK)tokenzN) r3generate_streamdata startswithlensplitrstripr on_llm_new_tokenrKreplace) r<rErFrGrHrPbufferrKcontent_rOs rrLTitanTakeoff._streams@;;..v@@D ii F& $$W--Fv||GQ/0A5!'gq!9JG$^^D1F6+8EF"#445::4FK$ #)CDE,,5::,>K s7DA!D"A9D)r3)NN)rrrrrr/r)r*r0r+r1r2boolr3rrrr8propertyrCrrrQrr rLr __classcell__)r>s@rr-r--s[0d'Hc&WD#EIsPIt8FCE+%' ,-,-,- ,-  ,- \" ,-,-\3%):> ,,tCy!,67 ,  , ,b%):> 88tCy!867 8  8 / " 88rr-N)enumrtypingrrrrlangchain_core.callbacksr#langchain_core.language_models.llmsr langchain_core.outputsr pydanticr r langchain_community.llms.utilsr r)rrr-rrrrnsG00=32*>S$A9A4[3[r