tjf ddlZddlZddlZddlZddlmZddlmZddlm Z ddl m Z ddl m Z ddlm Z ddlmZmZmZmZmZmZmZdd lmZd d lmZd d lmZmZmZejeZ er%ddl!Z!ddl"Z!ddl#Z!ddl$Z!ddl%Z!ddl&Z!dd l#m'Z'ddl(m)Z)ndZ!erddl*m+Z+e,e-e.efZ/e0e1e.e2ee3ddfZ4ej5dej6Z7ej5dej6Z8ej5dej6ej9zZ:ej5dej6Z;Gdde<Z=Gdde<Z>de3de-e.e.ffdZ?de.de-fdZ@dede-fdZAd e.deBe.dze-dze.dzffd!ZCdede-fd"ZDe ed#e.dzdeEe.fd$ZFd%ZGe d&ZHed'ZI d3d)e,e/d*e,e-ezdzd+e/dzd#e.dzd,e2d-e2e.zd.e2de.fd/ZJd0ZKGd1d2ZLdS)4N)Callable)contextmanager)deepcopy)datetime) lru_cache) isfunction)AnyLiteralUnionget_args get_originget_type_hints no_type_check)version)logging)is_jinja_availableis_torch_availableis_vision_available) Extension)ImmutableSandboxedEnvironment)Image.z(^(.*?)[\n\s]*(Args:|Returns:|Raises:|\Z)z0\n\s*Args:\n\s*(.*?)[\n\s]*(Returns:|Raises:|\Z)a1 (?:^|\n) # Match the start of the args block, or a newline \s*(\w+):\s* # Capture the argument name and strip spacing (.*?)\s* # Capture the argument description, which can span multiple lines, and strip trailing spacing (?=\n\s*\w+:|\Z) # Stop when you hit the next argument or the end of the block z*\n\s*Returns:\n\s*(.*?)[\n\s]*(Raises:|\Z)ceZdZdZdS)TypeHintParsingExceptionzJException raised for errors in parsing type hints to generate JSON schemasN__name__ __module__ __qualname____doc__d/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/utils/chat_template_utils.pyrrHTTTTr!rceZdZdZdS)DocstringParsingExceptionzJException raised for errors in parsing docstrings to generate JSON schemasNrr r!r"r%r%Lr#r!r% param_typereturnc tdditdditdditddit dddit ii}t r ddi|t<trddl }dd i||j <| |dd iS) Ntypeintegernumberstringbooleannullimageraudioobject) intfloatstrboolr)r rrrtorchTensorget)r& type_mappingr6s r"_get_json_schema_typer:Ps fi  ! fh  vy! T VV$ R L0%w/ U7 &,g%6 U\"   J(: ; ;;r!hintct|}t|}|. t|S#t$rt d|wxYw|t us#t tdr|tjuryd|D}t|dkr |d}n9td|Drdtd|Di}nd |i}td|vrd |d <|S|turt|dkrttt t"tdf}g}|D]l}t||vrt d tt|d}|||vr||mt|dkr|nt+|t+|d S|t*ur|sddiSdt-|ddS|t.uro|sddiSt|dkr4t dt!|dddd|vrt ddd|DdS|t2ur1ddi} t|dkrt-|d| d<| St d|)NzGCouldn't parse this type hint, likely due to a custom class or object: UnionTypecNg|]"}|tdut|#SN)r)_parse_type_hint.0ts r" z$_parse_type_hint..ps1MMMA$t**9L9L$Q''9L9L9Lr!rrc3TK|]#}d|vot|dtV$dS)r)N) isinstancer4rBsubtypes r" z#_parse_type_hint..ts:``G7"Gz'&/3'G'G``````r!r)cg|] }|d S)r)r rGs r"rDz$_parse_type_hint..vs*S*S*Sw76?*S*S*Sr!anyOfTnullablez?Only the valid python literals can be listed in typing.Literal.)r)enumarray)r)itemszThe type hint ztyping.a1 is a Tuple with a single element, which we do not automatically convert to JSON schema as it is rarely necessary. If this input can contain more than one element, we recommend using a list[] type instead, or if it really is a single element, remove the tuple[] wrapper and just pass the element directly..znConversion of '...' is not supported in Tuple type hints. Use list[] types for variable-length inputs instead.c,g|]}t|Sr )r@rAs r"rDz$_parse_type_hint..s!0S0S0S1A!1D1D0S0S0Sr!)r) prefixItemsr1additionalProperties)r r r:KeyErrorrr hasattrtypesr=lenallsortedr)r r2r3r4r5r8appendpoplistr@tuplereplacedict) r;originargssubtypes return_dict LITERAL_TYPES args_typesargarg_typeouts r"r@r@bsL   F D>>D ~ (.. .   *Y[_   5WUK88Vu=V=VMMMMM x==A  "1+KK ``W_``` ` ` .!6*S*S(*S*S*S#T#TUKK#H-K ::  &*K # 7  s4yy1}}eS$T ;   , ,CCyy --./pqqq,T#YY77;;FCCH# (B(B!!(+++(+J1(<(tAw.G.GHH H 5 %G$ $ t99>>*-T!2!29b!A!A---  $;;*#   0S0Sd0S0S0STTT 4x  t99>>*:47*C*CC& ' "#lnr s sss 1A funcct|}tj|}t|dd}t t |jd}|dvr(|j|jtjj kr|}nd}g}|j D]o\}}||kr |jtjj krtd|j d||j tjj kr||pi} | D]\}} ||kr t| | |<d| d} |r|| d<| S) Nr operation>clsselfz Argument z$ is missing a type hint in function r1)r) propertiesrequired)rinspect signaturegetattrnextiter parameters annotation ParameteremptyrOrnamedefaultr[r@) rj type_hintsrr func_namefirst_param_nameimplicit_arg_namerp param_nameparamror&schemas r""_convert_type_hints_to_json_schemars%%J!$''Ij+66ID!566==O++  !1 2 =ARAX X X, H&17799(( E * * *   w06 6 6*+ruz+r+rgp+r+rss s =G-3 3 3 OOJ ' ' 'J","2"2"4"4>> J * * * !1*!=!= :j 9 9F&%z Mr! docstringcdt|}t|}t|}|r'|dnd}|r'|dnd}|r'|dnd}|Ydd|dD}t |}d|D}ni}|||fS)a Parses a Google-style docstring to extract the function description, argument descriptions, and return description. Args: docstring (str): The docstring to parse. Returns: The function description, arguments, and return description. rN c:g|]}||Sr )strip)rBlines r"rDz1parse_google_format_docstring..s*#`#`#`TSWS]S]S_S_#`D#`#`#`r!c xi|]7}|dtjdd|d8S)rz \s*\n+\s* r)resubr)rBmatchs r" z1parse_google_format_docstring..s:```uU1XrvlCq9I9IJJ```r!) description_researchargs_re returns_regrouprjoinsplit args_split_refindall) rdescription_match args_match returns_match descriptiondocstring_argsreturnsmatches args_dicts r"parse_google_format_docstringrs,'--i88 **J%%i00M9JS#))!,,22444tK4>HZ%%a((..000DN0=Gm!!!$$**,,,4G!#`#`^5I5I$5O5O#`#`#`aa''77``X_```   7 **r!ctj|}t|dd}|std|d|}t |\}}}t |}|dddx}|||d<|dD]\}} ||vrtd|d |d ||} tj d | tj } | rgd tj | dD| d<| jd| } | | d<|||d} ||| d<d| dS)aO This function generates a JSON schema for a given function, based on its docstring and type hints. This is mostly used for passing lists of tools to a chat template. The JSON schema contains the name and description of the function, as well as the names, types and descriptions for each of its arguments. `get_json_schema()` requires that the function has a docstring, and that each argument has a description in the docstring, in the standard Google docstring format shown below. It also requires that all user-facing arguments have valid Python type hints. When passing methods, implicit receiver arguments (`self` or `cls`) are ignored. Although it is not required, a `Returns` block can also be added, which will be included in the schema. This is optional because most chat templates ignore the return value of the function. Args: func: The function to generate a JSON schema for. Returns: A dictionary containing the JSON schema for the function. Examples: ```python >>> def multiply(x: float, y: float): >>> ''' >>> A function that multiplies two numbers >>> >>> Args: >>> x: The first number to multiply >>> y: The second number to multiply >>> ''' >>> return x * y >>> >>> print(get_json_schema(multiply)) { "name": "multiply", "description": "A function that multiplies two numbers", "parameters": { "type": "object", "properties": { "x": {"type": "number", "description": "The first number to multiply"}, "y": {"type": "number", "description": "The second number to multiply"} }, "required": ["x", "y"] } } ``` The general use for these schemas is that they are used to generate tool descriptions for chat templates that support them, like so: ```python >>> from transformers import AutoTokenizer >>> from transformers.utils import get_json_schema >>> >>> def multiply(x: float, y: float): >>> ''' >>> A function that multiplies two numbers >>> >>> Args: >>> x: The first number to multiply >>> y: The second number to multiply >>> return x * y >>> ''' >>> >>> multiply_schema = get_json_schema(multiply) >>> tokenizer = AutoTokenizer.from_pretrained("CohereForAI/c4ai-command-r-v01") >>> messages = [{"role": "user", "content": "What is 179 x 4571?"}] >>> formatted_chat = tokenizer.apply_chat_template( >>> messages, >>> tools=[multiply_schema], >>> chat_template="tool_use", >>> return_dict=True, >>> return_tensors="pt", >>> add_generation_prompt=True >>> ) >>> # The formatted chat can now be passed to model.generate() ``` Each argument description can also have an optional `(choices: ...)` block at the end, such as `(choices: ["tea", "coffee"])`, which will be parsed into an `enum` field in the schema. Note that this will only be parsed correctly if it is at the end of the line: ```python >>> def drink_beverage(beverage: str): >>> ''' >>> A function that drinks a beverage >>> >>> Args: >>> beverage: The beverage to drink (choices: ["tea", "coffee"]) >>> ''' >>> pass >>> >>> print(get_json_schema(drink_beverage)) ``` { 'name': 'drink_beverage', 'description': 'A function that drinks a beverage', 'parameters': { 'type': 'object', 'properties': { 'beverage': { 'type': 'string', 'enum': ['tea', 'coffee'], 'description': 'The beverage to drink' } }, 'required': ['beverage'] } } rrlz Cannot generate JSON schema for z because it has no docstring!ror'Nrz< because the docstring has no description for the argument ''z\(choices:\s*(.*?)\)\s*$)flagscdg|]-}t|tr|n|.Sr )rFr4r)rBcs r"rDz#get_json_schema..vs3ppp:a+=+=Daggiii1pppr!rrM)rzrrvfunction)r)r)rqgetdocrsr%rrrr\rOrr IGNORECASEjsonloadsrr,start) rjdocr}main_docparam_descriptions return_doc json_schemardrgrdesc enum_choicesoutputs r"get_json_schemarsX .  Cj+66I u'(s9(s(s(sttt ))++C/LS/Q/Q,H *4T::K"<044XtDDD Q  !)3K &"<06688 % % V ( ( (+A9AAz}AAA "#&y!DDFFD $} T TF&xF 3 33r! chat_templatec|tSt|}|j|}ttj|S)acReturn the set of undeclared variables referenced by a chat template. Uses ``jinja2.meta.find_undeclared_variables`` so that callers can automatically distinguish template-level kwargs from processor kwargs without maintaining a manual allowlist. Needed only to support BC as we allowed all `kwargs` to be merged into one in the past ) frozenset_compile_jinja_template environmentparsejinja2metafind_undeclared_variables)rcompiledasts r"_get_template_variablesrsU{{&}55H   $ $] 3 3C V[::3?? @ @@r!c g}g}|j||5|jd||||d|D]}||d|} dddn #1swxYwY| |fS)Nmessagestools documentsadd_generation_promptrPr )ractivate_trackergenerater[r) compiled_templaterrrrtemplate_kwargsrendered_blocksgeneration_indicesblock rendered_chats r"_render_with_assistant_indicesrsO  & 7 7I[ \ \ 1 1/&/ "7       * *E  " "5 ) ) ) )00  1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 , ,,sAA,,A03A0c t|Sr?)_cached_compile_jinja_template)rs r"rrs )- 8 88r!ctstdGddt}tjt jtjdkrtdt jdd}dd }d }td d |t jj g }||j d <||j d<||j d<| |S)Nzbapply_chat_template requires jinja2 to be installed. Please install it using `pip install jinja2`.ceZdZdhZdeffd Zdejjdej j fdZ ej dej j dejjdefd Zdefd Zed eed eefd ZxZS)8_cached_compile_jinja_template..AssistantTracker generationrct|||jd|_d|_dS)N)r)super__init__extendr_rendered_blocks_generation_indices)rnr __class__s r"rzA_cached_compile_jinja_template..AssistantTracker.__init__sK GG  [ ) ) )   0E  F F F$(D !'+D $ $ $r!parserr'ct|jj}|dgd}tj|dgg||S)Nzname:endgenerationT) drop_needle_generation_support) rtstreamlinenoparse_statementsrnodes CallBlock call_method set_lineno)rnrrbodys r"rz>_cached_compile_jinja_template..AssistantTracker.parsesm&-((/F**,@+At*TTD<))$*:*:;P*Q*QSUWY[_``kklrss sr!contextcallerc|}|rUtd|j}|t|z}|j||f|S)NrP) is_activerXrrrr[)rnrrrv start_index end_indexs r"rzL_cached_compile_jinja_template..AssistantTracker._generation_supportsmB~~ J!"''$*?"@"@AA '#b''1 (//i0HIIIIr!c&|jdup|jduSr?)rr)rns r"rzB_cached_compile_jinja_template..AssistantTracker.is_actives(4\8PX\8\ \r!rrc3K |rtd||_||_dVd|_d|_dS#d|_d|_wxYw)Nz3AssistantTracker should not be reused before closed)r ValueErrorrr)rnrrs r"rzI_cached_compile_jinja_template..AssistantTracker.activate_trackers{ 0>>##\$%Z[[[(7%+=((,%+/((()-%+/(////s 5A A)rrrtagsrrrrParserrrrpass_eval_context EvalContextruntimeMacror4rr5rrr]r2r __classcell__)rs@r"AssistantTrackerrs~ ,(E , , , , , , t 4 t9O t t t t  ! v|/G QWQ_Qe jm    " !  ]t ] ] ] ]  0DI 0SWX[S\ 0 0 0  0 0 0 0 0r!rz3.1.0zLapply_chat_template requires jinja2>=3.1.0 to be installed. Your version is .c@tj|r?)r exceptions TemplateErrormessages r"raise_exceptionz7_cached_compile_jinja_template..raise_exceptions--g666r!Fc4tj|||||S)N) ensure_asciiindent separators sort_keys)rdumps)xrrrrs r"tojsonz._cached_compile_jinja_template..tojsons!z!,vR\hqrrrrr!cNtj|Sr?)rnowstrftime)formats r" strftime_nowz4_cached_compile_jinja_template..strftime_nows|~~&&v...r!T) trim_blocks lstrip_blocks extensionsr rr)FNNF) r ImportErrorrrrr __version__rext loopcontrolsfiltersglobals from_string)rrrr r jinja_envs r"rrs4     p   (0(0(0(0(09(0(0(0T}V'((7=+A+AAA p[a[m p p p   777ssss ///.:JFJLc9dI#)Ih+:I'((4In%   / //r!F conversationsrrreturn_assistant_tokens_maskcontinue_final_messagerc 2|r/tjd|stdt |}|g} |D]} t | t r| | -t| stj | r#| t| stdnd} |)|D]&} t | t std'g} g} d}|D]}t|dr|j}|rt!|}t |t"r|nd}|d|x}td |d ||vrtd |d t |t&t(fr>t+|D]}d |vr|d }|d |z|d <ntdn|d||z|d|<|r+t-d||| ||d|\}}| |n|jd|| ||d|}|r||vs||vr'td|d|||}|||t5|z|kr |d|}n|d|}| || | fS)Nz\{\%-?\s*generation\s*-?\%\}zareturn_assistant_tokens_mask==True but chat template does not contain `{% generation %}` keyword.zTools should either be a JSON schema, or a callable function with type hints and a docstring suitable for auto-conversion to a schema.zADocuments should be a list of dicts with 'title' and 'text' keys!zCONTINUE_FINAL_MESSAGE_TAG rcontentz>    D$%% ##D))))D!! W%5d%;%; ##OD$9$9:::: P  ! e eHh-- e cddd eH!>B'B' 4 $ $ !=D ! qD>>D?IJ`be?f?f%u%;%;lu "!%b.D!E!EE N ySiyyy&]:: J9OJJJMD%=99 q%-m%<%<  M..(5f(= 0=f0EHb0b f- / %x 488 > @ @  &&&& + ++r!cHt|tsdSd|vrd|vsdSdS)zf Check that input is a valid message in a chat, namely a dict with "role" and "content" keys. FrolerT)rFr`rs r"is_valid_messager8cs: gt $ $u g  )w"6"6u 4r!ceZdZdZdefdZdS)ChataThis class is intended to just be used internally for pipelines and not exposed to users. We convert chats to this format because the rest of the pipeline code tends to assume that lists of messages are actually a batch of samples rather than messages in the same conversation.rcZ|D] }t|std!||_dS)NzQWhen passing chat dicts as input, each dict must have a 'role' and 'content' key.)r8rr)rnrrs r"rz Chat.__init__ssD v vG#G,, v !tuuu v  r!N)rrrrr`rr r!r"r:r:ns;RR!!!!!!!r!r:)NNNFFF)MrqrrrWcollections.abcr contextlibrcopyrr functoolsrrtypingr r r r r rr packagingrrPr import_utilsrrr get_loggerrr"rjinja2.exceptions jinja2.ext jinja2.meta jinja2.nodesjinja2.runtimerjinja2.sandboxr PIL.Imagerr]r`r4ChatTyper2r3r5r) BASIC_TYPEScompileDOTALLrrVERBOSErr Exceptionrr%r:r@rr^rrrrrrrr5r8r:r r!r"rQs $$$$$$%%%%%%[[[[[[[[[[[[[[[[[[UUUUUUUUUU  H % % MMM$$$$$$<<<<<<< F  S#X E3c44::s; GSS "*H") T T  I  RZEry Q Q UUUUUyUUUUUUUU UUUn, x 4 'n,$n,: n, #' n, !3J n, n, n,n,n,n,b ! ! ! ! ! ! ! ! ! !r!