tj,tdZddlZddlmZddlmZmZddlm Z m Z m Z m Z m Z ddlmZmZmZddlmZmZejeZd Zd ed Dd ed DzZGdde ZGdde dZdefdZdZdZ dZ!eGdde Z"dgZ#dS)z Processor class for PaliGemma. N) BatchFeature) ImageInputis_valid_image)MultiModalDataProcessingKwargsProcessorMixin TextKwargsUnpack) AddedTokenPreTokenizedInput TextInput)auto_docstringloggingzcg|] }d|dd S)z4>.0is p/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/models/paligemma/processing_paligemma.py r$s"555Aq555icg|] }d|dd S)z3rrrs rrr$s"8]8]8]Q8]8]8]rcNeZdZUdZeezeezeezdzed<dS)PaliGemmaTextKwargsa suffix (`str`, `list[str]`, `list[list[str]]`): The suffixes or batch of suffixes to be encoded. Only necessary for finetuning. See https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/paligemma/README.md for more information. If your prompt is " What is on the image", the suffix corresponds to the expected prediction "a cow sitting on a bench". Nsuffix)__name__ __module__ __qualname____doc__rr list__annotations__rrrrr'sJ ) )DO ;dCT>U UX\ \\\\\\rrc.eZdZUeed<dddddidZdS)PaliGemmaProcessorKwargs text_kwargsF)paddingreturn_mm_token_type_ids data_formatchannels_first)r' images_kwargsN)rr r!rr$ _defaultsrrrr&r&1sG$$$$(-  + IIIrr&F)totalreturncVt|to|dS)Nhttp) isinstancestr startswith)vals ris_urlr6?s# c3   :CNN6$:$::rc>t|pt|SN)r6relems ris_image_or_image_urlr;Ds $<< />$///rcJt|tpt|Sr8)r2r3r;r9s r_is_str_or_imager=Hs dS " " A&;D&A&AArc ||z|z||dS)aZ Builds a string from the input prompt and image tokens. For example, for the call: build_string_from_input( prompt="Prefix str" bos_token="", image_seq_len=3, image_token="", ) The output will be: "Initial str" Args: prompt (`list[Union[str, ImageInput]]`): The input prompt. bos_token (`str`): The beginning of sentence token. image_seq_len (`int`): The length of the image sequence. image_token (`str`): The image token. num_images (`int`): Number of images in the prompt.  rprompt bos_token image_seq_len image_token num_imagess rbuild_string_from_inputrFLs'&M)J6 M M6 M M MMrc eZdZ d fd Ze d dedzdeezeezeezde e de fdZ d dZ ed ZxZS) PaliGemmaProcessorNc t|dstd|j|_t|ds]ttdd}d|gi}|||t|_t|_n|j|_|j|_| td|_ d|_ t|||dS) Nimage_seq_lengthz;Image processor is missing an `image_seq_length` attribute.rDFT) normalizedspecialadditional_special_tokens) chat_template)hasattr ValueErrorrJr IMAGE_TOKENadd_special_tokensconvert_tokens_to_idsimage_token_idrD add_tokens EXTRA_TOKENS add_bos_token add_eos_tokensuper__init__)selfimage_processor tokenizerrNkwargsrD tokens_to_add __class__s rrZzPaliGemmaProcessor.__init__ds(:;; \Z[[ [ / @y-00 5$[UDQQQK8;-HM  ( ( 7 7 7"+"A"A+"N"ND *D  "+":D (4D \***"' "'  )=QQQQQrimagestextr^r/c *jtfdjji|}|ddd}d}|t d|t dd}t|r|g}n+t|trt|d r |3|0td |Dskt d t|tret|trPt|t|kr0t d t|d t|dt|r|gg}nt|ttfr"t|d r d|D}nht|ttfr=t|d ttfrt|d d st dfdt!||D}ng}|D]} | t$t$jz} | t$} | dkr| tt$znd } | d| jjz| | dz} || d|D}|t|r|g}|fd|D}j|fi|dd} |ddd}|ddd}j|f||d|d}||dgi|d| i}|rRt3j|d}d|t3j|dd k<|d|i|r|d|d <t;||!S)"a Returns: [`BatchFeature`]: A [`BatchFeature`] with the following fields: - **input_ids** -- List of token ids to be fed to a model. Returned when `text` is not `None`. If `suffix` is provided, the `input_ids` will also contain the suffix input ids. - **attention_mask** -- List of indices specifying which tokens should be attended to by the model (when `return_attention_mask=True` or if *"attention_mask"* is in `self.model_input_names` and if `text` is not `None`). - **pixel_values** -- Pixel values to be fed to a model. Returned when `images` is not `None`. - **labels** -- Labels compatible with training if `suffix` is not None tokenizer_init_kwargsr'rNTzF`images` are expected as arguments to a `PaliGemmaProcessor` instance.z]You are using PaliGemma without a text prefix. It will perform as a picture-captioning model.rc3(K|] }t|vVdSr8)rQrsamples r z.PaliGemmaProcessor.__call__..s'@@{f,@@@@@@raLYou are passing both `text` and `images` to `PaliGemmaProcessor`. The processor expects special image tokens in the text, as many tokens as there are images per each text. It is recommended to add `` tokens in the very beginning of your text. For this call, we will infer how many images each text has and add special tokens.z Received z images for zK prompts. Each prompt should be associated with an image or list of images.cg|]}|gSrr)rimages rrz/PaliGemmaProcessor.__call__..s:::%ug:::rzAimages must be an image, list of images or list of list of imagesc g|]Q\}}t|jjjtt |t rt|ndRS)r@)rFr]rBrJrQr2r#len)rrA image_listr[s rrz/PaliGemmaProcessor.__call__..so ! ! !+ ,%"&.":&*&;$/6@T6R6R#Y3z???XY  ! ! !rcg|]}|dS)r?rrgs rrz/PaliGemmaProcessor.__call__..s N N N6F N N Nrc.g|]}|jjzSr)r] eos_token)rsfxr[s rrz/PaliGemmaProcessor.__call__..s#GGGcDN44GGGrr, pixel_valuesreturn_tensorsr)) text_pairreturn_token_type_idsrk) modalities input_idsitoken_type_idslabelsmm_token_type_ids)data tensor_type) _merge_kwargsr&r] init_kwargspoprPlogger warning_oncer=r2r#anywarningrnrtuplezipreplacerQrJrfindrBappendr\_check_special_mm_tokensnparrayupdatecreate_mm_token_type_idsr)r[rarbr^ output_kwargsrrx input_stringsexpanded_samplesrhexpanded_samplebos_rfind_index bos_indexrurvr)inputs return_datar|s` r__call__zPaliGemmaProcessor.__call__s(+* $  "&."<   }-11(DAA $ >eff f <   o   D D ! ! 6DD d # # (8a(A(A     2@@4@@@@@. O<dD))j.F.F6{{c$ii//(XF XXTXXX "&)) j%hZFFu 66j>&QR);T;Tj::6:::FFve}55j"6!9tUm<<j'vay|44j %%hiii ! ! ! !/2$.?.? ! ! ! $& "==F&,nn[+PTPeBe&f&fO&5&;&;K&H&HOFUY[F[F[#k2B2B B BabI'  3dn6NNQ`ajakakQll$%++O<<<< N N=M N N N  "26":": XF  GGGGGGGF+t+FUUmO6TUUVde &}599:JDQQ#0#?#C#CD^`d#e#e   "7  M*    %%mV %RRR>>>>  ! 3Xf[122F>BF28F#3455: ;   &1 2 2 2 # g/3/L/L[YdMe/f/fK+ ,.IIIIrc i}|C|jgt|z}dgt|z}|||dtdi|S)a Computes the number of placeholder tokens needed for multimodal inputs with the given sizes. Args: image_sizes (list[list[str]], *optional*): The input sizes formatted as (height, width) per each image. Returns: `MultiModalData`: A `MultiModalData` object holding number of tokens per each of the provided input modalities, along with other useful data. Nrm)num_image_tokensnum_image_patchesr)rJrnrr)r[ image_sizesr^ vision_datarrs r_get_num_multimodal_tokensz-PaliGemmaProcessor._get_num_multimodal_tokensso  " $ 56[9I9II !"c+&6&6 6    4D[lmm n n n,, ,,,rc`|jjddgz}|jj}t||zS)Nr{r|)r]model_input_namesr\r#)r[tokenizer_input_namesimage_processor_input_namess rrz$PaliGemmaProcessor.model_input_names s: $ @DTV^C_ _&*&:&L#),GGHHHr)NNN)NNr8)rr r!rZrrrr r#r r&rrrpropertyr __classcell__)r`s@rrHrHbs RRRRRR8%)Z^tJtJT!tJ++d9o=EV@WWtJ12 tJ  tJtJtJ^tJl----$IIXIIIIIrrH)$r"numpyrfeature_extraction_utilsr image_utilsrrprocessing_utilsrrr r r tokenization_utils_baser r rutilsrr get_loggerrrrQrangerVrr&boolr6r;r=rFrH__all__rrrrs"44444455555555POOOOOOOOO,,,,,,,,  H % % 55t5558]8]RWRWX[R\R\8]8]8]] ]]]]]*]]]     /u    ;4;;;; 000BBBNNN,jIjIjIjIjIjIjIjIZ r