tj!, ddlmZddlmZddlmZmZmZmZm Z ddl m Z m Z ddl mZmZddlmZerddlmZmZGd d ed ZGd ded ZeedGddeZdgZdS)) BatchFeature) ImageInput)MultiModalDataProcessingKwargsProcessorMixin TextKwargsUnpack)PreTokenizedInput TextInput)auto_docstringis_vision_available)requires)Emu3ImageProcessorKwargs smart_resizeceZdZUdZeed<dS)Emu3TextKwargsa return_for_image_generation (`bool`, *optional*, defaults to `False`): Whether the processed text is intended for image generation tasks. When `True`, the processor prepares inputs for image generation by appending image start tokens and size information to the prompt, and images should not be provided. When `False`, the processor prepares inputs for text generation from images and text, requiring both inputs to be provided. return_for_image_generationN)__name__ __module__ __qualname____doc__bool__annotations__f/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/models/emu3/processing_emu3.pyrrs*"&%%%%%rrF)totalc:eZdZUeed<eed<dddddddZd S) Emu3ProcessorKwargs text_kwargs images_kwargsF)rreturn_mm_token_type_idsz1:1i)ratio image_area)r!r"N)rrrrrr _defaultsrrrr r )sX++++,1(-      IIIrr )vision)backendsc eZdZ d fd Ze ddedzdeezeezeezdzde e de fdZ d dZ d Zdefd Z dd ZxZS) Emu3ProcessorNc |j|_|j|_|j|_|j|_|j|_|j|_|j |_ d|_ t |||dS)N) chat_template) image_tokenimage_token_id boi_tokenimage_start_token eoi_tokenimage_end_tokenimage_wrapper_tokenfake_token_around_image eof_token bos_tokendownsample_ratiosuper__init__)selfimage_processor tokenizerr-kwargs __class__s rr:zEmu3Processor.__init__;s}%0'6!*!4(2'0'D$",", ! )=QQQQQrimagestextr>returnc t|tr|g}n?t|ts*t|dtstdjt fdjji|}|ddd}|ddd }|dd d }|r|td |s||td i}j } j j } |s|j |fi|d}t|j} g} |D]} j| vrwt#| }|\}}|jz}|jz}||d zz}| |d|jd|z| }| j|d } j| } j| vw| | fd| D}nX|rV||j\}}| |d|jfd|D}||ggt1|z|d<|ddd }|ddd}j|fi|ddd i}||dg|r|d|d<t7i|||S)aA Returns: [`BatchFeature`]: A [`BatchFeature`] with the following fields: - **input_ids** -- List of token ids to be fed to a model. Returned when `text` is not `None`. - **attention_mask** -- List of indices specifying which tokens should be attended to by the model (when `return_attention_mask=True` or if *"attention_mask"* is in `self.model_input_names` and if `text` is not `None`). - **pixel_values** -- Pixel values to be fed to a model. Returned when `images` is not `None`. zAInvalid input text. Please provide a string, or a list of stringstokenizer_init_kwargsr!rFr"r$Nr%zGYou should not provide `images` when `return_for_image_generation=True`zOYou must provide either text or images when `return_for_image_generation=False`r* cFg|]}|djS)rG)replacer.).0sampler;s r z*Emu3Processor.__call__..s*ccc&FNN?D4DEEcccrc*g|]}j|Sr)r7)rJrK image_promptr;s rrLz*Emu3Processor.__call__..s+QQQ&t~=v=|==QQQr image_sizesreturn_tensorsr#image) modalities input_idsmm_token_type_ids)data tensor_type) isinstancestrlist TypeError _merge_kwargsr r= init_kwargspop ValueErrorr1r6r3r<iterrOr.nextr8r5rIr7appendcalculate_generate_sizelen_check_special_mm_tokenscreate_mm_token_type_idsr)r;r@rAr> output_kwargsrr$r%image_featuresimage_start_tokensimage_end_tokensrOprompt_stringsrK image_sizeheightwidthimage_seq_lengthimage_placeholderrPr# text_inputsrNs` @r__call__zEmu3Processor.__call__Ls& dC  a6DDD$'' a 47C0H0H a_`` `**   "&."<   '4M&B&F&FGdfk&l&l#o.227DAA"?377 dKK & h6+=fgg g* pt|noo o $ 68"nDd.BDD+ Jv/A1T1&[[M/OQRSSF $888F&&00%%f----ccccTbcccDD) J 88 DLabbMFE0`&``5`$B^``LQQQQQDQQQD.4e_,=D ,IN= )'}599:JDQQ#0#?#C#CD^`e#f#f $dnT__]=-I__Z^___  %%dKWI%NNN # g/3/L/L[YdMe/f/fK+ ,!BK!B>!BP^____rc Vi}|g}|D]j\}}t|||jj|jj|jj\}}||jz}||jz}||dzz}||kdgt|z}|||dtdi|S)a Computes the number of placeholder tokens needed for multimodal inputs with the given sizes. Args: image_sizes (`list[list[int]]`, *optional*): The input sizes formatted as (height, width) per each image. Returns: `MultiModalData`: A `MultiModalData` object holding number of tokens per each of the provided input modalities, along with other useful data. Nr)num_image_tokensnum_image_patchesr) rr<spatial_factor min_pixels max_pixelsr8rarcupdater) r;rOr> vision_datarsrlrmrnrts r_get_num_multimodal_tokensz(Emu3Processor._get_num_multimodal_tokenss  "! !, : :  ,(7(3(3 !!  4#88!66#)UQY#7  ''(89999!"c+&6&6 6    4D[lmm n n n,, ,,,rctt|d\}}||z}||z dz}tt||z|z }tt||z|z } || fS)N:g?)mapintsplitround) r;r$r%rurmrl current_area target_ratio token_height token_widths rrbz%Emu3Processor.calculate_generate_sizesCS!1!122 vv~ "\1c9 5,!6!GHHII % 4~ EFFGG [((rc (|jj|fi|SN)r< postprocess)r;r@r>s rrzEmu3Processor.postprocesss /t#/AA&AAArTc ||dkr|j|fd|i|S|dkr||d}|dSt|jjd|d ) a  Post-process the output of a multimodal model to return the requested modality output. If the model cannot generated the requested modality, an error will be raised. Args: generated_outputs (`torch.Tensor` or `np.ndarray`): The output of the model `generate` function. The output is expected to be a tensor of shape `(batch_size, sequence_length)` or `(sequence_length,)`. skip_special_tokens (`bool`, *optional*, defaults to `True`): Whether or not to remove special tokens in the output. Argument passed to the tokenizer's `batch_decode` method. generation_mode (`str`, *optional*): Generation mode indicated which modality to output and can be one of `["text", "image", "audio"]`. **kwargs: Additional arguments to be passed to the tokenizer's `batch_decode method`. Returns: `list[Union[str, PIL.Image.Image]]`: The decoded text or generated image. NrAskip_special_tokensrQzPIL.Image.Image)rP pixel_valuesz# got an unexpected generation_mode=z.. Supported options are only `text` and `image)post_process_image_text_to_textrr^r?r)r;generated_outputsrgeneration_moder>r@s rpost_process_multimodal_outputz,Emu3Processor.post_process_multimodal_outputs*  "o&?&?747!7JNT  ' '%%&7HY%ZZF.) )>*OOOOO rr)NN)TN)rrrr:r rr r rYr r rrqrzrbrr __classcell__)r?s@rr*r*8s8 RRRRRR"%)aeM`M`T!M`++d9o=EV@WWZ^^M`,- M`  M`M`M`^M`^ - - - -D)))B*BBBBLP!!!!!!!!rr*N)image_processing_utilsr image_utilsrprocessing_utilsrrrrr tokenization_utils_baser r utilsr r utils.import_utilsrimage_processing_emu3rrrr r*__all__rrrrs"322222%%%%%%ddddddddddddddCCCCCCCC88888888******NMMMMMMMM & & & & &Zu & & & &     *%     ;qqqqqNqq qh  r