tjdZddlmZddlmZddlmZmZmZm Z ddl m Z m Z ddl mZmZejeZdZGdd ed ZGd d ed ZeGddeZdgZdS)z Processor class for Janus. ) BatchFeature) ImageInput)ProcessingKwargsProcessorMixin TextKwargsUnpack)PreTokenizedInput TextInput)auto_docstringloggingzYou are a helpful language and vision assistant. You are able to understand the visual content that the user provides, and assist the user with a variety of tasks using natural language. ceZdZUdZeed<dS)JanusTextKwargsas generation_mode (`str`, *optional*, defaults to `"text"`): The generation mode indicating which modality to generate. Can be one of `"text"` or `"image"`. When set to `"text"`, the processor prepares inputs for text generation. When set to `"image"`, it prepares inputs for image generation by appending image start tokens to the prompt. generation_modeN)__name__ __module__ __qualname____doc__str__annotations__h/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/models/janus/processing_janus.pyrr"s*rrF)totalc0eZdZUeed<ddddddidZd S) JanusProcessorKwargs text_kwargsFlefttext)padding padding_siderreturn_tensorspt)r common_kwargsN)rrrrr _defaultsrrrrr-s>    #(&U[\\*D1IIIrrc eZdZeZ ddedzdedeffd Ze dde e ze e ze e zd e dzd eed effd Zd eded efdZd e fdZ ddZxZS)JanusProcessorNF@ chat_templateuse_default_system_promptnum_image_tokensc ||_|j|_|j|_|j|_||_t|||dS)a use_default_system_prompt (`bool`, *optional*, defaults to `False`): Use default system prompt for Text Generation. num_image_tokens (`int`, *optional*, defaults to `576`): The number of placeholder image tokens needed per one image. )r(N) r* image_token boi_tokenimage_start_token eoi_tokenimage_end_tokenr)super__init__)selfimage_processor tokenizerr(r)r*kwargs __class__s rr2zJanusProcessor.__init__9s[!1$0!*!4(2)B& )=QQQQQrrimagesr6returnc jtfdjji|}|dd}jr|dkr d|D}n|dkrfd|D}t jd ||d|}|S) aA Returns: [`BatchFeature`]: A [`BatchFeature`] with the following fields: - **input_ids** -- List of token ids to be fed to a model. Returned when `text` is not `None`. - **attention_mask** -- List of indices specifying which tokens should be attended to by the model (when `return_attention_mask=True` or if *"attention_mask"* is in `self.model_input_names` and if `text` is not `None`). - **pixel_values** -- Pixel values to be fed to a model. Returned when `images` is not `None`. tokenizer_init_kwargsrrrc$g|] }t|Sr)DEFAULT_SYSTEM_PROMPT).0samples r z+JanusProcessor.__call__..hs$JJJ6,6f66JJJrimagec&g|] }|jSr)r.)r>r?r3s rr@z+JanusProcessor.__call__..js'KKKFv7t577KKKr)r8rr) _merge_kwargsrr5 init_kwargspopr)r1__call__)r3rr8r6 output_kwargsr model_inputsr7s` rrFzJanusProcessor.__call__Ps$+*   8<8R V\  ( 6::;LMM  ) Lo.G.GJJTJJJDD  ' 'KKKKdKKKD'uww'RvDRRMRR r image_inputs image_idxcD|j|j|jzz|jz}|S)N)r.r,r*r0)r3rIrJone_img_tokenss rreplace_image_tokenz"JanusProcessor.replace_image_tokenos)/43CdF[3[\_c_ssrc (|jj|fi|S)z Forwards all arguments to the image processor's `postprocess` method. Refer to the original method's docstring for more details. )r4 postprocess)r3r8r6s rrOzJanusProcessor.postprocessss" 0t#/AA&AAArTc ||dkr|j|fd|i|S|dkr@t|}||d}|dSt |jjd|d ) a  Post-process the output of a multimodal model to return the requested modality output. If the model cannot generated the requested modality, an error will be raised. Args: generated_outputs (`torch.Tensor` or `np.ndarray`): The output of the model `generate` function. The output is expected to be a tensor of shape `(batch_size, sequence_length)` or `(sequence_length,)`. skip_special_tokens (`bool`, *optional*, defaults to `True`): Whether or not to remove special tokens in the output. Argument passed to the tokenizer's `batch_decode` method. generation_mode (`str`, *optional*): Generation mode indicated which modality to output and can be one of `["text", "image", "audio"]`. **kwargs: Additional arguments to be passed to the tokenizer's `batch_decode method`. Returns: `list[Union[str, PIL.Image.Image]]`: The decoded text or generated image. Nrskip_special_tokensrAzPIL.Image.Image)r! pixel_valuesz# got an unexpected generation_mode=z.. Supported options are only `text` and `image)post_process_image_text_to_textlistfloatrO ValueErrorr7r)r3generated_outputsrQrr6r8s rpost_process_multimodal_outputz-JanusProcessor.post_process_multimodal_outputzs*  "o&?&?747!7JNT  ' ' $%6%<%<%>%> ? ? %%&7HY%ZZF.) )>*OOOOO r)NFr')NN)TN)rrrrvalid_processor_kwargsrboolintr2r r r rTrrrrFdictrMrOrX __classcell__)r7s@rr&r&5sg1 %)*/ # RRTz R $( R  RRRRRR.[_$(++d9o=EV@WWT!-.   ^<B*BBBBLP""""""""rr&N)rfeature_extraction_utilsr image_utilsrprocessing_utilsrrrrtokenization_utils_baser r utilsr r get_loggerrloggerr=rrr&__all__rrrrfsX544444%%%%%%TTTTTTTTTTTTCCCCCCCC,,,,,,,,  H % %Nj+5fffff^fffR  r