tj:ddlmZddlZddlmZddlmZddlmZddlm Z m Z m Z ddl m Z d d lmZd d lmZe Gd deZe deGdde Ze dGddeZddgZdS)) dataclassN)nn)PreTrainedModel)Unpack) ModelOutputTransformersKwargsauto_docstring)can_return_tuple) AutoModel)ColModernVBertConfigcleZdZUeed<dZdZgZdZdZ dZ e j fdZ xZS)ColModernVBertPreTrainedModelconfigmodel)imagetextTcJt|dSN)super _init_weights)selfmodule __class__s x/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/models/colmodernvbert/modeling_colmodernvbert.pyrz+ColModernVBertPreTrainedModel._init_weights,s! f%%%%%)__name__ __module__ __qualname__r__annotations__base_model_prefixinput_modalities_no_split_modules_supports_sdpa_supports_flash_attn_supports_flex_attntorchno_gradr __classcell__rs@rrr"s{    (NU]__&&&&_&&&&&rrz: Base class for ColModernVBert embeddings output. ) custom_introceZdZUdZdZejdzed<dZej dzed<dZ e ejdzed<dZ e ejdzed<dZ e ejdzed<dS) ColModernVBertForRetrievalOutputa loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided): Language modeling loss (for next-token prediction). embeddings (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`): The embeddings of the model. image_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True` and `pixel_values` are provided): Tuple of `torch.FloatTensor` (one for the output of the image modality projection + one for the output of each layer) of shape `(batch_size, num_channels, image_size, image_size)`. Hidden-states of the image encoder at the output of each layer plus the initial modality projection outputs. Nloss embeddings hidden_statesimage_hidden_states attentions)rr r!__doc__r0r) FloatTensorr"r1Tensorr2tupler3r4rrr/r/1s  &*D% d ")))&*J t#***59M5*+d2999;?u01D8???26Je'(4/66666rr/u Following the ColPali approach, ColModernVBert leverages VLMs to construct efficient multi-vector embeddings directly from document images (“screenshots”) for document retrieval. The model is trained to maximize the similarity between these document embeddings and the corresponding query embeddings, using the late interaction method introduced in ColBERT. Using ColModernVBert removes the need for potentially complex and brittle layout recognition and OCR pipelines with a single model that can take into account both the textual and visual content (layout, charts, ...) of a document. ColModernVBert is trained on top of ModernVBert, and was introduced in the following paper: [*ModernVBERT: Towards Smaller Visual Document Retrievers*](https://arxiv.org/abs/2510.01149). ColModernVBert is part of the ColVision model family, which was introduced with ColPali in the following paper: [*ColPali: Efficient Document Retrieval with Vision Language Models*](https://huggingface.co/papers/2407.01449). c eZdZdZdeffd Zee d dej dzdej dzdej dzde e d ef d ZxZS) ColModernVBertForRetrievalvlmrcpt|||_|jjj|_t j|j|_|jj |_ tj |jjjj |j |_ |dSr)r__init__r vlm_config text_config vocab_sizer from_configr< embedding_dimrLinear hidden_sizeembedding_proj_layer post_init)rrrs rr>z#ColModernVBertForRetrieval.__init__^s      +7B():;;![6$&I K " . :  % % ! rN input_ids pixel_valuesattention_maskkwargsreturnc |jd|||d|}|d}|jjj}|||}||ddz }|9||j|j}||dz}t||j |j |j S) N)rHrJrIrT)dimkeepdim)dtypedevice)r1r2r4r3r9) r<rFweightrQtonormrR unsqueezer/r2r4r3) rrHrIrJrK vlm_outputlast_hidden_states proj_dtyper1s rforwardz"ColModernVBertForRetrieval.forwardlsTX )%     (].5; ../A/D/DZ/P/PQQ  *//b$/"G"GG  %+..Z5EjN_.``N#n&>&>r&B&BBJ/!$2!, * >     r)NNN)rr r!r#rr>r r r) LongTensorr6r7rr r/rZr+r,s@rr;r;Js$ 3      .215.2   #d* '$.  t+  +,  *    ^     rr;) dataclassesrr)rmodeling_utilsrprocessing_utilsrutilsrr r utils.genericr auto.modeling_autor configuration_colmodernvbertrrr/r;__all__r9rrrds*"!!!!! ------&&&&&&DDDDDDDDDD------******>>>>>> & & & & &O & & &  77777{77   7&"1 1 1 1 1 !>1 1 #"1 h ()H Ir