\\#"S=S#9"S>S?\!555r?\#"S@S#9"SASB\;\55r@/SCQrAg)EN) dataclass)CallableOptionalUnion)ACT2FN)Cache)GenerationMixin)use_kernel_forward_from_hub)FlashAttentionKwargs)GradientCheckpointingLayer)BaseModelOutputBaseModelOutputWithPastBaseModelOutputWithPooling)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack) ModelOutputTransformersKwargsauto_docstringcan_return_tuple torch_int) AutoModel)InternVLConfigInternVLVisionConfigRMSNormc8^\rSrSrSU4SjjrSrSrSrU=r$)InternVLVisionRMSNorm,c>[TU]5 [R"[R "U55UlX lg)z4 InternVLVisionRMSNorm is equivalent to T5LayerNorm N)super__init__nn Parametertorchonesweightvariance_epsilon)self hidden_sizeeps __class__s f/var/www/html/shao/venv/lib/python3.13/site-packages/transformers/models/internvl/modeling_internvl.pyr$InternVLVisionRMSNorm.__init__.s/ ll5::k#:; #cURnUR[R5nUR S5R SSS9nU[R "X0R-5-nURURU5-$)NrT)keepdim) dtypetor'float32powmeanrsqrtr*r))r+ hidden_states input_dtypevariances r/forwardInternVLVisionRMSNorm.forward6sw#)) %((7  $$Q',,R,>% H?T?T4T(UU {{]--k:::r1c^[URR5SUR3$)Nz, eps=)tupler)shaper*r+s r/ extra_repr InternVLVisionRMSNorm.extra_repr=s* ))*+6$2G2G1HIIr1)r*r))gư>) __name__ __module__ __qualname____firstlineno__r$r>rD__static_attributes__ __classcell__r.s@r/r r ,s$;JJr1r modulequerykeyvalueattention_maskscalingdropoutc UnUn [R"XRSS55U-n Ub"USS2SS2SS2SURS24n X-n [R R U SS9n [R RXURS9n [R"X5n U RSS5R5n X4$)Nrrr3dim)ptrainingr) r'matmul transposerBr% functionalsoftmaxrSrY contiguous) rMrNrOrPrQrRrSkwargs key_states value_states attn_weights causal_mask attn_outputs r/eager_attention_forwardreAsJL<<';';Aq'ABWLL!$Q1.D 0@0@0D.D%DE #1 ==((2(>L==((6??([L,,|:K''1-88:K  $$r1c ^\rSrSrSrS\4U4SjjrS S\RS\ \RS\ \RS\ \ 4S jjr S r U=r$) InternVLVisionAttention\z+Attention Class for InternVL Vision Encoderconfigc$>[TU]5 XlURUlUR UlURUR -UlURUR -UR:wa&[SURSUR S35eURS-Ul URUl URnURnSUl [R"URUR UR-UR S9Ul[R"URUR UR-UR S9Ul[R"URUR UR-UR S9Ul[R"URUR5UlUS:a[R*"U5O[R,"5Ul U(a[/UR5O[R,"5UlU(a[/UR5Ulg[R,"5Ulg)Nz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).gFbiasr)r#r$rir, embed_dimnum_attention_heads num_headshead_dim ValueErrorscaleattention_dropoutprojection_dropout use_qk_norm is_causalr%Linearattention_biasq_projk_projv_projprojection_layerDropoutIdentityr q_normk_norm)r+ri proj_dropoutqk_normr.s r/r$ InternVLVisionAttention.__init___s  ++33$..8 ==4>> )T^^ ;MdnnM]^NN#2' ]]D( !'!9!900 $$ii0NU[UjUjk ii0NU[UjUjk ii0NU[UjUjk " $..$.. I>JQ>N"**\":TVT_T_Ta?F+DNN;BKKM ?F+DNN; BKKM r1r;rQoutput_attentionsr_c UR5upVnURU5nURU5n URU5n UR U5nUR U 5n UR XVURUR5RSS5nU R XVURUR5RSS5n U RXVURUR5RSS5n [n URRS:wa[URRn U "UUU U U4UR(dSO UR UR"SS.UD6upU R XVUR$5n UR'U 5nUR)U5nU(aX4nU$US4nU$)NrreagerF)rSrRrv)sizeryrzr{rrreshaperorpr[viewreri_attn_implementationrrYrsrrrmr|rt)r+r;rQrr_ batch_sizeseq_len_ query_statesr`raattention_interfacerdrboutputoutputss r/r>InternVLVisionAttention.forward{s"/!3!3!5 Q{{=1 [[/ {{=1 {{<0 [[, #++JQUQ^Q^_iijkmno '' T^^T]][eefgijk #((dnndmm\ffghjkl (? ;; + +w 6"9$++:Z:Z"[ $7      % $}}C$2H2HJJ %  % ! "))*t~~N &&{3((0,=6(EKD>r1)rsrirmrprvrrzrortr|rryrrr{NN)rFrGrHrI__doc__rr$r'Tensorrrr r>rJrKrLs@r/rgrg\si5Z3Z>2648 )||)!.)$ELL1 ) -. ))r1rgcV^\rSrSr%\\S'SrSrSrS/r Sr Sr Sr Sr U4SjrSrU=r$) InternVLVisionPreTrainedModelriinternvl_vision pixel_valuesTInternVLVisionLayercn>[TU]U5 [U[5(aURR R 5 URb$URR R 5 URb%URR R 5 gg[U[5(asURR RURR5 URR RURR5 gg)zInitialize the weightsN)r# _init_weights isinstanceInternVLVisionEmbeddings cls_tokendatazero_ mask_tokenposition_embeddingsrlambda_1fill_rilayer_scale_init_valuelambda_2)r+rMr.s r/r+InternVLVisionPreTrainedModel._init_weightss f% f6 7 7    ! ! ' ' )  ,!!&&,,.))5**//5576  3 4 4 OO & &t{{'I'I J OO & &t{{'I'I J5r1)rFrGrHrIr__annotations__base_model_prefixmain_input_namesupports_gradient_checkpointing_no_split_modules_supports_sdpa_supports_flash_attn_supports_flex_attn_supports_attention_backendrrJrKrLs@r/rrsF  )$O&*#./N"& K Kr1rz7 Class for outputs of [`InternVLVisionModel`]. ) custom_introc\rSrSrSrSrg)$InternVLVisionModelOutputWithPoolinga2 pooler_output (`torch.FloatTensor` of shape `(batch_size, hidden_size)`): Average of the last layer hidden states of the patch tokens (excluding the *[CLS]* token) if *config.use_mean_pooling* is set to True. If set to False, then the final hidden state of the *[CLS]* token will be returned. rN)rFrGrHrIrrJrr1r/rrsr1rcf^\rSrSrSrU4SjrS\RS\R4SjrSr U=r $)InternVLVisionPatchEmbeddingsz This class turns `pixel_values` of shape `(batch_size, num_channels, height, width)` into the initial `hidden_states` (patch embeddings) of shape `(batch_size, seq_length, hidden_size)` to be consumed by a Transformer. cH>[TU]5 URURp2URUR pTUSUS-USUS--nUSUS-USUS-4nX lX0lX@lX`lXpl[R"XEX3S9Ul g)Nrr) kernel_sizestride) r#r$ image_size patch_size num_channelsr, num_patches patch_shaper%Conv2d projection) r+rirrrr,rrr.s r/r$&InternVLVisionPatchEmbeddings.__init__s !'!2!2F4E4EJ$*$7$79K9Kk!!} 1 5*Q-:VW=:XY !!} 1 5z!} ST 7UV $$(&&))L:ir1rreturncURup#pEX0R:wa [S5eURU5nURSURSpUR S5R SS5nXgU44$)NzeMake sure that the channel dimension of the pixel values match with the one set in the configuration.rrr)rBrrqrflattenr[) r+rrrheightwidth embeddings patch_height patch_widths r/r>%InternVLVisionPatchEmbeddings.forwards2>2D2D/ & ,, ,w __\2 $.$4$4Q$79I9I!9Lk''*44Q: +666r1)rrrrrr) rFrGrHrIrr$r'rr>rJrKrLs@r/rrs. j 7ELL 7U\\ 7 7r1rc^\rSrSrSrS\SS4U4SjjrS\RS\ S \ S\R4S jr SS \RS \ \RS\R4S jjr SrU=r$)rz[ Construct the CLS token, position and patch embeddings. Optionally, also the mask token. rirNc^>[TU]5 [R"[R "SSUR 55UlUR(a<[R"[R "SSUR 55Ul OSUl [U5Ul URUl [UR[R R"5(a UROURUR4UlURR$nUR&(a?[R"[R "SUS-UR 55UlOSUl[R*"UR,5Ulg)Nr)r#r$r%r&r'zerosr,ruse_mask_tokenrrpatch_embeddingsrrr collectionsabcIterabler use_absolute_position_embeddingsrr}hidden_dropout_probrS)r+rirr.s r/r$!InternVLVisionEmbeddings.__init__s' ekk!Q8J8J&KL   ll5;;q!V=O=O+PQDO"DO =f E ++&++[__-E-EFF   ##V%6%67  ++77  2 2')||EKK;QR?TZTfTf4g'hD $'+D $zz&"<"<= r1rrrcURSS- nURRSS- n[RR 5(dXE:XaX#:Xa UR$URSS2SS24nURSS2SS24nURSnX R S-n X0R S-n [ US-5n URSXU5nURSSSS5n[RRUX4SS S 9nURSSSS5RSSU5n[R"Xg4SS 9$) a This method allows to interpolate the pre-trained position encodings, to be able to use the model on higher resolution images. This method is also adapted to support torch.jit tracing. Adapted from: - https://github.com/facebookresearch/dino/blob/de9ee3df6cf39fac952ab558447af1fa1365362a/vision_transformer.py#L174-L194, and - https://github.com/facebookresearch/dinov2/blob/e1277af2ba9496fbadf7aec6eba56e8d882d1e35/dinov2/models/vision_transformer.py#L179-L211 rNr3r?rrbicubicF)rmode align_cornersrV)rBrr'jit is_tracingrrrpermuter%r\ interpolatercat) r+rrrr num_positionsclass_pos_embedpatch_pos_embedrW new_height new_widthsqrt_num_positionss r/interpolate_pos_encoding1InternVLVisionEmbeddings.interpolate_pos_encodings]!&&q)A- 0066q9A= yy##%%+*F6?++ +221bqb59221ab59r"q11 __Q// &}c'9:)11!5G]`a)11!Q1=--33 ( 4 *11!Q1=BB1b#Nyy/;CCr1rbool_masked_poscURu p4nURU5unupxUR5upnUbIURR XS5n UR S5R U 5n USU - -X--nURR U SS5n [R"X4SS9nURbX`RXdU5-nURU5nXgU44$)Nr3rrV) rBrrrexpand unsqueezetype_asrr'rrrrS)r+rrrrrrrrrr mask_tokensw cls_tokenss r/r> InternVLVisionEmbeddings.forward:s +001e262G2G 2U/ /\!+!2 Q  &//00bIK))"-55kBA#q1u- ?J^^**:r2> YY 7Q?  # # /#&C&CJX]&^^J\\*- +666r1)rrSrrrrrN)rFrGrHrIrrr$r'rintrr BoolTensorr>rJrKrLs@r/rrs >3>>,&D5<<&D&DUX&D]b]i]i&DV7;7ll7"%"2"237  77r1rcb^\rSrSrU4SjrS\R S\R 4SjrSrU=r $)InternVLVisionMLPiTc>[TU]5 Xl[URUl[ R"URUR5Ul [ R"URUR5Ul gr) r#r$rir hidden_act activation_fnr%rwr,intermediate_sizefc1fc2r+rir.s r/r$InternVLVisionMLP.__init__Usb  #F$5$5699V//1I1IJ99V55v7I7IJr1r;rclURU5nURU5nURU5nU$r)rrr)r+r;s r/r>InternVLVisionMLP.forward\s4/ **=9 / r1)rrirr) rFrGrHrIr$r'rr>rJrKrLs@r/rrTs)KU\\ellr1r) layer_normrms_normc ^\rSrSrSrS\SS4U4SjjrS S\RS\ S\ \ \R\ \R\R444S jjr S r U=r$) rifz?This corresponds to the Block class in the timm implementation.rirNc>[TU]5 URUlSUl[ U5Ul[ U5Ul[UR"URURS9Ul [UR"URURS9Ul URn[R "U["R$"UR5-SS9Ul[R "U["R$"UR5-SS9Ul[R*"UR,5Ulg)Nrr-T) requires_grad)r#r$chunk_size_feed_forward seq_len_dimrg attentionrmlpNORM2FN norm_typer,layer_norm_epslayernorm_beforelayernorm_afterrr%r&r'r(rrr}rrS)r+ri init_valuesr.s r/r$InternVLVisionLayer.__init__is '-'E'E$08$V, '(8(8 9&:L:LRXRgRg h&v'7'789K9KQWQfQfg33  [5::f>P>P3Q%Qaef  [5::f>P>P3Q%Qaef zz&"<"<= r1r;rcURURU5US9up4URU-nX1-nURU5nUR U5nUR U5nUR bUR U-nXQ-nXT4$)N)r)rrrrrrSr)r+r;rattention_outputattention_weights layer_outputs r/r>InternVLVisionLayer.forwardxs /3nn  ! !- 0//=/ +  ==+;;)8 ++M: xx - ||L1 == $==<7L$3 ..r1) rr rSrrrrrr )F)rFrGrHrIrrr$r'rboolrrAr>rJrKrLs@r/rrfsnI >3 > >$#(/||/ / uU\\"E%,, *D$EE F //r1rc ~^\rSrSrS\SS4U4Sjjr\S S\RS\ S\ S\ \ \ 44S jj5r S rU=r$) InternVLVisionEncoderirirNc>[TU]5 Xl[R"[ UR 5Vs/sHn[U5PM sn5UlSUl gs snf)NF) r#r$rir% ModuleListrangenum_hidden_layersrlayergradient_checkpointing)r+riir.s r/r$InternVLVisionEncoder.__init__sS  ]]vOgOgIh#iIhA$7$?Ih#ij &+#$jsA%r;routput_hidden_statescU(aSOSnU(aSOSn[UR5H/upgU(aXA4-nU"X5nUSnU(dM'XXS4-nM1 U(aXA4-n[UUUS9$)Nrrrlast_hidden_stater; attentions) enumerater#r) r+r;rr'all_hidden_statesall_self_attentionsr% layer_module layer_outputss r/r>InternVLVisionEncoder.forwards#7BD$5b4(4OA#$58H$H!(JM)!,M  &91=MrJrKrLs@r/rrsg,3,, #(%*  ||   #  uo% &   r1rc^\rSrSrS\SS4U4SjjrSr\\S S\ RS\ \ RS \ \ S \ \ S\\\44 S jj55rS rU=r$)InternVLVisionModelirirNc8>[TU]U5 Xl[U5Ul[ U5UlUR(a[R"5O([R"URURS9Ul UR5 g)Nr )r#r$rirrrencoderuse_mean_poolingr%r~ LayerNormr,r layernorm post_initrs r/r$InternVLVisionModel.__init__sm   26:,V4 $44BKKM",,vGYGY_e_t_t:u  r1c.URR$r)rrrCs r/get_input_embeddings(InternVLVisionModel.get_input_embeddingss///r1rrrr'cUbUOURRnUbUOURRnURXS9upVUR UUUS9nUSnUR U5n[ UURURS9$)z bool_masked_pos (`torch.BoolTensor` of shape `(batch_size, num_patches)`, *optional*): Boolean masked positions. Indicates which patches are masked (1) and which aren't (0). )r)rr'rr)) rirr'rr5r8rr;r+) r+rrrr'embedding_outputrencoder_outputssequence_outputs r/r>InternVLVisionModel.forwards2C1N-TXT_T_TqTq$8$D $++JjJj #oolo\,, /!5'  *!,..93-)77&11  r1)rirr5r8)NNN)rFrGrHrIrr$r<rrr'rrrrrrArr>rJrKrLs@r/r3r3s 3  07;,0/3  ll "%"2"23 $D>  'tn  u:: ;   r1r3c@\rSrSr%\\S'SrSrSrSr Sr Sr Sr Sr Srg)InternVLPreTrainedModeliriTpast_key_valuesrN)rFrGrHrIrrrr_skip_keys_device_placementrr_can_compile_fullgraphrrrJrr1r/rDrDs7 &*#"3N!"&r1rDc6^\rSrSrS\4U4SjjrSrSrU=r$)InternVLMultiModalProjectoriric0>[TU]5 [R"URR [ SUR- 5S--5Ul[R"URR [ SUR- 5S--URR 5Ul [URUl[R"URR URR 5Ulg)Nrr)r#r$r%r7 vision_configr,rdownsample_ratiorrw text_configlinear_1rprojector_hidden_actactlinear_2rs r/r$$InternVLMultiModalProjector.__init__s ,,v';';'G'G#aRXRiRiNiJjnoJo'op  , ,s1v7N7N3N/OST/T TV\VhVhVtVt  &556 &"4"4"@"@&BTBTB`B`a r1cURU5nURU5nURU5nURU5nU$r)rrOrQrR)r+image_featuresr;s r/r>#InternVLMultiModalProjector.forward s@7  m4 /  m4 r1)rQrrOrR) rFrGrHrIrr$r>rJrKrLs@r/rJrJsb~br1rJzM Base class for InternVL outputs, with hidden states and attentions. cB\rSrSr%SrSr\\R\ S'Sr g)InternVLModelOutputWithPastia past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. image_hidden_states (`torch.FloatTensor`, *optional*): A `torch.FloatTensor` of size `(batch_size, num_images, sequence_length, hidden_size)`. image_hidden_states of the model produced by the vision encoder and after projecting the last hidden state. Nimage_hidden_statesr) rFrGrHrIrrYrr' FloatTensorrrJrr1r/rXrXs 8<%"3"34;r1rXzx The InternVL model which consists of a vision backbone and a language model, without a language modeling head. c#l^\rSrSrSS0rS\4U4SjjrSrSrSr S r S!S \ RS \ \\\\4S \ \4S jjrS\ R&S\ RS\ R4Sjr\\S"S\ R&S \ RS\ \ R.S\ \ R&S\ \S\ \ RS \ \\\\4S \ \S\ \S\ \S\ \S\ \S\ \ R&S\\S\\\44Sjj55rS#S\ R.S\4Sjjr S r!U=r"$)$ InternVLModeli)zlanguage_model.modellanguage_modelric>[TU]U5 [R"UR5Ul[ U5Ul[R"UR5Ul UR5 gr) r#r$r from_configrL vision_towerrJmulti_modal_projectorrNr]r9rs r/r$InternVLModel.__init__1sY  %11&2F2FG%@%H"'33F4F4FG r1c6URR5$r)r]r<rCs r/r<"InternVLModel.get_input_embeddings9s""7799r1c:URRU5 gr)r]set_input_embeddingsr+rPs r/rf"InternVLModel.set_input_embeddings<s 007r1cXlgrr]r+decoders r/ set_decoderInternVLModel.set_decoder?s%r1cUR$rrjrCs r/ get_decoderInternVLModel.get_decoderBs"""r1rvision_feature_layervision_feature_select_strategyc <UbUOURRnUbUOURRnURRnUS:XaUR US9R nOUR US9RUnUS:XaUSS2SS2SS24nURSn[US-5nURSn URXUS5nURXeS9nURU SURS5nURU5nU$) a Obtains image last hidden states from the vision tower and apply multimodal projection. Args: pixel_values (`torch.FloatTensor]` of shape `(batch_size, channels, height, width)`) The tensors corresponding to the input images. vision_feature_layer (`int` or `list[int]`): Layer index or list of layer indices to extract features from. Returns: vision_features (`torch.Tensor`): Image feature tensor of shape `(num_images, image_length, embed_dim)`. Nr3)rdefaultrrr) scale_factor) rirrrsrMr`r* vision_modelr;rBrr pixel_shufflera) r+rrrrsr_rMvision_featureschannels feature_sizers r/get_image_features InternVLModel.get_image_featuresEs?&%9$D $++JjJj  .9 +;; '  ;;77 2 %"//\/J\\O"//\/JXXYmnO )Y 6-aQh7O#((+8S=) $**1- *11*LZ\],,_,\*11*b/BWBWXZB[\44_Er1 input_ids inputs_embedsrUc JUcjX R5"[R"URR[R UR S95:HnURS5nOXRR:HnUR5nURS5RU5RUR 5nURSURS-nX$R5UR5:wa[SUSU35eU$)z Obtains multimodal placeholdr mask from `input_ids` or `inputs_embeds`, and checks that the placeholder token count is equal to the length of multimodal features. If the lengths are different, an error is raised. )r5devicer3rrz6Image features and image tokens do not match: tokens: z , features )r<r'tensorriimage_token_idlongrallsumr expand_asr6rBnumelrq)r+r~rrUspecial_image_maskn_image_tokensn_image_featuress r/get_placeholder_mask"InternVLModel.get_placeholder_maskzs  !.2K2K2M T[[77uzzR_RfRfg3" "4!7!7!; !*kk.H.H!H +//1/99"=GG VYYZgZnZno)//2^5I5I!5LL  , 2 2 48L8L8N NHHXXcdtcuv "!r1rQ position_idsrF use_cacherr' return_dictcache_positionr_rc U bU OURRn U bU OURRn U bU OURRn UbUOURRnUbUOURR nUSLUSL- (a [ S5eUcUR5"U5nUbYURUUUS9nURURUR5nURXUS9nURUU5nUR"SUUUUU U U SU S. UD6n[UR UR"UR$UR&UbWS9$SS9$)Nz:You must specify exactly one of input_ids or inputs_embedsrrrrs)rrUT) rQrrFrrrr'rr)r*rFr;r+rYr)rirr'use_return_dictrrrsrqr<r|r6rr5rmasked_scatterr]rXr*rFr;r+)r+r~rrQrrFrrrrsrrr'rrr_rUrrs r/r>InternVLModel.forwards&2C1N-TXT_T_TqTq$8$D $++JjJj &1%2J   QU   r1ryrvc UR5up4pVXR-S:wdXB-S:wa [S5eURX4[XR-5[Xb- 55nUR SSSS5R 5nURU[XR-5[XB-5[XbS-- 55nUR SSSS5R 5nU$)aPerform pixel shuffle downsampling on vision features. Args: vision_features (`torch.Tensor`): Input tensor of shape (batch_size, width, height, channels). scale_factor (`float`, *optional*, defaults to `0.5`): Factor by which to downsample. Default is 0.5, which halves the dimensions. Returns: vision_features (`torch.Tensor`): Downsampled tensor of shape (batch_size, height*scale_factor, width*scale_factor, channels/(scale_factor^2)). rzKHeight and width must be divisible by scale_factor for proper downsampling.rrr)rrqrrrr^)r+ryrvrrrrzs r/rxInternVLModel.pixel_shuffles/>.B.B.D+ 6  A %)=)Bjk k*.. s6#893x?V;W *11!Q1=HHJ*.. F12C8L4MsS[mn_nSoOp  *11!Q1=HHJr1)r]rar`r) NNNNNNNNNNNNN)r)#rFrGrHrI_checkpoint_conversion_mappingrr$r<rfrmrpr'rZrrrliststrr| LongTensorrrrrr rrr rArXr>floatrxrJrKrLs@r/r\r\)s  '=>N%O"~:8&# AE8< 3''3'uS$s)^'<=3)1 3j"))":?:K:K"]b]n]n"0'+*.1537+/59@D8<$(,0/3&*59D ##D ''D !. D u//0 D "% D   1 12D 'uS$s)^'<=D )1 D D>D $D>D 'tnD d^D !!1!12D -.D u11 2!D D L!U\\!!!r1r\zT Base class for InternVL causal language model (or autoregressive) outputs. c\rSrSr%SrSr\\R\ S'Sr \\R\ S'Sr \\ \R\ S'Sr \\\R\ S'Sr\\\R\ S'Sr\\R\ S 'S rg) InternVLCausalLMOutputWithPastia) loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided): Language modeling loss (for next-token prediction). logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.vocab_size)`): Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax). past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. image_hidden_states (`torch.FloatTensor`, *optional*): A `torch.FloatTensor` of size `(batch_size, num_images, sequence_length, hidden_size)`. image_hidden_states of the model produced by the vision encoder and after projecting the last hidden state. NlosslogitsrFr;r+rYr)rFrGrHrIrrrr'rZrrrFrr;rAr+rYrJrr1r/rrs )-D(5$$ %,*.FHU&& '.9=OXd5#4#456=8[TU]U5 [U5Ul[R "UR RUR RSS9Ul UR5 g)NFrk) r#r$r\modelr%rwrNr, vocab_sizerr9rs r/r$)InternVLForConditionalGeneration.__init__+sS  "6* yy!3!3!?!?ASASA^A^ejk  r1c6URR5$r)rr<rCs r/r<5InternVLForConditionalGeneration.get_input_embeddings1szz..00r1c:URRU5 gr)rrfrgs r/rf5InternVLForConditionalGeneration.set_input_embeddings4s ''.r1rcUR$r)rrCs r/get_output_embeddings6InternVLForConditionalGeneration.get_output_embeddings7s ||r1c:URRU5 gr)rrmrks r/rm,InternVLForConditionalGeneration.set_decoder:s w'r1c6URR5$r)rrprCs r/rp,InternVLForConditionalGeneration.get_decoder=szz%%''r1rrrrsc BURR"SUUUS.UD6$)Nrr)rr|)r+rrrrsr_s r/r|3InternVLForConditionalGeneration.get_image_features@s3zz,, %!5+I    r1c.URR$r)rr]rCs r/r]/InternVLForConditionalGeneration.language_modelOszz(((r1c.URR$r)rr`rCs r/r`-InternVLForConditionalGeneration.vision_towerSszz&&&r1c.URR$r)rrarCs r/ra6InternVLForConditionalGeneration.multi_modal_projectorWszz///r1r~rQrrFrlabelsrrr'rrlogits_to_keep image_sizesr_c U bU OURRn U bU OURRn U bU OURRn UbUOURRnUbUOURR nUR "SUUUUUUUUU U U SUUS.UD6nUSn[U[5(a [U*S5OUnURUSS2USS245nSnU b3UR"SUXRRRS.UD6n[UUURUR UR"UR$S9$)ay Example: ```python >>> import torch >>> from transformers import AutoProcessor, AutoModelForImageTextToText >>> torch_device = "cuda" >>> processor = AutoProcessor.from_pretrained("OpenGVLab/InternVL3-1B-hf") >>> model = AutoModelForImageTextToText.from_pretrained( ... "OpenGVLab/InternVL3-1B-hf", torch_dtype=torch.bfloat16, device_map=torch_device ... ) >>> messages = [ ... { ... "role": "user", ... "content": [ ... { ... "type": "image", ... "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg", ... }, ... { ... "type": "image", ... "url": "https://thumbs.dreamstime.com/b/golden-gate-bridge-san-francisco-purple-flowers-california-echium-candicans-36805947.jpg", ... }, ... {"type": "text", "text": "These images depict two different landmarks. Can you identify them?"}, ... ], ... }, ... ] >>> inputs = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt").to(torch_device) >>> generate_ids = model.generate(**inputs, max_new_tokens=200) >>> print(processor.decode(generate_ids[0, inputs["input_ids"].shape[1] :], skip_special_tokens=True)) The images depict the Statue of Liberty and the Golden Gate Bridge. ```NT)r~rrQrrFrrrrsrrr'rrrr)rrr)rrrFr;r+rYr)rirr'rrrrsrrrslicer loss_functionrNrrrFr;r+rY)r+r~rrQrrFrrrrsrrrr'rrrrr_rr; slice_indicesrrs r/r>(InternVLForConditionalGeneration.forward[sr2C1N-TXT_T_TqTq$8$D $++JjJj &1%SV8W8W~ot4]k mA}a,?@A  %%f9P9P9[9[_eD.#33!//)) ' ; ;   r1c P>[T U]"U4UUUUUS.UD6n USS:XaXIS'U $)N)rFrrQrrrr)r#prepare_inputs_for_generation) r+r~rFrrrQrrr_ model_inputsr.s r/r>InternVLForConditionalGeneration.prepare_inputs_for_generationsTw<  +')))    !  !,8 (r1)rrr)NNNNNNNNNNNNNNrN)NNNNNN))rFrGrHrIr_tied_weights_keysrr$r<rfr%Modulerrmrpr'rZrrrrrr|propertyr]r`rarrrrr rrrrArr>rrJrKrLs@r/rrs"8-"?#, &" ++~ 1/ryy(( AE8<  ''  'uS$s)^'<=  )1  ))''00'+*.1537+/59@D8<-1$(,0/3&*5934.2#i ##i ''i !. i u//0 i "% i   1 12i 'uS$s)^'<=i )1 i ))*i D>i $D>i 'tni d^i !!1!12i c5<</0!i "ell+#i $+,%i & u44 5'i i \r1r)rr3rDr\r)r)Bcollections.abcr dataclassesrtypingrrrr'torch.nnr% activationsr cache_utilsr generationr integrationsr modeling_flash_attention_utilsr modeling_layersr modeling_outputsrrrmodeling_utilsrrprocessing_utilsrutilsrrrrrautorconfiguration_internvlrrrr rrrergrrrrrr7rrrr3rDrJrXr\rr__all__rr1r/rs.!,, ! )7B9ddF&aaHY'JBIIJ(J6% II% <<% % << % U\\* %  %%6HbiiHVKOKK2  +E  !7BII!7L[7ryy[7|  3H I-/4-/`# BII# L2 72 2 j 'o ' '"))$  <"9 <  <  M+M  M`  <[<  <2 B'>B  BJ r1