tjȖdZddlmZddlZddlmZmZddlmZddl m Z m Z ddl m Z dd lmZmZmZmZmZdd lmZdd lmZdd lmZdd lmZmZmZddlmZm Z ddl!m"Z"ddl#m$Z$ddl%m&Z&ddl'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-ddl.m/Z/edeGddeZ0Gdde,Z1Gdde)Z2Gddej3Z4Gd d!e(Z5Gd"d#e+Z6Gd$d%e&Z7Gd&d'e*Z8eGd(d)e-Z9eGd*d+e9Z:Gd,d-ej3Z;ed.Gd/d0e9Z<ed1Gd2d3e9Z=Gd4d5e$Z>Gd6d7ej3Z?Gd8d9ej3Z@Gd:d;ej3ZAGd<d=ej3ZBGd>d?ej3ZCGd@dAej3ZDeGdBdCe9ZEedDGdEdFe e9ZFgdGZGdS)HzPyTorch BEiT model.) dataclassN)Tensornn)initialization) BackboneMixinfilter_output_hidden_states)create_bidirectional_mask)BackboneOutputBaseModelOutputWithPoolingImageClassifierOutputMaskedLMOutputSemanticSegmenterOutput)PreTrainedModel)Unpack)#compile_compatible_method_lru_cache)TransformersKwargsauto_docstring torch_int)can_return_tuplemerge_with_config_defaults)capture_outputs)ResNetConvLayer) SwinDropPath) ViTAttention ViTEmbeddingsViTLayerViTMLPViTPatchEmbeddingsViTPreTrainedModel) BeitConfigz- Class for outputs of [`BeitModel`]. ) custom_introceZdZdZdS)BeitModelOutputWithPoolingaF pooler_output (`torch.FloatTensor` of shape `(batch_size, hidden_size)`): Average of the last layer hidden states of the patch tokens (excluding the *[CLS]* token) if *config.use_mean_pooling* is set to True. If set to False, then the final hidden state of the *[CLS]* token will be returned. N)__name__ __module__ __qualname____doc__c/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/models/beit/modular_beit.pyr&r&+sr,r&ceZdZdS)BeitPatchEmbeddingsNr'r(r)r+r,r-r/r/:Dr,r/cZeZdZdeddfdZ ddejdejdzdejfdZdS) BeitEmbeddingsconfigreturnNc.tj|tjt jdd|j|_|jr-tjt jdd|jnd|_ t||_ |j |_ |j j }|jr0tjt jd|dz|jnd|_tj|j|_dS)Nr")rModule__init__ Parametertorchzeros hidden_size cls_tokenuse_mask_token mask_tokenr/patch_embeddings patch_size num_patches use_absolute_position_embeddingsposition_embeddingsDropouthidden_dropout_probdropout)selfr4rBs r-r8zBeitEmbeddings.__init__?s 4   ek!Q8J&K&KLLQWQfp",u{1a9K'L'LMMMlp 3F ; ; ++7 6 BLQ a9KLL M M M  z&"<== r, pixel_valuesbool_masked_posc|j\}}}}||}|\}}}|R|j||d} |d| } |d| z z| | zz}|j|dd} tj | |fd}|j || |||z}| |}|SNr"dim) shaper@sizer?expand unsqueezetype_asr=r:catrDinterpolate_pos_encodingrG) rHrIrJ_heightwidth embeddings batch_sizeseq_len mask_tokensmask cls_tokenss r-forwardzBeitEmbeddings.forwardNs +01fe**<88 !+!2!2 GQ  &/00WbIIK",,R0088EED#q4x0;3EEJ^**:r2>> Y J7Q???  # /#d&C&CJPVX]&^&^^J\\*-- r,N) r'r(r)r#r8r:r BoolTensorr`r+r,r-r3r3>s| >z >d > > > >$48l)D0  r,r3ceZdZdeddffd Zeeddeeefde j fdZ d d e de j fd Z xZS) BeitRelativePositionBiasr4r5Nct|j}t|tt fs||f}|d|jz|d|jzf|_d|jdzdz d|jdzdz zdz|_tj tj |j|j |_dSNrr"rr)superr8 image_size isinstancetuplelistrA window_sizenum_relative_distancerr9r:r;num_attention_headsrelative_position_bias_table)rHr4rh __class__s r-r8z!BeitRelativePositionBias.__init__is & *udm44 2$j1J&qMV->> 1 QWQb@bc&'$*:1*=&=&Aa$JZ[\J]F]`aFa%bef%f",.L K2F4N O O- - )))r, )maxsizerlc Td|dzdz d|dzdz zdz}|d|dz}tjtjtjtj|dtj|ddd}|dddddf|dddddfz ddd}|dddddfxx|ddz z cc<|dddddfxx|ddz z cc<|dddddfxxd|dzdz zcc<tj|dzfdz|j }| d |ddddf<|dz |dddf<|dz |dddf<|dz |d <|S) z This method creates the relative position index, modified to support arbitrary window sizes, as introduced in [MiDaS v3.1](https://huggingface.co/papers/2307.14460). rrr"rij)indexing) start_dimN)rQdtyperM)rr) r:flattenstackmeshgridarangepermute contiguousr;rwsum)rlrm window_areacoords_flattenrelative_coordsrelative_position_indexs r- generate_relative_position_indexz9BeitRelativePositionBias.generate_relative_position_indexus!"#[^!3a!7A A)B122&)>)BA&(=(A%&&r,FrVc0d|jdzdz }d|jdzdz }d|dzdz }d|dzdz }|j}|j} ||zdz} |d| dz } | d||ddddd} t j| t|t|fd} | dddd| dz d} tj | || dz dg} | |}| | d}| |d|dzdz|d|dzdzd}|ddd }|rKt j|d||fdd d}|dS) zu Modification of timm.models.beit.py: Attention._get_rel_pos_bias to support arbitrary window sizes. rrr"rNrMbilinear)rQmodeFrQr align_corners)rlrormreshaper|r functional interpolaterr:rUrviewr}rSsqueeze)rHrlrVdim_size old_height old_width new_height new_width old_relative_position_bias_tableold_num_relative_distancenew_num_relative_distance old_sub_table new_sub_table new_relative_position_bias_tablerrelative_position_biass r-r`z BeitRelativePositionBias.forwards`)!,,q0 (++a/ Q'!+  A&* +/+L($($>!$.$:Q$>!89X;TWX;X9XY %--aJKKSSTUWXZ[]^__  11 :!6!6 )8L8L MT^2  &--aAq99AAB[^_B_acdd +09 <=VYZ=Z=\=\] ^, , (#'"G"G "T"T!ABYB^B^_aBbBb!c"8!%>&0033)# &?&& gajj #&//222r,)FN)r'r(r)r#r8 staticmethodrrjintr:rrboolr` __classcell__rps@r-rdrdhs  z  d       ((444'eCHo'%,'''54\'4-3-3T-3]b]i-3-3-3-3-3-3-3-3r,rdc$eZdZdeffd ZxZS) BeitAttentionr4ct|tj|j|j|jz|_tj|j|j|jzd|_tj|j|j|jz|_ tj|j|jz|j|_ dS)NF)bias) rgr8rLinearr<rnhead_dimq_projk_projv_projo_projrHr4rps r-r8zBeitAttention.__init__s    i 2F4NQUQ^4^__ i 2F4NQUQ^4^ejkkk i 2F4NQUQ^4^__ i :T] JFL^__ r,)r'r(r)r#r8rrs@r-rrsO`z``````````r,rceZdZdS)BeitMLPNr0r+r,r-rrr1r,rceZdZdS) BeitDropPathNr0r+r,r-rrr1r,rc eZdZdZddedeffd Z ddejd ejdzd e d e e e fdzd e e d ejf dZxZS) BeitLayerz?This corresponds to the Block class in the timm implementation.r4drop_path_ratect|j|_|dkrt|nt j|_|j}|dkr0t j|tj |j zdnd|_ |dkr0t j|tj |j zdnd|_ |jrt|nd|_dS)NrrT) requires_gradg?)rgr8rArrIdentity drop_pathlayer_scale_init_valuer9r:onesr<lambda_1lambda_2use_relative_position_biasrdr)rHr4r init_valuesrps r-r8zBeitLayer.__init__s  +9G#9M9Mn555SUS^S`S`3 ^ilm^m^mBLuz&2D'E'EEUY Z Z Z Zsv _jlm^m^mBLuz&2D'E'EEUY Z Z Z Zsv KQJk&u&>v&F&F&Fqu###r,NF hidden_statesattention_maskrV resolutionkwargsr5c &|jE|\}}||jz||jzf}||||jd} || |zn| }|} ||}|j|fd|i|\}} ||}|j|z}||| z}|} ||}| |}||}|j |z}||| z}|S)Nr")rr) rrArPlayernorm_before attentionrGrrlayernorm_aftermlpr) rHrrrVrrrXrYrlrresidualrWs r-r`zBeitLayer.forwards\  & 2&MFE!T_4et6NOK%)%@%@5 @STU@V&A&& "   )    q  ]33  5 }55@ !,,];; //  ]33  5 }55@ r,)rNFN)r'r(r)r*r#floatr8r:rrrjrrrr`rrs@r-rrsII v vz v5 v v v v v v"/3).-1 &&|& t+&#' & #s(Od* & +, & &&&&&&&&r,rc(eZdZdgZdgZdZdZdZdS)BeitPreTrainedModelrz.*relative_position_index.*Fcttj||t|tr]t j|j|jt j|j|jt j|jdSdSt|trt j|j dSt|trit|j tjrLt j|j |jjt j|j|jjdSdSdS)zInitialize the weightsN)r _init_weightsrir3initzeros_r=r?rDrdrorrrr9 constant_r4rr)rHmodules r-rz!BeitPreTrainedModel._init_weightss(%dF333 fn - - T K( ) ) ) , F-...)5 F67777765  8 9 9 T K; < < < < <  * * T&/2<88 Tv 0RSSSv 0RSSSSS T T T Tr,N)r'r(r)_no_split_modules"_keys_to_ignore_on_load_unexpected_supports_flash_attn_supports_flex_attnrr+r,r-rr sE$ *H)I& TTTTTr,rceZdZddededdffd Zeede dd e j d e j dzd ed e j dzd e e def dZxZS) BeitModelTr4add_pooling_layerr5Nct|_t|_jrt nd|_fdtj D}tj fd|D|_ j rtjntjjj|_|rt'nd|_|dS)zv add_pooling_layer (bool, *optional*, defaults to `True`): Whether to add a pooling layer NcVg|]%}j|ztjdz dz &S)r")rmaxnum_hidden_layers).0ir4s r- z&BeitModel.__init__..2sE   QRF !A %F,Dq,H!(L(L L   r,c2g|]}t|S))r)r)rrr4s r-rz&BeitModel.__init__..5s&$b$b$bQYva%H%H%H$b$b$br,eps)rgr8r4r3rZ!use_shared_relative_position_biasrdshared_position_biasrangerr ModuleListlayersuse_mean_poolingr LayerNormr<layer_norm_eps layernorm BeitPoolerpooler post_init)rHr4rdrop_path_ratesrps ` r-r8zBeitModel.__init__&s     (00060X b $V , , ,^b !    V[\b\tVuVu   m$b$b$b$bRa$b$b$bcc $4 uBKMMM",vGY_e_t:u:u:u ->Gj(((4  r,F)tie_last_hidden_statesrIrJrVrrc |||}|jdd}t|j||}|jO|\}} ||jjz| |jjzf} || ||jd} || |zn| }|} |jD]} | | f|||d|} || }|j||nd}t||S) z bool_masked_pos (`torch.BoolTensor` of shape `(batch_size, num_patches)`, *optional*): Boolean masked positions. Indicates which patches are masked (1) and which aren't (0). )rJrN)r4 inputs_embedsrr")rVr)rrVr)last_hidden_state pooler_output) rZrPr r4rrArrrr&)rHrIrJrVrrembedding_outputrrXrYrlshared_relative_position_biasrlayersequence_output pooled_outputs r-r`zBeitModel.forward?sW ??<?YY!'+ 2;*)     $ 0&MFE!T[%;;UdkF\=\]K,0,E,E6NYiYopqYr-F-- ) "-.>>2  ) [  E!E-)A%   MM..778< 8O O444UY )O[hiiiir,)Tr)r'r(r)r#rr8rrrr:rrbrrr&r`rrs@r-rr$s zdd2 _E22248)..2 -j-jl-j)D0-j#' -j  t+ -j +, -j $-j-j-j^32 -j-j-j-j-jr,rcLeZdZdeddffd ZdejdejfdZxZS)rr4r5Nct|jr tj|j|jnd|_dS)Nr)rgr8rrrr<rrrs r-r8zBeitPooler.__init__ssJ KQKb lBL+1F G G G Ghl r,rc|j7||ddddddfdn |dddfS)Nr"r)rmeanrHrs r-r`zBeitPooler.forwardysXBF.B\t~~mAAAqrr111H5::1==>>>bopqpqpqstptbuur,) r'r(r)r#r8r:rr`rrs@r-rrrs{ z d      vU\velvvvvvvvvr,ra Beit Model transformer with a 'language' modeling head on top. BEiT does masked image modeling by predicting visual tokens of a Vector-Quantize Variational Autoencoder (VQ-VAE), whereas other vision models like ViT and DeiT predict RGB pixel values. As a result, this class is incompatible with [`AutoModelForMaskedImageModeling`], so you will need to use [`BeitForMaskedImageModeling`] directly if you wish to do masked image modeling with BEiT. ceZdZdeddffd ZdZee ddej dzdej dzd ej dzd e d ej dzd e e deezfd ZxZS)BeitForMaskedImageModelingr4r5NcHt||j|_t|d|_t j|j|j|_ t j |j|j |_ | dS)NFrr)rgr8 num_labelsrbeitrrr<rrr vocab_sizelm_headrrs r-r8z#BeitForMaskedImageModeling.__init__s     +f>>> f&8f>STTTy!3V5FGG  r,cdSrar+)rHs r-get_output_embeddingsz0BeitForMaskedImageModeling.get_output_embeddingsstr,FrIrJlabelsrVrrc ,|j|f|||d|}|j}||}||ddddf} d} |%t j} | | ||} t | | |j|jS)a bool_masked_pos (`torch.BoolTensor` of shape `(batch_size, num_patches)`): Boolean masked positions. Indicates which patches are masked (1) and which aren't (0). labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the image classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Examples: ```python >>> from transformers import AutoImageProcessor, BeitForMaskedImageModeling >>> import torch >>> from PIL import Image >>> import requests >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> image_processor = AutoImageProcessor.from_pretrained("microsoft/beit-base-patch16-224-pt22k") >>> model = BeitForMaskedImageModeling.from_pretrained("microsoft/beit-base-patch16-224-pt22k") >>> num_patches = (model.config.image_size // model.config.patch_size) ** 2 >>> pixel_values = image_processor(images=image, return_tensors="pt").pixel_values >>> # create random boolean mask of shape (batch_size, num_patches) >>> bool_masked_pos = torch.randint(low=0, high=2, size=(1, num_patches)).bool() >>> outputs = model(pixel_values, bool_masked_pos=bool_masked_pos) >>> loss, logits = outputs.loss, outputs.logits >>> list(logits.shape) [1, 196, 8192] ```)rJrVrNr"losslogitsr attentions) rrrrrCrossEntropyLossrrr ) rHrIrJrrVrroutputsrprediction_scoresmasked_lm_lossloss_fcts r-r`z"BeitForMaskedImageModeling.forwardsX$)  +%=)      "3..99 LLABB)?@@  *,,H%X&7&H&QQN$!/)     r,)NNNFN)r'r(r)r#r8rrrr:rrbrrrrjrr`rrs@r-rr~s z d      -137&*)..2 @ @ lT)@ )D0@  t# @ #' @  t+ @ +,@   @ @ @ ^@ @ @ @ @ r,rz Beit Model transformer with an image classification head on top (a linear layer on top of the average of the final hidden states of the patch tokens) e.g. for ImageNet. ceZdZdeddffd Zee d dejdzdejdzde d e e de e zf d ZxZS) BeitForImageClassificationr4r5Nc:t||j|_t|d|_|jdkrt j|j|jnt j|_ | dS)NTrr) rgr8rrrrrr<r classifierrrs r-r8z#BeitForImageClassification.__init__s     +f=== OUN_bcNcNc")F$68IJJJikitiviv r,FrIrrVrc |j|fd|i|}|j}||}d}|||||j}t |||j|jS)a labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the image classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). rVNr )rrr loss_functionr4r rr ) rHrIrrVrrrr r s r-r`z"BeitForImageClassification.forwards$)   %=     - //  %%ffdkBBD$!/)     r,NNF)r'r(r)r#r8rrr:rrrrrjr r`rrs@r-rrs z d      -1&*).   lT)   t#  #'  +,  & &    ^      r,rceZdZ ddededeeeefzd ed eeeefzezd ed eeeefzd edeffd ZxZS) BeitConvLayerrr"rFrelu in_channels out_channels kernel_sizestridepaddingrdilationgroups activationc ttj|||||||||_dS)N)rrrr r!r"r#r)rgr8rConv2d convolution) rHrrrr r!rr"r#r$rps r-r8zBeitConvLayer.__init__sR 9#%#    r,)rr"rFr"r"r) r'r(r)rrjstrrr8rrs@r-rrs .//0*+     5c?*    uS#X&,   c3h'            r,rcheZdZdedededdffd Zdejdeeefdejfd ZxZ S) BeitPyramidPoolingBlock pool_scalerchannelsr5Ncttj||_t ||d|_dS)Nr"r)rgr8rAdaptiveAvgPool2dpoolingrconv)rHr+rr,rps r-r8z BeitPyramidPoolingBlock.__init__/sE +J77 !+xQGGG r,inputrQc||}||}tj||dd}|S)NrFr)r0r1rrr)rHr2rQ hidden_states r-r`zBeitPyramidPoolingBlock.forward4sI||E** yy.. }00Dzin0oo r,) r'r(r)rr8r:rrjr`rrs@r-r*r*.sH3HSHCHDHHHHHH U\sCxU\r,r*cteZdZdZdeedfdededdffd Zd ejde ejfd Z xZ S) BeitPyramidPoolingModuleak Pyramid Pooling Module (PPM) used in PSPNet. Args: pool_scales (tuple[int]): Pooling scales used in Pooling Pyramid Module. in_channels (int): Input channels. channels (int): Channels after modules, before conv_seg. Based on OpenMMLab's implementation, found in https://github.com/open-mmlab/mmsegmentation. pool_scales.rr,r5Nct||_|_|_t jfd|D|_dS)Nc4g|]}t|S))r+rr,)r*)rr+r,rs r-rz5BeitPyramidPoolingModule.__init__..Ns9   (:;aijjj   r,)rgr8r7rr,rrblocks)rHr7rr,rps ``r-r8z!BeitPyramidPoolingModule.__init__Hst &&  m     "-      r,rcfddfd|jDS)Nrc*g|]}|S))rQr+)rblockr original_sizes r-rz4BeitPyramidPoolingModule.forward..Vs(RRRUm-888RRRr,)rQr:)rHrr>s `@r-r`z BeitPyramidPoolingModule.forwardTs=%**,,QRR0 RRRRRdkRRRRr,) r'r(r)r*rjrr8r:rrkr`rrs@r-r6r6;s    E#s(O  #  QT  Y]       SU\Sd5<6HSSSSSSSSr,r6ceZdZdZdeddffd ZdeejdejfdZ deejdejfd Z xZ S) BeitUperHeadz Unified Perceptual Parsing for Scene Understanding. This head is the implementation of [UPerNet](https://huggingface.co/papers/1807.10221). Based on OpenMMLab's implementation, found in https://github.com/open-mmlab/mmsegmentation. r4r5Nc ~t|j|_|jgdz|_|j|_t j|j|jd|_ t|j|jd|j|_ t|jdt|j|jzz|jdd|_t j|_t j|_|jddD]f}|jt||jd|jt|j|jddgtt|j|jz|jdd|_dS)Nr"r.rMrrr!)rgr8r7r<rr,rr&rrr6 psp_modulesrlenpsp_bottleneckr lateral_convs fpn_convsappendfpn_bottleneck)rHr4rrps r-r8zBeitUperHead.__init__as !-"./!3* )DM63DRSTTT4    R M   ,  R 3t'7#8#84=#H H M     ]__+CRC0 i iK   % %mK\]&^&^&^ _ _ _ N ! !- t}Z[ef"g"g"g h h h h+  ! !DM 1 M    r,rc|d}tj|g||d}||SrL)r:rUrDrF)rHrr4s r- psp_forwardzBeitUperHead.psp_forwardsK$R( y,!P1A1A,1O1O!PVWXXX ""<000r,encoder_hidden_statescg}t|j|D]#\}}|||$|||t |}t |dz ddD]U}||dz jdd}||dz tj |||ddz||dz <Vg}t |dz D]1}||j |||2||dt |dz ddD]A}tj |||djdddd||<Btj |d}| |} || } | S) Nr"rrMrrFrrN)ziprGrIrLrErrPrrrrHr:rUrJr) rHrMlaterals lateral_convr4used_backbone_levelsr prev_shapefpn_outsoutputs r-r`zBeitUperHead.forwards*-d.@BW*X*X 8 8 &L, OOLL66 7 7 7 7(()>??@@@ #8}}+a/B77  A!!a%.qrr2J&q1uo 0I0I *:U1J11HQUOO +a/00 < ). Args: config (BeitConfig): Configuration. in_channels kernel_size (int): The kernel size for convs in the head. Default: 3. dilation (int): The dilation rate for convs in the head. Default: 1. Based on OpenMMLab's implementation, found in https://github.com/open-mmlab/mmsegmentation. rrr"r4in_indexrr"r5Nc t|j|_|j|_|j|_|j|_ ||_ |dz|z}tj |_ |jdkr|j t|j|j|||t!|jdz D]8}|j t|j|j|||9|j r-t|j|jz|j||dz|_tj|j|jd|_dS)Nrr)rr!r"r"rCr.)rgr8r<rauxiliary_channelsr,auxiliary_num_convs num_convsauxiliary_concat_input concat_inputrXrrconvsrIrrconv_catr&rr)rHr4rXrr" conv_paddingrWrps r-r8zBeitFCNHead.__init__sz !-1 3"9  #q(H4 ]__ >A   J  $dmVbmu    4>A-..   !!!  $/ ,!)    ) 4=0$-[bmqrbrDM)DM63DRSTTTr,rMc||j}|}|jD] }||}|jr+|t j||gd}||}|S)Nr"rN)rXr_r^r`r:rUr)rHrMrrr1s r-r`zBeitFCNHead.forwards~(7 J 0 0D D//MM   W MM%)X}4MST*U*U*UVVM 66 r,)rrr") r'r(r)r*r#rrjr8rkr:rr`rrs@r-rWrWs  no!U!U !U,/!UBE!UUX[`adfiai[jUj!U !U!U!U!U!U!UFT%,-?ELr,rWc ZeZdZdZd dedededdffd Zd ejdejfd ZxZ S) BeitFPNUpBlockuE4x upsampling block: ConvTranspose → BN → GELU → ConvTranspose.rr<rr r5Ncttj|||||_tj||_tj|_tj|||||_ dS)Nrr ) rgr8rConvTranspose2dconv_transpose1 BatchNorm2d normalizationGELUr$conv_transpose2)rHr<rr rps r-r8zBeitFPNUpBlock.__init__sz !1+{Xclrsss^K88'))!1+{Xclrsssr,rc||}||}||}||}|Sra)rhrjr$rlrs r-r`zBeitFPNUpBlock.forwardsR,,];; **=99  66 ,,];; r,)rr) r'r(r)r*rr8r:rr`rrs@r-rdrdsOOttCtctstSWttttttU\elr,rdcleZdZdZdeffd ZdeejdfdeejdffdZ xZ S) BeitFPNNeckz 4-level feature pyramid neck for BeiT. Produces x4 upsample, x2 upsample, identity, and x2 downsample outputs from the four selected ViT feature maps. r4ctt|j|_t j|j|jdd|_t jdd|_ dS)Nrrf) rgr8rdr<fpn1rrgfpn2 MaxPool2dfpn4rs r-r8zBeitFPNNeck.__init__sh "6#566 &v'96;M[\efggg LQq999 r, feature_maps.r5c||d||d|d||dfSrf)rqrrrt)rHrus r-r`zBeitFPNNeck.forwardsL IIl1o & & IIl1o & & O IIl1o & &   r,) r'r(r)r*r#r8rjr:rr`rrs@r-roros :z::::::  E%,*;$< u|UXGXAY        r,roceZdZdeddffd Zeee d dej dzdej dzde d e e de ezf d ZxZS) BeitForSemanticSegmentationr4r5Nct||j|_t|d|_t |jjdkrtdt||_ t||_ |j rt|nd|_|dS)NFrrBzBeitForSemanticSegmentation requires config.out_indices to be a list of 4 integers, specifying which features to use from the backbone. One can use [3, 5, 7, 11] in case of a base-sized architecture.)rgr8rrrrEr4 out_indices ValueErrorrofpnr@ decode_headuse_auxiliary_headrWauxiliary_headrrs r-r8z$BeitForSemanticSegmentation.__init__s     +f>>> t{& ' '1 , ,-  v&&(//5;5NXk&111TX r,FrIrrVrc L ||jjdkrtd|j|fd|i|}|j|j\ }}}||jjz||jjzt fd|jjD} | | } | | } d} |j | | } d} |.| | ||jj | |jj} t| | |j|jS)aD labels (`torch.LongTensor` of shape `(batch_size, height, width)`, *optional*): Ground truth semantic segmentation maps for computing the loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels > 1`, a classification loss is computed (Cross-Entropy). Examples: ```python >>> from transformers import AutoImageProcessor, BeitForSemanticSegmentation >>> from PIL import Image >>> import requests >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> image_processor = AutoImageProcessor.from_pretrained("microsoft/beit-base-finetuned-ade-640-640") >>> model = BeitForSemanticSegmentation.from_pretrained("microsoft/beit-base-finetuned-ade-640-640") >>> inputs = image_processor(images=image, return_tensors="pt") >>> outputs = model(**inputs) >>> # logits are of shape (batch_size, num_labels, height, width) >>> logits = outputs.logits ```Nr"z/The number of labels should be greater than onerVc3K|]E}|dz ddddfdddVFdS)r"NrrM) transposer)rrr[rM patch_height patch_widths r- z6BeitForSemanticSegmentation.forward..Usx   "!a% (ABB / 9 9!Q ? ? G G TVXdfq r r      r,) ignore_indexauxiliary_logitsauxiliary_loss_weightr )r4rr{rrrPrArjrzr|r}rrsemantic_loss_ignore_indexrrr )rHrIrrVrrrWrXrYrur rr r[rMrrs @@@@r-r`z#BeitForSemanticSegmentation.forward%sB  $+"8A"="=NOO O$)   %=    !( 5'3'9$ Avu!77 t{55        [,     xx -- !!,//   *#22<@@   %%![C!1&*k&G &D'!/)     r,r)r'r(r)r#r8rr rr:rrrrrjrr`rrs@r-rxrxszd* -1&*). G G lT)G  t#G #' G +, G ( ( G G G ^! G G G G G r,rxzM BEiT backbone, to be used with frameworks like DETR and MaskFormer. c neZdZfdZeeededee de fdZ xZ S) BeitBackbonecLtfdtjdzD|_t d|_jrtntj |_ | dS)Ncg|] }j Sr+)r<)rrWr4s r-rz)BeitBackbone.__init__..{s]]]AV/]]]r,r"Fr) rgr8rr num_featuresrradd_fpnrorrr|rrs `r-r8zBeitBackbone.__init__xs    ]]]]v?WZ[?[9\9\]]]f>>> *0.K;v&&&bkmm r,rIrr5c |j\}}}}||jjz}||jjz}|j|fi|} | j} d} t |j| D]_\} } | |jvrQ|jjr?| ddddddf} | dd} | |d||} | | fz } `| | } t| | j| j S)a: Examples: ```python >>> from transformers import AutoImageProcessor, AutoBackbone >>> import torch >>> from PIL import Image >>> import requests >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> processor = AutoImageProcessor.from_pretrained("microsoft/beit-base-patch16-224") >>> model = AutoBackbone.from_pretrained( ... "microsoft/beit-base-patch16-224", out_features=["stage1", "stage2", "stage3", "stage4"] ... ) >>> inputs = processor(image, return_tensors="pt") >>> outputs = model(**inputs) >>> feature_maps = outputs.feature_maps >>> list(feature_maps[-1].shape) [1, 768, 14, 14] ```r+Nr"rrM)rurr )rPr4rArrrO stage_names out_featuresreshape_hidden_statesrrr|r r )rHrIrr[rWrXrYrrrrrustager4s r-r`zBeitBackbone.forwards&@(4'9$ Avu!77 t{55 $)L33F33-  #&t'7#G#G 0 0 E<)));4c#/122qqq#9L#/#9#9!Q#?#?L#/#7#7 B Va#b#bL/ xx -- %!/)    r,) r'r(r)r8rr rrrrr r`rrs@r-rrrs  3 3 +,3   3 3 3 ^! 3 3 3 3 3 r,r)rrrxrrr)Hr* dataclassesrr:rrrrbackbone_utilsrr masking_utilsr modeling_outputsr r r rrmodeling_utilsrprocessing_utilsr pytorch_utilsrutilsrrr utils.genericrrutils.output_capturingrresnet.modeling_resnetrswin.modeling_swinrvit.modeling_vitrrrrr r!configuration_beitr#r&r/r3r7rdrrrrrrrrrrr*r6r@rWrdrorxr__all__r+r,r-rs!!!!!! &&&&&&HHHHHHHH666666.-----&&&&&&@@@@@@BBBBBBBBBBIIIIIIII555555444444------tttttttttttttttt******  !;        ,   ''''']'''TV3V3V3V3V3ryV3V3V3r`````L```     f        <   77777777tTTTTT,TTT.JjJjJjJjJj#JjJjJjZ v v v v v v v vS S S S S !4S S S l / / / / / !4/ /  / d     O   4     bi   SSSSSrySSS