tjsdZddlZddlZddlZddlmZddlmZddl m Z ddl m Z m Z ddlmZdd lmZmZdd lmZdd lmZmZdd lmZd dlmZejeZGddejZ ej!j"dZ#dZ$GddejZ%GddejZ&GddejZ'GddejZ(dZ)dZ*GddejZ+Gdd eZ,Gd!d"ejZ-eGd#d$eZ.eGd%d&e.Z/ed'(Gd)d*e e.Z0gd+Z1dS),zPyTorch ViTDet backbone.N)nn)initialization)ACT2FN) BackboneMixinfilter_output_hidden_states)GradientCheckpointingLayer)BackboneOutputBaseModelOutput)PreTrainedModel)auto_docstringlogging)can_return_tuple) VitDetConfigcLeZdZdZfdZdZdejdejfdZxZ S)VitDetEmbeddingsz This class turns `pixel_values` of shape `(batch_size, num_channels, height, width)` into the initial `hidden_states` (patch embeddings) to be consumed by a Transformer. cXt|j|j}}|j|j}}t |tjj r|n||f}t |tjj r|n||f}|d|dz|d|dzz}||_ ||_||_||_ |j r8|dz}tjtjd||j|_nd|_tj|||||_dS)Nrr) kernel_sizestride)super__init__pretrain_image_size patch_size num_channels hidden_size isinstance collectionsabcIterable image_size num_patches use_absolute_position_embeddingsr Parametertorchzerosposition_embeddingsConv2d projection) selfconfigr!rrrr" num_positions __class__s h/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/models/vitdet/modeling_vitdet.pyrzVitDetEmbeddings.__init__*s$ !'!;V=NJ $*$79Kk #-j+/:R#S#SqZZZdfpYq #-j+/:R#S#SqZZZdfpYq !!} 1 5*Q-:VW=:XY $$(&  2 ,'!OM')|EK=RXRd4e4e'f'fD $ $'+D $)L+:^hiiic|r|ddddf}|jd}ttj|}||z|krt dt js ||ks||kritj | d||d dddd||fdd }| ddddS| d||dS) a Calculate absolute positional embeddings. If needed, resize embeddings and remove cls_token dimension for the original embeddings. Args: abs_pos_embeddings (`torch.Tensor`): Absolute positional embeddings with (1, num_position, num_channels). has_cls_token (`bool`): If true, has 1 embedding in abs_pos_embeddings for cls token. height (`int`): Height of input image tokens. width (`int`): Width of input image tokens. Returns: Absolute positional embeddings after processing with shape (1, height, width, num_channels) Nrz5Absolute position embeddings must be a square number.rrbicubicF)sizemode align_corners) shapeintmathsqrt ValueErrorr%jit is_tracingr functional interpolatereshapepermute)r*abs_pos_embeddings has_cls_tokenheightwidth num_positionr4new_abs_pos_embeddingss r.get_absolute_positionsz'VitDetEmbeddings.get_absolute_positions@s$  ;!3AAAqrrE!: )/2 49\**++ $;, & &TUU U 9   ! ! Ddfnn %']%>%>"**1dD"==EEaAqQQe_# &?&& "*11!Q1== =%--aCC Cr/ pixel_valuesreturncp|jd}||jkrtd|jd|d||}|jf|dddd}|||jd|jd|jdz}|dddd}|S) NrzoMake sure that the channel dimension of the pixel values match with the one set in the configuration. Expected z but got .rr2rT)r7rr;r)r'rArH)r*rIr embeddingss r.forwardzVitDetEmbeddings.forwardfs#)!, 4, , ,I!.II9EIII __\22  # /#++Aq!Q77J#d&A&A($ 0@0CZEUVWEX''J$++Aq!Q77Jr/) __name__ __module__ __qualname____doc__rrHr%TensorrN __classcell__r-s@r.rr$s jjjjj,$D$D$DLELU\r/rctdt||zdz }|jd|krtj|d|jddddd|d}|d|dd}n|}tj |dddft||z dz}tj |dddft||z dz}||z |dz t||z dzz}|| S) a Get relative positional embeddings according to the relative positions of query and key sizes. Args: q_size (`int`): Size of query q. k_size (`int`): Size of key k. rel_pos (`torch.Tensor`): Relative position embeddings (num_embeddings, num_channels). Returns: Extracted positional embeddings according to relative positions. r2rrr1linear)r4r5Ng?) r8maxr7rr>r?r@rAr%arangelong)q_sizek_sizerel_pos max_rel_distrel_pos_resizedq_coordsk_coordsrelative_coordss r. get_rel_posrc|sQ q3vv...233L}Q<''-33 OOAw}Q/ 4 4 < bhwkzbhwc,wkc->bhwkN)rcr7r@r%einsumview)attnqueries rel_pos_h rel_pos_wr[r\queries_height queries_width keys_height keys_widthrelative_heightrelative_width batch_size_dimr_qrelative_weights r.!add_decomposed_relative_positionsrvs,%+!NM$K!.+yIIO  IFFN J3 //*nmS I ICl#3S/JJOl#3S.IIO *nm[*UU !!!QQQ111d* + , !!!QQQ4* + , d:~ 5{Z7OPP  Kr/c,eZdZdZdfd ZddZxZS)VitDetAttentionz=Multi-head Attention block with relative position embeddings.Nct|j}|j}||_||z}|dz|_t j||dz|j|_ t j|||_ |j |_ |j rrt j tjd|dzdz ||_t j tjd|dzdz ||_dSdS)z Args: config (`VitDetConfig`): Model configuration. input_size (`tuple[int]`, *optional*): Input resolution, only required in case relative position embeddings are added. grbiasr2rrN)rrrnum_attention_heads num_headsscalerLinearqkv_biasqkvproj use_relative_position_embeddingsr$r%r&rirj)r*r+ input_sizersr}head_dimr-s r.rzVitDetAttention.__init__s  . ")#t^ 9S#'@@@Ic3'' 060W-  0 X\%+a*Q-6G!6KX*V*VWWDN\%+a*Q-6G!6KX*V*VWWDNNN X Xr/Fc 4|j\}}}}|||||zd|jdddddd}|d||jz||zdd\}} } ||jz| ddz} |jr"t| ||j |j ||f||f} | d} | | z}| ||j||d}|ddddd}||||d}||}|r8| ||j| jd| jd} || f} n|f} | S) Nrr1r2rr)rs)r7rr@r}rAunbindr~ transposerrvrirjsoftmaxrfr)r* hidden_stateoutput_attentionsrqrDrErrrrhkeysvaluesattention_scoresattention_probsoutputss r.rNzVitDetAttention.forwards'3'9$ FE1hh|$$,,Z%DN\^__gghiklnoqrtuvv # AzDN/JFUZN\^ _ _ f fgh i iv#dj0DNN2r4J4JJ  0 @ '4>4>FTY?]cej\k   +22r2::&/ #((T^VUTVWW #++Aq!Q:: #++JrJJ yy..  &-55DNO,A",EG\]_G`O$_5GG#oGr/NFrOrPrQrRrrNrTrUs@r.rxrxs]GGXXXXXX4r/rxc*eZdZdZdfd ZdZxZS)VitDetLayerNormaL A LayerNorm variant, popularized by Transformers, that performs point-wise mean and variance normalization over the channel dimension for inputs that have shape (batch_size, channels, height, width). https://github.com/facebookresearch/ConvNeXt/blob/d1fa8f6fef0a165b27399986cc2bdacc92777e40/models/convnext.py#L119 ư>cttjt j||_tjt j||_||_ |f|_ dSr) rrrr$r%onesweightr&r{epsnormalized_shape)r*rrr-s r.rzVitDetLayerNorm.__init__si l5:.>#?#?@@ L-=!>!>?? !1 3r/c"|dd}||z ddd}||z tj||jzz }|jddddf|z|jddddfz}|S)NrT)keepdimr2)meanpowr%r:rrr{)r*xuss r.rNzVitDetLayerNorm.forwards FF1dF # # UKKNN  4  0 0 UejTX.. . K4 & *TYqqq$}-E Er/)rrrUs@r.rrsV 444444r/rc(eZdZdZfdZdZxZS)VitDetResBottleneckBlockz The standard bottleneck residual block without the last activation layer. It contains 3 conv layers with kernels 1x1, 3x3, 1x1. cttj||dd|_t ||_t|j|_ tj||ddd|_ t ||_ t|j|_ tj||dd|_ t ||_dS)ar Args: config (`VitDetConfig`): Model configuration. in_channels (`int`): Number of input channels. out_channels (`int`): Number of output channels. bottleneck_channels (`int`): Number of output channels for the 3x3 "bottleneck" conv layers. rFrzr)paddingr{N)rrrr(conv1rnorm1r hidden_actact1conv2norm2act2conv3norm3)r*r+ in_channels out_channelsbottleneck_channelsr-s r.rz!VitDetResBottleneckBlock.__init__$s Y{,?OOO $%899 6,- Y24GTU\abbb $%899 6,- Y2L!%PPP $\22 r/cX|}|D] }||}||z}|Sr)children)r*routlayers r.rNz VitDetResBottleneckBlock.forward<s;]]__  E%**CC#g r/rrUs@r.rrsQ 333330r/rcPeZdZdededdffd ZdejdejfdZxZS) VitDetMlp in_featureshidden_featuresrJNcttj|||_t |j|_tj|||_tj |j |_ dSr) rrrrfc1rractfc2Dropout dropout_probdrop)r*r+rrr-s r.rzVitDetMlp.__init__Fsf 9[/::&+,9_k::Jv233 r/rc||}||}||}||}||}|Sr)rrrr)r*rs r.rNzVitDetMlp.forwardMsR HHQKK HHQKK IIaLL HHQKK IIaLLr/) rOrPrQr8rr%rSrNrTrUs@r.rrEsx4C4#4$444444%,r/rc j|j\}}}}| |z}| |z}tj|ddd|d|f}||z||z} }||z} | |z} ||| || ||}|ddddddd|||} | || ffS)a  Partition into non-overlapping windows with padding if needed. Args: hidden_state (`torch.Tensor`): Input tokens with [batch_size, height, width, num_channels]. window_size (`int`): Window size. Returns: `tuple(torch.FloatTensor)` comprising various elements: - windows: windows after partition with [batch_size * num_windows, window_size, window_size, num_channels]. - (padded_height, padded_width): padded height and width before partition rrrr2rr1)r7rr>padrfrA contiguous) r window_sizerqrDrEr pad_height pad_width padded_height padded_widthn_hn_wwindowss r.window_partitionrWs/;.@+J|'[(J;&I=$$\Aq!Y:3VWWL"(:"5uy7H`_. drop_probrJNcVt||_dSr)rrr)r*rr-s r.rzVitDetDropPath.__init__s$ "r/ hidden_statesc|jdks|js|Sd|jz }|jdfd|jdz zz}t j||j|j}t j||z}| ||zS)Nrrr)r)dtypedevice) rtrainingr7ndimr%randrrfloordiv)r*r keep_probr7 random_tensors r.rNzVitDetDropPath.forwards >S   & $Q')DM4F4J,KK 5 0CML`aaa  MI$=>>   ++m;;r/cd|jS)Nzp=)rr*s r. extra_reprzVitDetDropPath.extra_reprs$DN$$$r/)r) rOrPrQrRfloatrr%rSrNstrrrTrUs@r.rrs ##%#$######}dN^._._ +M+!% /"0" " /q1 (,  a  .}d>NP`cikpbqrrM!4>>-#@#@@ %txx =@Y@Y7Z7Z([([[ %--aAq99  " 9 MM-88M "W,r/)rrFr)rOrPrQrRrrr8boolrr%rSrrNrTrUs@r.rrsMMqv!!"!49!LO!im! !!!!!!L#(''|' ' u|U\) *U5<-@ @ ''''''''r/rc \eZdZdeddffd Z d dejded ed edee zf d Z xZ S) VitDetEncoderr+rJNc t||_|j}dt jd|j|dD}g}t|D]E}|t|||||j vr|j nd||j vFtj||_d|_dS)Nc6g|]}|S)item).0rs r. z*VitDetEncoder.__init__..s jjjq!&&((jjjr/rcpu)r)rrrF)rrr+num_hidden_layersr%linspacerrangeappendrwindow_block_indicesrresidual_block_indicesr ModuleListrgradient_checkpointing)r*r+depthrlayersir-s r.rzVitDetEncoder.__init__s  (kjEN1f>SUZch,i,i,ijjju  A MM#1!#4676;V6V6V 2 2\]'(F,I'I     ]6** &+###r/FTrroutput_hidden_states return_dictc|rdnd}|rdnd}t|jD]/\}}|r||fz}|||} | d}|r || dfz}0|r||fz}|std|||fDSt|||S)Nrrrc3K|]}||V dSrr)rvs r. z(VitDetEncoder.forward..,s(mmq_`_l_l_l_l_lmmr/last_hidden_stater attentions) enumeraterrr ) r*rrrrall_hidden_statesall_self_attentionsr layer_module layer_outputss r.rNzVitDetEncoder.forwards#7@BBD$5?bb4(44 P POA|# I$58H$H!(L8IJJM)!,M  P&9]1=M>> from transformers import VitDetConfig, VitDetModel >>> import torch >>> config = VitDetConfig() >>> model = VitDetModel(config) >>> pixel_values = torch.randn(1, 3, 224, 224) >>> with torch.no_grad(): ... outputs = model(pixel_values) >>> last_hidden_states = outputs.last_hidden_state >>> list(last_hidden_states.shape) [1, 768, 14, 14] ```Nz You have to specify pixel_values)rrrrrr) r+rrrr;rMr<r rr) r*rIrrrkwargsembedding_outputencoder_outputssequence_outputs r.rNzVitDetModel.forwardfs82C1N--TXT_Tq$8$D $+Jj &1%.s]]]AV/]]]r/r) rrrrMrr<r r num_featuresr=r>s `r.rzVitDetBackbone.__init__s    *622$V,, ]]]]v?WZ[?[9\9\]]] r/rJc|jjSrr@rs r.rAz#VitDetBackbone.get_input_embeddingsrBr/NrIrrrc ||n |jj}||n |jj}||n |jj}||}||d||}|r|jn|d}d} t|j|D]\} } | |j vr| | fz } |s!|r| f|ddz} n| f|ddz} | St| |r|jnd|j S)a Examples: ```python >>> from transformers import VitDetConfig, VitDetBackbone >>> import torch >>> config = VitDetConfig() >>> model = VitDetBackbone(config) >>> pixel_values = torch.randn(1, 3, 224, 224) >>> with torch.no_grad(): ... outputs = model(pixel_values) >>> feature_maps = outputs.feature_maps >>> list(feature_maps[-1].shape) [1, 768, 14, 14] ```NT)rrrrrr2) feature_mapsrr) r+rrrrMr<rzip stage_names out_featuresr r) r*rIrrrrDrErrrPstageroutputs r.rNzVitDetBackbone.forwardsH<&1%res &&&&&&!!!!!!HHHHHHHH999999????????------,,,,,,,,------......  H % %UUUUUryUUUp!3!3!3H&&&R;;;;;bi;;;|bi,$$$$$ry$$$N $222><<<<%%%%%RY%%%0MMMMM,MMM`5 5 5 5 5 BI5 5 5 p+++++O+++DD D D D D 'D D D N M M M M M ]$9M M  M ` E D Dr/