tj1nJddlZddlZddlmZddlZddlmZddlmcmZ ddl m Z ddl mZmZddlmZddlmZddlmZmZdd lmZdd lmZdd lmZmZmZdd lm Z dd l!m"Z"ddl#m$Z$m%Z%Gddej&Z'Gddej&Z(Gddej&Z)GddeZ*Gddej&Z+GddeZ,edeGddeZ-Gd d!ej&Z.Gd"d#ej/Z0Gd$d%ej&Z1Gd&d'e*Z2Gd(d)e*Z3Gd*d+e*Z4eeGd,d-eZ5ed.Gd/d0e*Z6gd1Z7dS)2N) dataclass)initialization)ACT2CLSACT2FN)filter_output_hidden_states)GradientCheckpointingLayer)BaseModelOutput ModelOutput)PreTrainedModel)Unpack)TransformersKwargsauto_docstringcan_return_tuple)merge_with_config_defaults)capture_outputs) SLANeXtConfigSLANeXtVisionConfigc eZdZdZfdZdededejdejfdZdejd ejd ejde eefde eefdejf d Z dd ejde ejejffdZ xZ S)SLANeXtVisionAttentionz=Multi-head Attention block with relative position embeddings.ct|dkr|j|jz|j|jzfn||f}|j|_|j|jz}|dz|_|j|_tj |j|jdz|j |_ tj |j|j|_ |j|_|jr|tdtjt#jd|dzdz ||_tjt#jd|dzdz ||_dSdS)NrgrbiaszBInput size must be provided if using relative positional encoding.r)super__init__ image_size patch_sizenum_attention_heads hidden_sizescaleattention_dropoutdropoutnnLinearqkv_biasqkvproj use_rel_pos ValueError Parametertorchzeros rel_pos_h rel_pos_w)selfconfig window_size input_sizehead_dim __class__s j/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/models/slanext/modeling_slanext.pyrzSLANeXtVisionAttention.__init__.sZ a &"3 3V5F&J[5[ \ \{+  $*#= %)CCt^ / 9V/1Ca1Gfo^^^If0&2DEE !-   X! !efff \%+a*Q-6G!6KX*V*VWWDN\%+a*Q-6G!6KX*V*VWWDNNN  X Xq_sizek_sizerel_posreturncltdt||zdz }tj|d|jdddd|d}|d|dd}tj |dddft||z dz}tj |dddft||z dz}||z |dz t||z dzz}|| S) a Get relative positional embeddings according to the relative positions of query and key sizes. Args: q_size (int): size of the query. k_size (int): size of key k. rel_pos (`torch.Tensor`): relative position embeddings (L, channel). Returns: Extracted positional embeddings according to relative positions. rrrlinear)sizemodeN?) intmaxF interpolatereshapeshape transposepermuter-arangelong) r1r9r:r; max_rel_distrel_pos_resizedq_coordsk_coordsrelative_coordss r7 get_rel_posz"SLANeXtVisionAttention.get_rel_posGs0 1s66222Q677 - OOAw}Q/ 4 4 > >q! D D    *11"lCCKKAqQQ<''403v3L3LL<''aaa03v3L3LL#h.6A:Vf_VYAZAZ2ZZ335566r8queryr/r0cj|\}}|\}} ||||} ||| |} |j\} } }|| |||}tjd|| }tjd|| }|dddddddddf|dddddddddfz}|S)a Calculate decomposed Relative Positional Embeddings from :paper:`mvitv2`. https://github.com/facebookresearch/mvit/blob/19786631e330df9f3622e5402b4a419a263a2c80/mvit/models/attention.py Args: query (`torch.Tensor`): query q in the attention layer with shape (batch_size, query_height * query_width, channel). rel_pos_h (`torch.Tensor`): relative position embeddings (Lh, channel) for height axis. rel_pos_w (`torch.Tensor`): relative position embeddings (Lw, channel) for width axis. q_size (tuple): spatial sequence size of query q with (query_height, query_width). k_size (tuple): spatial sequence size of key k with (key_height, key_width). Returns: decomposed_rel_pos (`torch.Tensor`): decomposed relative position embeddings. zbhwc,hkc->bhwkzbhwc,wkc->bhwkN)rRrHrGr-einsum)r1rSr/r0r9r: query_height query_width key_height key_widthrelative_position_heightrelative_position_width batch_size_dimreshaped_queryrel_hrel_wdecomposed_rel_poss r7get_decomposed_rel_posz-SLANeXtVisionAttention.get_decomposed_rel_posgs8%+! k & I#'#3#3L*i#X#X "&"2"2; 9"U"U"[ Asz<cRR -~?WXX -~?VWW"111aaaAAAt#34uQQQ111dAAA=M7NN!!r8N hidden_statesc|j\}}}}|||||zd|jdddddd}|d||jz||zdd\}} } ||jz| ddz} |jrA| ||j |j ||f||f} | | } | | z} tjj| tjd|j} tj| |j|j } | | z||j||d}|ddddd|||d}||}|| fS) Nrr>rrr)dtyper^)ptraining)rHr(rGr rJunbindr"rIr*rcr/r0 reshape_asr-r% functionalsoftmaxfloat32torhr$rjr))r1rdoutput_attentionsr\heightwidthr]r(rSkeyvalue attn_weightsrb attn_probs attn_outputs r7forwardzSLANeXtVisionAttention.forwards'4':$ FE1 HH] # # WZ%D4Lb Q Q WQ1a # #  KK:8P+PRX[`R`bdeellmnoosE *cmmB.C.CC   =!%!!>|!L!L '*<9;GGr8 prev_hidden batch_hidden char_onehotskwargsc (||}||d}||z}tj|}||}t j|dtj |j }| dd}tj || d} tj| |gd} || |} | |fS)Nrr^rhr)rr unsqueezer-tanhrrErnrorprhrImatmulsqueezecatr) r1rrrrbatch_hidden_projprev_hidden_projattention_scoresrvcontextconcat_contextrds r7ryzSLANeXtAttentionGRUCell.forwards!00>>00==GGJJ,/?? :&677::&677y!1q NNNQQRbRhii #--a33 ,|\::BB1EEG\#:A>>== l**r8) r{r|r}rr- FloatTensorr rryrrs@r7rrsHHHHH+&+'+' + +, ++++++++r8rc&eZdZdfd ZdZxZS) SLANeXtMLPNc ttj|||_tj|||_|tjnt||_dSrz) rrr%r&fc1fc2Identityract_fn)r1r! out_channels activationr6s r7rzSLANeXtMLP.__init__sd 9[+669[,77'1'9bkmmmwz?R?T?T r8c||}||}||}|Srz)rrrr1rds r7ryzSLANeXtMLP.forwards;// //  M22 r8rz)r{r|r}rryrrs@r7rrsRUUUUUU r8rcleZdZUeed<dZdZdZdZddgZ e j fdZ xZ S) SLANeXtPreTrainedModelr2backbone pixel_valuesimageTstructure_attention_cellstructure_generatorcDt|t|tr!|jt j|jdt|tr;|jr4t j|j dt j|j dt|tj r|j dkrdtj|j z nd}t j|j| |t j|j| ||jt j|j| ||jt j|j| |t|t*rdtj|jj dzz }|jfD]t}|D][}t|tjr?t j|j| ||jt j|j| |\sdSdS)zInitialize the weightsNgrrB)r _init_weights isinstanceSLANeXtVisionEncoder pos_embedinit constant_rr*r/r0r%rr!mathsqrtuniform_ weight_ih weight_hhbias_ihbias_hhSLANeXtSLAHeadr2rchildrenr&weightr)r1modulestd generatorlayerr6s r7rz$SLANeXtPreTrainedModel._init_weightss f%%% f2 3 3 6+v/555 f4 5 5 6! 6v/555v/555 fbj ) ) 99?9Ka9O9O# &"45555UVC M&*SD# 6 6 6 M&*SD# 6 6 6~) fnsdC888~) fnsdC888 fn - - A $+"9C"?@@@C$8: A A &//11AAE!%33A elSD#>>> :1 M%*sdC@@@ A  A A A Ar8)r{r|r}r__annotations__base_model_prefixmain_input_nameinput_modalitiessupports_gradient_checkpointing_keep_in_fp32_modules_strictr-no_gradrrrs@r7rrs "$O!&*#$>@U#V U]__"A"A"A"A_"A"A"A"A"Ar8rcBeZdZfdZdejdejfdZxZS)SLANeXtMLPBlockcttj|j|j|_tj|j|j|_t|j |_ dSrz) rrr%r&r!mlp_dimlin1lin2r hidden_actactr1r2r6s r7rzSLANeXtMLPBlock.__init__s\ If0&.AA Ifnf.@AA &+,r8rdr<c||}||}||}|Srz)rrrrs r7ryzSLANeXtMLPBlock.forwards; -00 //  -00 r8)r{r|r}rr-rryrrs@r7rr s^----- U\elr8rc eZdZfdZdejdedeejeeefffdZdejdedeeefdeeefdejf d Z dejdeej fd Z xZ S) SLANeXtVisionLayerc<ttj|j|j|_t|||_tj|j|j|_ t||_ ||_ dS)N)eps) rrr% LayerNormr!layer_norm_eps layer_norm1rattn layer_norm2rmlpr3)r1r2r3r6s r7rzSLANeXtVisionLayer.__init__s <(:@UVVV*6;?? <(:@UVVV"6**&r8rdr3r<c `|j\}}}}|||zz |z}|||zz |z}tj|ddd|d|f}||z||z} } ||| |z|| |z||}|ddddddd|||} | | | ffS)a Args: Partition into non-overlapping windows with padding if needed. hidden_states (tensor): input tokens with [batch_size, height, width, channel]. window_size (int): window size. Returns: windows: windows after partition with [batch_size * num_windows, window_size, window_size, channel]. (pad_height, pad_width): padded height and width before partition rrrrrfr>)rHrEpadrGrJ contiguous) r1rdr3r\rrrschannelpad_hpad_w pad_height pad_widthwindowss r7window_partitionz#SLANeXtVisionLayer.window_partition$s.;-@* FE7v 33{Bu{22kAmaAua-GHH & I %--  k1; [@XZegn   ''1aAq99DDFFNNrS^`kmtuuY///r8r padding_shapeoriginal_shapec\|\}}|\}}|jd||z|z|zz} || ||z||z||d} | dddddd| ||d} | ddd|d|ddf} | S) aS Args: Window unpartition into original sequences and removing padding. hidden_states (tensor): input tokens with [batch_size * num_windows, window_size, window_size, channel]. window_size (int): window size. padding_shape (Tuple): padded height and width (pad_height, pad_width). original_shape (Tuple): original height and width (height, width) before padding. Returns: hidden_states: unpartitioned sequences with [batch_size, height, width, channel]. rr>rrrrfrN)rHrGrJr) r1rr3rrrrrrrsr\rds r7window_unpartitionz%SLANeXtVisionLayer.window_unpartition<s"!. I& ]1%*y*@K*OS^*^_   k19 3K[Zegi    ! !!Q1a 3 3 > > @ @ H HU_ajln o o &aaa&&5&!!!&;<GGII r8c|}||}|jdkr8|jd|jd}}|||j\}}||\}}|jdkr|||j|||f}||z}||}|||z}|S)Nrrr)rd)rr3rHrrrrr)r1rdresidualrrrsrrvlayernorm_outputs r7ryzSLANeXtVisionLayer.forwardZs ((77  a  )/2M4G4JEF+/+@+@PTP`+a+a (M=&*ii''0' ' # |  a   33M4CSUbekmrdsttM =0 ++M::%1A(B(BB r8) r{r|r}rr-rrCrrrrryrrs@r7rrs'''''0el00QVW\WcejknpsksetWtQu00000|25FKCQTHoglmprumugv <U\eEM5*C/047>>>7;Je',-4;;;;;r8rc(eZdZdZfdZdZxZS)SLANeXtPatchEmbeddingsz This class turns `pixel_values` of shape `(batch_size, num_channels, height, width)` into the initial `hidden_states` (patch embeddings) of shape `(batch_size, seq_length, hidden_size)` to be consumed by a Transformer. ct|j|j}}|j|j}}t |tjj r|n||f}t |tjj r|n||f}|d|dz|d|dzz}||_||_||_||_ tj |||||_ dS)Nrr) kernel_sizestride)rrrr num_channelsr!r collectionsabcIterable num_patchesr%Conv2d projection)r1r2rrrr!rr6s r7rzSLANeXtPatchEmbeddings.__init__s !'!2F4EJ $*$79Kk #-j+/:R#S#SqZZZdfpYq #-j+/:R#S#SqZZZdfpYq !!} 1 5*Q-:VW=:XY $$(&)L+:^hiiir8c P|j\}}}}||jkrtd||jdks||jdkr2td|d|d|jdd|jdd ||ddd d}|S) NzeMake sure that the channel dimension of the pixel values match with the one set in the configuration.rrzInput image size (*z) doesn't match model (z).rr)rHrr+rrrJ)r1rr\rrrrs embeddingss r7ryzSLANeXtPatchEmbeddings.forwards2>2D/ L&% 4, , ,w  T_Q' ' '5DOA4F+F+FwVwwewwDO\]L^wwaeapqraswww __\22::1aAFF r8)r{r|r}r~rryrrs@r7rrsV j j j j j       r8rcReZdZdZdddfd Zdejdejffd ZxZS) SLANeXtLayerNormaALayerNorm that supports two data formats: channels_last (default) or channels_first. The ordering of the dimensions in the inputs. channels_last corresponds to inputs with shape (batch_size, height, width, channels) while channels_first corresponds to inputs with shape (batch_size, channels, height, width). gư> channels_last)r data_formatc ztj|fd|i||dvrtd|||_dS)Nr)r channels_firstzUnsupported data format: )rrNotImplementedErrorr )r1normalized_shaperr rr6s r7rzSLANeXtLayerNorm.__init__sY)==s=f=== A A A%&O+&O&OPP P&r8featuresr<c|jdkrR|dddd}t|}|dddd}n!t|}|S)z Args: features: Tensor of shape (batch_size, channels, height, width) OR (batch_size, height, width, channels) r rrrr)r rJrry)r1rr6s r7ryzSLANeXtLayerNorm.forwardsw  / / /''1a33Hwwx00H''1a33HHwwx00Hr8) r{r|r}r~rr-rryrrs@r7r r s 15/'''''''            r8r c*eZdZdeffd ZdZxZS)SLANeXtVisionNeckr2c`t||_tj|j|jdd|_t|jd|_ tj|j|jddd|_ t|jd|_ dS)NrF)rrr )r r)rpaddingr) rrr2r%rr!output_channelsconv1r rconv2rrs r7rzSLANeXtVisionNeck.__init__s  Yv163IWX_deee +F,BP`aaaYv5v7M[\fgnsttt +F,BP`aaar8c|dddd}||}||}||}||}|S)Nrrrr)rJrrrrrs r7ryzSLANeXtVisionNeck.forwardsi%--aAq99  =11 ((77  =11 ((77 r8)r{r|r}rrryrrs@r7rrs[b2bbbbbbr8rc eZdZeedZdZdeffd ZdZ e e d d d e j dzd eed eezfd ZxZS)r)rdrrr2ct|||_|j|_t ||_d|_|jrMtj tj d|j|j z|j|j z|j |_tj|_t!|jD]=}t%|||jvr|jnd}|j|>t-||_d|_|dS)Nrr)r3F)rrr2rr patch_embedr use_abs_posr%r,r-r.rr! ModuleListlayersrangenum_hidden_layersrglobal_attn_indexesr3appendrneckgradient_checkpointing post_init)r1r2irr6s r7rzSLANeXtVisionEncoder.__init__s6      +1&99   \ %)::%)::& DNmoo v/00 & &A&236;U2U2UF..[\E K  u % % % %%f-- &+# r8c|jSrz)r)r1s r7get_input_embeddingsz)SLANeXtVisionEncoder.get_input_embeddingss r8F)tie_last_hidden_statesNrrr<c |td||}|j ||jz}|jD] }||}||}t |S)Nz You have to specify pixel_values)r)r+rrrr$r)r1rrrd layer_modules r7ryzSLANeXtVisionEncoder.forwards  ?@@ @((66 > %)DN:M K 8 8L(L77MM -00 )+    r8rz)r{r|r}rr_can_record_outputsrrrr)rrr-rr rrrryrrs@r7rrs,>Ndee!2>    _E2227;  !-4 GMN`Ga + +   32      r8rcTeZdZ ddedzffd ZdejdeefdZ xZ S)SLANeXtBackboneNr2c t|t|j|_t j|j|jdddd|_ | dS)NrrrF)rrrr) rrr vision_config vision_towerr%rpost_conv_in_channelspost_conv_out_channels post_convr&r1r2rr6s r7rzSLANeXtBackbone.__init__ sw    01EFF  (&*GUV_`jkrw    r8rdrc |j|fi|}||j}|ddd}t ||j|jS)Nrr)rrdr)r2r5rflattenrIr rdr)r1rdr vision_outputs r7ryzSLANeXtBackbone.forwardsy))-BB6BB }'FGG %--a00::1a@@ +'5$/    r8rz) r{r|r}dictrr-rr rryrrs@r7r/r/ s}#  t        U\ VDV=W        r8r/c eZdZdeiZ d dedzffd Zeee d de j de j dzde efdZxZS) rrNr2c t|t|j|j|j|_t|j|j|_| dSrz) rrrr4r!rrrrr&r6s r7rzSLANeXtSLAHead.__init__)so    (?  )6+=v?R) ) %$.f.@&BU#V#V  r8rdtargetsrc tj|jd|jjftj|j}tj|jdgtj|j}g}g}t|jj dzD]}tj ||jj  } ||| | \}}||} | d}|| ||tj|d|jj dz drntjtj|ddtj|j} t3| |S) Nr)rhdevice)r@rhr?r)r^r>r)rrd)r-r.rHr2r!ror?rLr max_text_lengthrEone_hotrfloatrrargmaxr#stackeqanyallrnrprhr ) r1rdr=rrpredicted_charsstructure_preds_liststructure_ids_listr]embedding_featurestructure_stepstructure_predss r7ryzSLANeXtSLAHead.forward7s;   #T[%< =U][h[o    +M,?,B+C5:^k^rsss!t{2Q677  A ! /4;;S T T Z Z \ \ 77-BUBUBWBWYjkkKHa!55h??N,333::O ' ' 7 7 7  % %o 6 6 6{-1555889QTU9UVVZZ[]^^bbdd  )EK0D!$L$L$LRT\a\ijjjmm    Pdeeeer8rz)r{r|r}rr-r:rrrrr-rrr rryrrs@r7rr$s- #  t         (,ff(f$f+, fff! _ fffffr8rcPeZdZUdZdZejdzed<dZejdzed<dS) SLANeXtForTableRecognitionOutputam head_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Hidden-states of the SLANeXtSLAHead at each prediction step, varies up to max `self.config.max_text_length` states (depending on early exits). head_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Attentions of the SLANeXtSLAHead at each prediction step, varies up to max `self.config.max_text_length` attentions (depending on early exits). Nhead_hidden_stateshead_attentions) r{r|r}r~rPr-rrrQrr8r7rOrOXsO48)D077704OU&-44444r8rOz SLANeXt Table Recognition model for table recognition tasks. Wraps the core SLANeXtPreTrainedModel and returns outputs compatible with the Transformers table recognition API. c eZdZdeffd Zeedejde e de eje zfdZ xZS)SLANeXtForTableRecognitionr2ct|t||_t ||_|dS)N)r2)rrr/rrheadr&rs r7rz#SLANeXtForTableRecognition.__init__msU    'v666 "&111  r8rrr<c |j|fi|}|j|jfi|}t|j|j|j|j|jS)N)rrdrrPrQ)rrUrrOrdr)r1rrbackbone_outputs head_outputss r7ryz"SLANeXtForTableRecognition.forwardssp )4=@@@@ ty!1!CNNvNN /*<*8'2+9(3     r8)r{r|r}rrrrr-rr rrrOryrrs@r7rSrSfs}   !-  9?@R9S  u !$D D    ^      r8rS)rr/rSr)8rr dataclassesrr-torch.nnr%torch.nn.functionalrmrErr activationsrrbackbone_utilsrmodeling_layersr modeling_outputsr r modeling_utilsr processing_utilsr utilsrrr utils.genericrutils.output_capturingrconfiguration_slanextrrModulerrrrrrrrrr rrr/rrOrS__all__rr8r7risv, !!!!!! &&&&&&********999999999999<<<<<<<<------&&&&&&IIIIIIIIII777777555555EEEEEEEEB)B)B)B)B)RYB)B)B)J+++++bi+++B        +A+A+A+A+A_+A+A+A\     bi   QQQQQ3QQQh   < < < < < < <   <     RY   Fr|4 (6 6 6 6 6 16 6 6 r     ,   01f1f1f1f1f+1f1f1fh  5 5 5 5 5 5 5  5      !7    . h g gr8