tjxfdZddlZddlmZddlZddlmZddlmZddlm Z m Z ddl m Z dd l mZdd lmZmZmZmZmZdd lmZd d lmZer ddlmZmZndZdZedeGddeZedeGddeZedeGddeZGddejZ GddejZ!Gdd ejZ"Gd!d"ejZ#Gd#d$ejZ$Gd%d&ejZ%Gd'd(ejZ&Gd)d*ejZ'Gd+d,ejZ(Gd-d.ejZ)Gd/d0ejZ*Gd1d2ejZ+eGd3d4eZ,eGd5d6e,Z-ed7Gd8d9e,Z.ed:Gd;dz9PyTorch Dilated Neighborhood Attention Transformer model.N) dataclass)nn)ACT2FN) BackboneMixinfilter_output_hidden_states)BackboneOutput)PreTrainedModel) ModelOutputOptionalDependencyNotAvailableauto_docstringis_natten_availablerequires_backends)can_return_tuple) DinatConfig) natten2dav natten2dqkrpbctNr argskwargss f/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/models/dinat/modeling_dinat.pyrr),...ctrrrs rrr,rrzO Dinat encoder's outputs, with potential hidden states and attentions. ) custom_introceZdZUdZdZejdzed<dZe ejdfdzed<dZ e ejdfdzed<dZ e ejdfdzed<dS)DinatEncoderOutputa reshaped_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each stage) of shape `(batch_size, hidden_size, height, width)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs reshaped to include the spatial dimensions. Nlast_hidden_state. hidden_states attentionsreshaped_hidden_states) __name__ __module__ __qualname____doc__r"torch FloatTensor__annotations__r#tupler$r%rrr!r!0s37u(4/666:>M5*C/047>>>7;Je',-4;;;CGE%"3S"89D@GGGGGrr!zW Dinat model's outputs that also contains a pooling of the last hidden states. ceZdZUdZdZejdzed<dZejdzed<dZ e ejdfdzed<dZ e ejdfdzed<dZ e ejdfdzed<dS) DinatModelOutputa pooler_output (`torch.FloatTensor` of shape `(batch_size, hidden_size)`, *optional*, returned when `add_pooling_layer=True` is passed): Average pooling of the last layer hidden-state. reshaped_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each stage) of shape `(batch_size, hidden_size, height, width)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs reshaped to include the spatial dimensions. Nr" pooler_output.r#r$r%) r&r'r(r)r"r*r+r,r1r#r-r$r%r.rrr0r0Fs  37u(4/666.2M5$t+222:>M5*C/047>>>7;Je',-4;;;CGE%"3S"89D@GGGGGrr0z1 Dinat outputs for image classification. ceZdZUdZdZejdzed<dZejdzed<dZ e ejdfdzed<dZ e ejdfdzed<dZ e ejdfdzed<dS) DinatImageClassifierOutputa7 loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided): Classification (or regression if config.num_labels==1) loss. logits (`torch.FloatTensor` of shape `(batch_size, config.num_labels)`): Classification (or regression if config.num_labels==1) scores (before SoftMax). reshaped_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each stage) of shape `(batch_size, hidden_size, height, width)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs reshaped to include the spatial dimensions. Nlosslogits.r#r$r%) r&r'r(r)r4r*r+r,r5r#r-r$r%r.rrr3r3_s  &*D% d ")))'+FE  $+++:>M5*C/047>>>7;Je',-4;;;CGE%"3S"89D@GGGGGrr3cXeZdZdZfdZdejdzdeejfdZ xZ S)DinatEmbeddingsz6 Construct the patch and position embeddings. ctt||_t j|j|_t j|j |_ dSr) super__init__DinatPatchEmbeddingspatch_embeddingsr LayerNorm embed_dimnormDropouthidden_dropout_probdropoutselfconfig __class__s rr:zDinatEmbeddings.__init__sU  4V < <L!122 z&"<== r pixel_valuesNreturnc||}||}||}|Sr)r<r?rB)rDrG embeddingss rforwardzDinatEmbeddings.forwards=**<88 YYz** \\*-- r) r&r'r(r)r:r*r+r-TensorrK __classcell__rFs@rr7r7zss>>>>>E$5$<u|ATrr7cLeZdZdZfdZdejdzdejfdZxZ S)r;z This class turns `pixel_values` of shape `(batch_size, num_channels, height, width)` into the initial `hidden_states` (patch embeddings) of shape `(batch_size, height, width, hidden_size)` to be consumed by a Transformer. c Rt|j}|j|j}}||_|dkrnt dt jt j|j|dzdddt j|dz|ddd|_ dS)Nz2Dinat only supports patch size of 4 at the moment.rrrRrRrr) kernel_sizestridepadding) r9r: patch_size num_channelsr> ValueErrorr SequentialConv2d projection)rDrErYrZ hidden_sizerFs rr:zDinatPatchEmbeddings.__init__s & $*$79Ik ( ?? QRR R- Id')9vV\fl m m m IkQ& PV`f g g g  rrGNrHc|j\}}}}||jkrtd||}|dddd}|S)NzeMake sure that the channel dimension of the pixel values match with the one set in the configuration.rrRrr)shaperZr[r^permute)rDrG_rZheightwidthrJs rrKzDinatPatchEmbeddings.forwardsh)5);&< 4, , ,w __\22 ''1a33 r) r&r'r(r)r:r*r+rLrKrMrNs@rr;r;sn      " E$5$<         rr;cleZdZdZejfdedejddffd Zde j de j fdZ xZ S) DinatDownsamplerz Convolutional Downsampling Layer. Args: dim (`int`): Number of input channels. norm_layer (`nn.Module`, *optional*, defaults to `nn.LayerNorm`): Normalization layer class. dim norm_layerrHNct||_tj|d|zdddd|_|d|z|_dS)NrRrSrTrUF)rVrWrXbias)r9r:rhrr] reductionr?)rDrhrirFs rr:zDinatDownsampler.__init__s] 3CVF\binoooJq3w'' r input_featurec||dddddddd}||}|S)NrrrrR)rlrbr?)rDrms rrKzDinatDownsampler.forwardsV}'<'