PhOdZddlZddlmZmZddlZddlmZddlmZddl m Z m Z m Z ddl mZdd lmZmZd d lmZej(eZd ed efdZd+deeefdefdZGddej8ZGddej<ZGddej8Z Gddej8Z!Gddej8Z"Gddej8Z#Gddej8Z$Gd d!ej8Z%eGd"d#eZ&eGd$d%e&Z'ed&'Gd(d)e&Z(gd*Z)y),zPyTorch EfficientNet model.N)OptionalUnion)nn)ACT2FN)BaseModelOutputWithNoAttention(BaseModelOutputWithPoolingAndNoAttention$ImageClassifierOutputWithNoAttention)PreTrainedModel)auto_docstringlogging)EfficientNetConfigconfig num_channelsc|j}||jz}t|t||dz z|z|z}|d|zkr||z }t|S)z< Round number of filters based on depth multiplier. g?) depth_divisorwidth_coefficientmaxint)rrdivisornew_dims p/var/www/html/saasai/venv/lib/python3.12/site-packages/transformers/models/efficientnet/modeling_efficientnet.py round_filtersr%sf""GF,,,L'3|gk9:gEOPG|##7 w< kernel_sizeadjustct|tr||f}|ddz|ddzf}|r|ddz |d|ddz |dfS|d|d|d|dfS)aJ Utility function to get the tuple padding value for the depthwise convolution. Args: kernel_size (`int` or `tuple`): Kernel size of the convolution layers. adjust (`bool`, *optional*, defaults to `True`): Adjusts padding value to apply to right and bottom sides of the input. rrr) isinstancer)rrcorrects r correct_padr"4s}+s#"K0 1~"KNa$78G  Q GAJNGAJGG GAJ GAJ??rc`eZdZdZdeffd ZdejdejfdZxZ S)EfficientNetEmbeddingszL A module that corresponds to the stem module of the original work. rct|t|d|_t j d|_t j|j|jdddd|_ t j|j|j|j |_ t|j|_y) N )rrrrpaddingrrvalidFrstrider(bias)epsmomentum)super__init__rout_dimr ZeroPad2dr(Conv2dr convolution BatchNorm2dbatch_norm_epsbatch_norm_momentum batchnormr hidden_act activationselfr __class__s rr0zEfficientNetEmbeddings.__init__Ms $VR0 ||L9 99   1QPW^c  &:O:OZ`ZtZtu !2!23r pixel_valuesreturnc|j|}|j|}|j|}|j|}|SN)r(r4r8r:)r<r>featuress rforwardzEfficientNetEmbeddings.forwardXsA<< -##H->>(+??8,r) __name__ __module__ __qualname____doc__rr0torchTensorrC __classcell__r=s@rr$r$Hs0 41 4ELLU\\rr$c.eZdZ dfd ZxZS)EfficientNetDepthwiseConv2dc @||z} t ||| ||||||| y)N) in_channels out_channelsrr+r(dilationgroupsr, padding_mode)r/r0) r<rOdepth_multiplierrr+r(rQr,rSrPr=s rr0z$EfficientNetDepthwiseConv2d.__init__bs=#%55  #%#%  r)rrrrrTzeros)rDrErFr0rJrKs@rrMrMas$   rrMcleZdZdZdedededeffd Zdejdejfd Z xZ S) EfficientNetExpansionLayerz_ This corresponds to the expansion phase of each block in the original implementation. rin_dimr1r+ct|tj||ddd|_tj ||j |_t|j|_ y)NrsameFrOrPrr(r,) num_featuresr-) r/r0rr3 expand_convr5r6 expand_bnrr9 expand_act)r<rrXr1r+r=s rr0z#EfficientNetExpansionLayer.__init__sZ 99   W&BWBWX !2!23r hidden_statesr?cl|j|}|j|}|j|}|SrA)r]r^r_r<r`s rrCz"EfficientNetExpansionLayer.forwards4((7 }5  6 r) rDrErFrGrrr0rH FloatTensorrIrCrJrKs@rrWrW{sH 41 43 4 4VY 4U%6%65<<rrWc peZdZdZdededededef fd Zdejd ejfd Z xZ S) EfficientNetDepthwiseLayerzk This corresponds to the depthwise convolution phase of each block in the original implementation. rrXr+radjust_paddingcbt|||_|jdk(rdnd}t||}t j ||_t||||d|_t j||j|j|_ t|j|_y) Nrr)rZ)rr'Fr*r\r-r.)r/r0r+r"rr2depthwise_conv_padrMdepthwise_convr5r6r7depthwise_normrr9 depthwise_act) r<rrXr+rrfconv_padr(r=s rr0z#EfficientNetDepthwiseLayer.__init__s  "kkQ.7Fk.A"$,,w"?9  FHSX !nnV%:%:VE_E_ $F$5$56rr`r?c|jdk(r|j|}|j|}|j|}|j |}|S)Nr)r+rirjrkrlrbs rrCz"EfficientNetDepthwiseLayer.forwardsT ;;!  33MBM++M: ++M: **=9 r rDrErFrGrrboolr0rHrcrIrCrJrKs@rreresZ7"77 7  7  7, U%6%6 5<< rrec neZdZdZd dedededeffd Zdejdejfd Z xZ S) EfficientNetSqueezeExciteLayerzl This corresponds to the Squeeze and Excitement phase of each block in the original implementation. rrX expand_dimexpandct||r|n||_tdt ||j z|_tjd|_ tj|j|j dd|_ tj|j |jdd|_ t|j|_tj |_y)Nr) output_sizerZ)rOrPrr()r/r0dimrrsqueeze_expansion_ratiodim_serAdaptiveAvgPool2dsqueezer3reducertrr9 act_reduceSigmoid act_expand)r<rrXrsrtr=s rr0z'EfficientNetSqueezeExciteLayer.__init__s !':V!S&*H*H!HIJ ++: ii   ii    !!2!23**,rr`r?c|}|j|}|j|}|j|}|j|}|j |}t j ||}|SrA)r{r|r}rtrrHmul)r<r`inputss rrCz&EfficientNetSqueezeExciteLayer.forwardsc ]3  M2  6  M2  6  &-8 r)FrorKs@rrrrrsH'1'3'C'Y]'* U%6%6 5<< rrrc eZdZdZdedededededef fd Zd e jd e jd e jfd Z xZ S) EfficientNetFinalBlockLayerz[ This corresponds to the final phase of each block in the original implementation. rrXr1r+ drop_rateid_skipct||dk(xr| |_tj||ddd|_tj ||j|j|_ tj||_ y)NrrZFr[rhp) r/r0 apply_dropoutrr3 project_convr5r6r7 project_bnDropoutdropout)r<rrXr1r+rrr=s rr0z$EfficientNetFinalBlockLayer.__init__sz #q[8[II   .. f&;&;fF`F` zzI. r embeddingsr`r?c|j|}|j|}|jr|j|}||z}|SrA)rrrr)r<rr`s rrCz#EfficientNetFinalBlockLayer.forwardsG))-8  6    LL7M)J6MrrDrErFrGrrfloatrpr0rHrcrIrCrJrKs@rrrsj/(/25/@C/MP/]b/mq/"%"3"3EDUDUZ_ZfZfrrceZdZdZdededededededed ed effd Zd e jd e jfdZ xZ S)EfficientNetBlocka This corresponds to the expansion and depthwise convolution phase of each block in the original implementation. Args: config ([`EfficientNetConfig`]): Model configuration class. in_dim (`int`): Number of input channels. out_dim (`int`): Number of output channels. stride (`int`): Stride size to be used in convolution layers. expand_ratio (`int`): Expand ratio to set the output dimensions for the expansion and squeeze-excite layers. kernel_size (`int`): Kernel size for the depthwise convolution layer. drop_rate (`float`): Dropout rate to be used in the final phase of each block. id_skip (`bool`): Whether to apply dropout and sum the final hidden states with the input embeddings during the final phase of each block. Set to `True` for the first block of each stage. adjust_padding (`bool`): Whether to apply padding to only right and bottom side of the input kernel before the depthwise convolution operation, set to `True` for inputs with odd input sizes. rrXr1r+ expand_ratiorrrrfc lt |||_|jdk7|_||z} |jrt ||| ||_t ||jr| n|||| |_t||| |j|_ t||jr| n||||||_ y)Nr)rrXr1r+)rrXr+rrf)rrXrsrt)rrXr1r+rr) r/r0rrtrW expansionrerjrrsqueeze_exciter projection) r<rrXr1r+rrrrrf expand_in_dimr=s rr0zEfficientNetBlock.__init__s (''1, - ;;7fmFDN9$(KK=V#)  =&]4;; 6$(KK=V  rr`r?c|}|jdk7r|j|}|j|}|j|}|j ||}|S)Nr)rrrjrr)r<r`rs rrCzEfficientNetBlock.forwardHs["    ! NN=9M++M: ++M:  MB rrrKs@rrrs4' "' '  '  '  ' ' ' ' ' R U%6%6 5<< rrc feZdZdZdeffd Z d dejdee dee de fdZ xZ S) EfficientNetEncoderz Forward propagates the embeddings through each EfficientNet block. Args: config ([`EfficientNetConfig`]): Model configuration class. rc t|_|j_fdt |j }t fd|jD}d}g}t|D]}t||j |}t||j|}|j|} |j|} |j|} t|j|D]c} | dk(} | dkDrdn| } | dkDr|n|}||jv}|j|z|z }t!|||| | | || | }|j#||dz }et%j&|_t%j*t|dddd _t%j.|j0|j2|j4 _t8|j:_y) NcZttjj|zSrA)rmathceildepth_coefficient)repeatsr<s r round_repeatsz3EfficientNetEncoder.__init__..round_repeatscs"tyy!7!7'!ABC Crc3.K|] }|ywrA).0nrs r z/EfficientNetEncoder.__init__..hsL3Kaq)3Ksrr) rrXr1r+rrrrrfirZFr[rh)r/r0rrlenrOsumnum_block_repeatsrangerrPstrides kernel_sizes expand_ratiosdepthwise_paddingdrop_connect_raterappendr ModuleListblocksr3top_convr5 hidden_dimr6r7top_bnrr9top_activation)r<rnum_base_blocks num_blockscurr_block_numrirXr1r+rrjrrfrblockrr=s` @rr0zEfficientNetEncoder.__init__^s  !'!9!9 Df001L63K3KLL 'A"66+=+=a+@AF#FF,?,?,BCG^^A&F --a0K!//2L=)A)A!)DEFq&!e$%Ev!/v7O7O!O"44~E R )!!#! +!-'##1  e$!#'G(8mmF+  &vt4   nn**0E0EPVPjPj  %V%6%67rr`output_hidden_states return_dictr?c|r|fnd}|jD]}||}|s||fz }|j|}|j|}|j|}|st d||fDSt ||S)Nc3&K|] }|| ywrAr)rvs rrz.EfficientNetEncoder.forward..sX$Fq!-$Fs)last_hidden_stater`)rrrrtupler)r<r`rrall_hidden_statesrs rrCzEfficientNetEncoder.forwards 1E],$[[E!-0M#!m%55!!  m4  M2 ++M: X]4E$FXX X-++  r)FT) rDrErFrGrr0rHrcrrprrCrJrKs@rrrUsW58158t05&*  (( 'tn d^  (  rrcDeZdZUeed<dZdZgZdejfdZ y)EfficientNetPreTrainedModelr efficientnetr>modulecPt|tjtjtjfrm|j j jd|jj|j%|jj jyyy)zInitialize the weightsg)meanstdN) r rLinearr3r5weightdatanormal_rinitializer_ranger,zero_)r<rs r _init_weightsz)EfficientNetPreTrainedModel._init_weightsss fryy"))R^^D E MM   & &CT[[5R5R & S{{&   &&(' FrN) rDrErFr__annotations__base_model_prefixmain_input_name_no_split_modulesrModulerrrrrrs) &$O)BII)rrc ~eZdZdeffd Ze ddeejdee dee de e e ffdZ xZS) EfficientNetModelrct||||_t||_t ||_|jdk(r'tj|jd|_ nN|jdk(r'tj|jd|_ ntd|j|jy)NrT) ceil_moderz2config.pooling must be one of ['mean', 'max'] got )r/r0rr$rrencoder pooling_typer AvgPool2drpooler MaxPool2d ValueErrorpooling post_initr;s rr0zEfficientNetModel.__init__s   08*62    & (,,v'8'8DIDK  E ),,v'8'8DIDKQRXR`R`Qabc c rr>rrr?c~||n|jj}||n|jj}| td|j |}|j |||}|d}|j |}|j|jdd}|s ||f|ddzSt|||jS)Nz You have to specify pixel_valuesrrrrr)r pooler_outputr`) rruse_return_dictrrrrreshapeshaper r`)r<r>rrembedding_outputencoder_outputsr pooled_outputs rrCzEfficientNetModel.forwards%9$D $++JjJj &1%> ? " " rrz EfficientNet Model with an image classification head on top (a linear layer on top of the pooled features), e.g. for ImageNet. ) custom_introceZdZfdZe ddeejdeejdee dee de e e ff dZ xZS) "EfficientNetForImageClassificationct|||j|_||_t ||_t j|j|_ |jdkDr*t j|j|jnt j|_ |jy)Nrr)r/r0 num_labelsrrrrr dropout_raterrrIdentity classifierrr;s rr0z+EfficientNetForImageClassification.__init__s   ++ -f5zzF$7$78 KO??]^K^"))F$5$5tGdfdododq rr>labelsrrr?cd||n|jj}|j|||}|r |jn|d}|j |}|j |}d}||j |||j}|s|f|ddz} ||f| zS| St|||jS)a labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the image classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Nrrr)losslogitsr`) rrrrrr loss_functionr r`) r<r>rrroutputsrrroutputs rrCz*EfficientNetForImageClassification.forward s&1%r se" " ! .,:   H % , C @U3:.@@(RYY2 ")) 44$$N$RYY$N"))BN NbW "))W t )/ ) ) 5 35 5 p / )D/  / d er