tjOdZddlZddlZddlmZddlmZddlmZddl m Z m Z m Z ddl mZdd lmZmZd d lmZejeZd ed efdZd,deezdefdZGddejZGddejZGddejZ GddejZ!GddejZ"GddejZ#Gdd ejZ$Gd!d"ejZ%eGd#d$eZ&eGd%d&e&Z'ed'(Gd)d*e&Z(gd+Z)dS)-zPyTorch EfficientNet model.N)nn)initialization)ACT2FN)BaseModelOutputWithNoAttention(BaseModelOutputWithPoolingAndNoAttention$ImageClassifierOutputWithNoAttention)PreTrainedModel)auto_docstringlogging)EfficientNetConfigconfig num_channelsc|j}||jz}t|t||dz z|z|z}|d|zkr||z }t|S)z< Round number of filters based on depth multiplier. g?) depth_divisorwidth_coefficientmaxint)rrdivisornew_dims t/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/models/efficientnet/modeling_efficientnet.py round_filtersr$sk"GF,,L'3|gk9::gEOPPG|###7 w<<T kernel_sizeadjustct|tr||f}|ddz|ddzf}|r$|ddz |d|ddz |dfS|d|d|d|dfS)aJ Utility function to get the tuple padding value for the depthwise convolution. Args: kernel_size (`int` or `tuple`): Kernel size of the convolution layers. adjust (`bool`, *optional*, defaults to `True`): Adjusts padding value to apply to right and bottom sides of the input. rrr ) isinstancer)rrcorrects r correct_padr!3s+s##1"K0 1~"KNa$78G @ Q GAJNGAJGG GAJ GAJ??rcLeZdZdZdeffd ZdejdejfdZxZ S)EfficientNetEmbeddingszL A module that corresponds to the stem module of the original work. rc|tt|d|_t jd|_t j|j|jdddd|_ t j |j|j |j |_ t|j|_dS) N )rr rr paddingrrvalidFrstrider'bias)epsmomentum)super__init__rout_dimr ZeroPad2dr'Conv2dr convolution BatchNorm2dbatch_norm_epsbatch_norm_momentum batchnormr hidden_act activationselfr __class__s rr/zEfficientNetEmbeddings.__init__Ls $VR00 |L999 9  1QPW^c    &:OZ`Ztuuu !23r pixel_valuesreturnc||}||}||}||}|SN)r'r3r7r9)r;r=featuress rforwardzEfficientNetEmbeddings.forwardWsM<< --##H-->>(++??8,,r) __name__ __module__ __qualname____doc__rr/torchTensorrB __classcell__r<s@rr#r#Gsu 41 4 4 4 4 4 4ELU\rr#c.eZdZ dfd ZxZS)EfficientNetDepthwiseConv2dr rrTzerosc f||z} t|| ||||||| dS)N) in_channels out_channelsrr*r'dilationgroupsr+ padding_mode)r.r/) r;rOdepth_multiplierrr*r'rQr+rSrPr<s rr/z$EfficientNetDepthwiseConv2d.__init__asV#%55  #%#%  r)r rr rr TrM)rCrDrEr/rIrJs@rrLrL`sT           rrLcXeZdZdZdedededeffd Zdejdej fd Z xZ S) EfficientNetExpansionLayerz_ This corresponds to the expansion phase of each block in the original implementation. rin_dimr0r*cttj||ddd|_tj||j|_t|j |_ dS)Nr sameFrOrPrr'r+) num_featuresr,) r.r/rr2 expand_convr4r5 expand_bnrr8 expand_act)r;rrWr0r*r<s rr/z#EfficientNetExpansionLayer.__init__so 9     W&BWXXX !23r hidden_statesr>c||}||}||}|Sr@)r\r]r^r;r_s rrBz"EfficientNetExpansionLayer.forwards=((77 }55  66 r) rCrDrErFrrr/rG FloatTensorrHrBrIrJs@rrVrVzs 41 43 4 4VY 4 4 4 4 4 4U%65<rrVc \eZdZdZdededededef fd Zdej d ej fd Z xZ S) EfficientNetDepthwiseLayerzk This corresponds to the depthwise convolution phase of each block in the original implementation. rrWr*radjust_paddingcvt||_|jdkrdnd}t||}t j||_t||||d|_t j ||j |j |_ t|j|_dS) Nrr(rY)rr&Fr)r[r,r-)r.r/r*r!rr1depthwise_conv_padrLdepthwise_convr4r5r6depthwise_normrr8 depthwise_act) r;rrWr*rreconv_padr'r<s rr/z#EfficientNetDepthwiseLayer.__init__s  "kQ..77Fk.AAA"$,w"?"?"?9  FHSX   !nV%:VE_   $F$56rr_r>c|jdkr||}||}||}||}|S)Nr)r*rhrirjrkras rrBz"EfficientNetDepthwiseLayer.forwardsa ;!   33MBBM++M:: ++M:: **=99 r rCrDrErFrrboolr/rGrbrHrBrIrJs@rrdrds7"77 7  7  777777, U%6 5<        rrdc ZeZdZdZd dedededeffd Zdej d ej fd Z xZ S) EfficientNetSqueezeExciteLayerzl This corresponds to the Squeeze and Excitement phase of each block in the original implementation. FrrW expand_dimexpandct|r|n||_tdt ||jz|_tjd|_ tj |j|jdd|_ tj |j|jdd|_ t|j|_tj|_dS)Nr ) output_sizerY)rOrPrr')r.r/dimrrsqueeze_expansion_ratiodim_serAdaptiveAvgPool2dsqueezer2reducersrr8 act_reduceSigmoid act_expand)r;rrWrrrsr<s rr/z'EfficientNetSqueezeExciteLayer.__init__s !'3::V!S&*H!HIIJJ +::: i     i      !!23*,,rr_r>c|}||}||}||}||}||}t j||}|Sr@)rzr{r|rsr~rGmul)r;r_inputss rrBz&EfficientNetSqueezeExciteLayer.forwardss ]33  M22  66  M22  66  &-88 r)FrnrJs@rrqrqs''1'3'C'Y]''''''* U%6 5<        rrqc neZdZdZdedededededef fd Zd e j d e j d e j fd Z xZ S) EfficientNetFinalBlockLayerz[ This corresponds to the final phase of each block in the original implementation. rrWr0r* drop_rateid_skipc t|dko| |_tj||ddd|_tj||j|j|_ tj ||_ dS)Nr rYFrZrgp) r.r/ apply_dropoutrr2 project_convr4r5r6 project_bnDropoutdropout)r;rrWr0r*rrr<s rr/z$EfficientNetFinalBlockLayer.__init__s #q[8[I     . f&;fF`   zI... r embeddingsr_r>c||}||}|jr||}||z}|Sr@)rrrr)r;rr_s rrBz#EfficientNetFinalBlockLayer.forwardsR))-88  66   7 LL77M)J6MrrCrDrErFrrfloatror/rGrbrHrBrIrJs@rrrs/(/25/@C/MP/]b/mq//////"%"3EDUZ_ZfrrcleZdZdZdededededededed ed effd Zd e j d e j fdZ xZ S)EfficientNetBlocka This corresponds to the expansion and depthwise convolution phase of each block in the original implementation. Args: config ([`EfficientNetConfig`]): Model configuration class. in_dim (`int`): Number of input channels. out_dim (`int`): Number of output channels. stride (`int`): Stride size to be used in convolution layers. expand_ratio (`int`): Expand ratio to set the output dimensions for the expansion and squeeze-excite layers. kernel_size (`int`): Kernel size for the depthwise convolution layer. drop_rate (`float`): Dropout rate to be used in the final phase of each block. id_skip (`bool`): Whether to apply dropout and sum the final hidden states with the input embeddings during the final phase of each block. Set to `True` for the first block of each stage. adjust_padding (`bool`): Whether to apply padding to only right and bottom side of the input kernel before the depthwise convolution operation, set to `True` for inputs with odd input sizes. rrWr0r* expand_ratiorrrrec t||_|jdk|_||z} |jrt ||| ||_t ||jr| n|||| |_t||| |j|_ t||jr| n||||||_ dS)Nr )rrWr0r*)rrWr*rre)rrWrrrs)rrWr0r*rr) r.r/rrsrV expansionrdrirqsqueeze_exciter projection) r;rrWr0r*rrrrre expand_in_dimr<s rr/zEfficientNetBlock.__init__s ('1, - ; 7fmFDN9$(K;==V#)    =&]4;   6$(K;==V    rr_r>c|}|jdkr||}||}||}|||}|S)Nr )rrrirr)r;r_rs rrBzEfficientNetBlock.forwardGsg"   ! ! NN=99M++M:: ++M::  MBB rrrJs@rrrs4' "' '  '  '  ' ' ' ' ' ' ' ' ' ' R U%6 5<        rrc \eZdZdZdeffd Z d dejdedzd edzd e fd Z xZ S) EfficientNetEncoderz Forward propagates the embeddings through each EfficientNet block. Args: config ([`EfficientNetConfig`]): Model configuration class. rc t|_|j_fdt |j}t fd|jD}d}g}t|D]}t||j|}t||j |}|j |} |j |} |j |} t|j|D]d} | dk} | dkrdn| } | dkr|n|}||jv}|j|z|z }t!|||| | | || | }|||dz }et%j|_t%j|t|dddd _t%j|j|j|j _t8|j_dS) NcVttjj|zSr@)rmathceildepth_coefficient)repeatsr;s r round_repeatsz3EfficientNetEncoder.__init__..round_repeatsbs#ty!7'!ABBCC Crc3.K|]}|VdSr@).0nrs r z/EfficientNetEncoder.__init__..gs-LLaq))LLLLLLrrr ) rrWr0r*rrrrreirYFrZrg)r.r/rrlenrOsumnum_block_repeatsrangerrPstrides kernel_sizes expand_ratiosdepthwise_paddingdrop_connect_raterappendr ModuleListblocksr2top_convr4 hidden_dimr5r6top_bnrr8top_activation)r;rnum_base_blocks num_blockscurr_block_numrirWr0r*rrjrrerblockrr<s` @rr/zEfficientNetEncoder.__init__]sE  !'!9 D D D D Df011LLLL63KLLLLL '' $ $A"66+=a+@AAF#FF,?,BCCG^A&F -a0K!/2L==)A!)DEEFF $ $q&!ee$%EEv!/v7O!O"4~E R )!!#! +!-'##1    e$$$!#' $*mF++  &vt44     n*0EPVPj    %V%67rFTr_output_hidden_statesN return_dictr>c$|r|fnd}|jD]}||}|r||fz }||}||}||}|st d||fDSt ||S)Nc3K|]}||V dSr@r)rvs rrz.EfficientNetEncoder.forward..s"XXq!-----XXr)last_hidden_stater_)rrrrtupler)r;r_rrall_hidden_statesrs rrBzEfficientNetEncoder.forwards 1EN],,$[ 6 6E!E-00M# 6!m%55! m44  M22 ++M::  YXX]4E$FXXXXX X-++    r)FT) rCrDrErFrr/rGrbrorrBrIrJs@rrrTs581585858585858t-2#'   ( #Tk D[  (         rrcveZdZUeed<dZdZdZdgZe j de j ffd Z xZS)EfficientNetPreTrainedModelr efficientnetr=)imagermodulect|t|tjtjtjfrtj|j d|j j |j tj |j t|ddOtj |jtj|jtj |jdSdSdS)zInitialize the weightsg)meanstdN running_mean)r. _init_weightsrrLinearr2r4initnormal_weightrinitializer_ranger+zeros_getattrrones_ running_varnum_batches_tracked)r;rr<s rrz)EfficientNetPreTrainedModel._init_weightss f%%% fry")R^D E E 8 LSdk6S T T T T{& FK(((v~t44@ F/000 6-... F677777 8 8A@r)rCrDrEr__annotations__base_model_prefixmain_input_nameinput_modalities_no_split_modulesrGno_gradrModulerrIrJs@rrrs &$O!,-U]__ 8BI 8 8 8 8 8_ 8 8 8 8 8rrc veZdZdeffd Ze d dejdzdedzdedzde e zfdZ xZ S) EfficientNetModelrct|||_t||_t ||_|jdkr!tj |j d|_ nC|jdkr!tj |j d|_ ntd|j|dS)NrT) ceil_moderz2config.pooling must be one of ['mean', 'max'] got )r.r/rr#rrencoder pooling_typer AvgPool2drpooler MaxPool2d ValueErrorpooling post_initr:s rr/zEfficientNetModel.__init__s     088*622   & ( (,v'8DIIIDKK  E ) ),v'8DIIIDKKbRXR`bbcc c rNr=rrr>c ||n |jj}||n |jj}|td||}||||}|d}||}||jdd}|s||f|ddzSt|||j S)Nz You have to specify pixel_valuesrrrrr )r pooler_outputr_) rrrrrrrreshapeshaperr_) r;r=rrkwargsembedding_outputencoder_outputsr pooled_outputs rrBzEfficientNetModel.forwards%9$D $+Jj &1%c j||n |jj}||||}|r|jn|d}||}||}d} |||||j} |s|f|ddz} | | f| zn| St| ||jS)a labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the image classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Nrr r)losslogitsr_) rrrrrr loss_functionr r_) r;r=rrrroutputsrrroutputs rrBz*EfficientNetForImageClassification.forwards&1%rst"! &&&&&&!!!!!! .-----,,,,,,,,::::::  H % % , C    @@S5[@$@@@@(RY2     ")   44$$$$$$$$N$$$$$RY$$$N")BNNNNN NNNbW W W W W ")W W W t88888/888*6 6 6 6 6 36 6 6 r 0 0 0 0 0 )D0 0  0 f e d dr