[TU]5 [US5Ul[R "SS9Ul[R"URURSSSSS9Ul [R"URURURS 9Ul [URUlg) N )rrrrpaddingr rvalidFr strider,bias)epsmomentum)super__init__rout_dimr ZeroPad2dr,Conv2dr convolution BatchNorm2dbatch_norm_epsbatch_norm_momentum batchnormr hidden_act activationselfr __class__s rr4EfficientNetEmbeddings.__init__Os $VR0 ||L9 99   1QPW^c  &:O:OZ`ZtZtu !2!23r pixel_valuesreturncURU5nURU5nURU5nURU5nU$N)r,r8r<r>)r@rCfeaturess rforwardEfficientNetEmbeddings.forwardZsA<< -##H->>(+??8,r)r>r<r8r5r,) __name__ __module__ __qualname____firstlineno____doc__rr4torchTensorrH__static_attributes__ __classcell__rAs@rr'r'Js5 41 4ELLU\\rr'c:^\rSrSrSU4SjjrSrU=r$)EfficientNetDepthwiseConv2dcc 8>X-n [T U]UU UUUUUUUS9 g)N) in_channels out_channelsr r/r,dilationgroupsr0 padding_mode)r3r4) r@rXdepth_multiplierr r/r,rZr0r\rYrAs rr4$EfficientNetDepthwiseConv2d.__init__ds:#5  #%#%  r)rr rrrTzeros)rJrKrLrMr4rQrRrSs@rrUrUcs$   rrUcz^\rSrSrSrS\S\S\S\4U4SjjrS\RS \R4S jr S r U=r $) EfficientNetExpansionLayer}zW This corresponds to the expansion phase of each block in the original implementation. rin_dimr5r/c>[TU]5 [R"UUSSSS9Ul[R "X1R S9Ul[URUl g)NrsameFrXrYr r,r0) num_featuresr1) r3r4rr7 expand_convr9r: expand_bnr r= expand_act)r@rrdr5r/rAs rr4#EfficientNetExpansionLayer.__init__sX 99   WBWBWX !2!23r hidden_statesrDclURU5nURU5nURU5nU$rF)rirjrkr@rms rrH"EfficientNetExpansionLayer.forwards4((7 }5  6 r)rkrjri)rJrKrLrMrNrrr4rO FloatTensorrPrHrQrRrSs@rrbrb}sM 41 43 4 4VY 4U%6%65<<rrbc ~^\rSrSrSrS\S\S\S\S\4 U4SjjrS \ RS \ R4S jr S r U=r$) EfficientNetDepthwiseLayerzc This corresponds to the depthwise convolution phase of each block in the original implementation. rrdr/r adjust_paddingcF>[TU]5 X0lURS:XaSOSn[XES9n[R "US9Ul[X$X6SS9Ul[R"X!RURS9Ul [URUlg) Nrr-rf)r!r+Fr.rhr1r2)r3r4r/r%rr6depthwise_conv_padrUdepthwise_convr9r:r;depthwise_normr r= depthwise_act) r@rrdr/r ruconv_padr,rAs rr4#EfficientNetDepthwiseLayer.__init__s  "kkQ.7FkA"$,,w"?9 FSX !nn%:%:VE_E_ $F$5$56rrmrDcURS:XaURU5nURU5nURU5nUR U5nU$)Nr)r/rxryrzr{ros rrH"EfficientNetDepthwiseLayer.forwardsT ;;!  33MBM++M: ++M: **=9 r)r{ryrxrzr/rJrKrLrMrNrrboolr4rOrqrPrHrQrRrSs@rrsrss_7"77 7  7  7, U%6%6 5<<  rrsc ~^\rSrSrSrS S\S\S\S\4U4SjjjrS\ RS \ R4S jr S r U=r$) EfficientNetSqueezeExciteLayerzd This corresponds to the Squeeze and Excitement phase of each block in the original implementation. rrd expand_dimexpandc>[TU]5 U(aUOUUl[S[ X!R -55Ul[R"SS9Ul [R"URUR SSS9Ul [R"UR URSSS9Ul [URUl[R "5Ulg)Nr) output_sizerf)rXrYr r,)r3r4dimrrsqueeze_expansion_ratiodim_serAdaptiveAvgPool2dsqueezer7reducerr r= act_reduceSigmoid act_expand)r@rrdrrrAs rr4'EfficientNetSqueezeExciteLayer.__init__s !':V!S*H*H!HIJ ++: ii   ii    !!2!23**,rrmrDcUnURU5nURU5nURU5nURU5nUR U5n[ R "X!5nU$rF)rrrrrrOmul)r@rminputss rrH&EfficientNetSqueezeExciteLayer.forwardsa ]3  M2  6  M2  6  &8 r)rrrrrrr)FrrSs@rrrsR'1'3'C'Y]''* U%6%6 5<<  rrc ^\rSrSrSrS\S\S\S\S\S\4 U4S jjr S \ RS \ RS \ R4S jr SrU=r$)EfficientNetFinalBlockLayerzS This corresponds to the final phase of each block in the original implementation. rrdr5r/ drop_rateid_skipc>[TU]5 US:H=(a U(+Ul[R"UUSSSS9Ul[R "X1RURS9Ul [R"US9Ul g)NrrfFrgrwp) r3r4 apply_dropoutrr7 project_convr9r:r; project_bnDropoutdropout)r@rrdr5r/rrrAs rr4$EfficientNetFinalBlockLayer.__init__sx #q[8[II   .. &;&;fF`F` zzI. r embeddingsrmrDcURU5nURU5nUR(aURU5nX!-nU$rF)rrrr)r@rrms rrH#EfficientNetFinalBlockLayer.forwardsE))-8  6    LL7M)6Mr)rrrrrJrKrLrMrNrrfloatrr4rOrqrPrHrQrRrSs@rrrso/(/25/@C/MP/]b/mq/"%"3"3EDUDUZ_ZfZfrrc^\rSrSrSrS\S\S\S\S\S\S \S \S \4U4S jjr S \ RS\ R4Sjr SrU=r$)EfficientNetBlockiaA This corresponds to the expansion and depthwise convolution phase of each block in the original implementation. Args: config ([`EfficientNetConfig`]): Model configuration class. in_dim (`int`): Number of input channels. out_dim (`int`): Number of output channels. stride (`int`): Stride size to be used in convolution layers. expand_ratio (`int`): Expand ratio to set the output dimensions for the expansion and squeeze-excite layers. kernel_size (`int`): Kernel size for the depthwise convolution layer. drop_rate (`float`): Dropout rate to be used in the final phase of each block. id_skip (`bool`): Whether to apply dropout and sum the final hidden states with the input embeddings during the final phase of each block. Set to `True` for the first block of each stage. adjust_padding (`bool`): Whether to apply padding to only right and bottom side of the input kernel before the depthwise convolution operation, set to `True` for inputs with odd input sizes. rrdr5r/ expand_ratior rrruc f>[T U]5 XPlURS:gUlX%-n UR(a[ XXS9Ul[ UUR(aU OUUUU S9Ul[XXRS9Ul [UUR(aU OUUUUUS9Ul g)Nr)rrdr5r/)rrdr/r ru)rrdrr)rrdr5r/rr) r3r4rrrb expansionrsryrsqueeze_exciter projection) r@rrdr5r/rr rrru expand_in_dimrAs rr4EfficientNetBlock.__init__!s (''1, - ;;7mDN9$(KK=V#)  =];; 6$(KK=V  rrmrDcUnURS:waURU5nURU5nURU5nUR X!5nU$)Nr)rrryrr)r@rmrs rrHEfficientNetBlock.forwardJsY"    ! NN=9M++M: ++M:  B r)ryrrrrrrrSs@rrrs4' "' '  '  '  ' ' ' ' ' R U%6%6 5<<  rrc v^\rSrSrSrS\4U4SjjrS S\RS\ \ S\ \ S\ 4S jjr S r U=r$) EfficientNetEncoderiWz Forward propagates the embeddings through each EfficientNet block. Args: config ([`EfficientNetConfig`]): Model configuration class. rc>^^[TT]5 UTlURTlU4Sjm[ UR 5n[ U4SjUR55nSn/n[U5Hn[XR U5n[XRU5nURUn URUn URUn [T"URU55Hcn U S:Hn U S:aSOU n U S:aUOUnXAR;nURU-U- n[!UUUU U U UU US9 nUR#U5 US- nMe M [$R&"U5Tl[$R*"W[US5SSSS 9Tl[$R."UR0UR2UR4S 9TlR:Tlg) Nc\>[[R"TRU-55$rF)rmathceildepth_coefficient)repeatsr@s r round_repeats3EfficientNetEncoder.__init__..round_repeatses"tyy!7!7'!ABC Crc32># UH nT"U5v M g7frFr_).0nrs r /EfficientNetEncoder.__init__..jsL3Kaq))3Ksrr) rrdr5r/r rrrruirfFrgrw)r3r4rrlenrXsumnum_block_repeatsrangerrYstrides kernel_sizes expand_ratiosdepthwise_paddingdrop_connect_raterappendr ModuleListblocksr7top_convr9 hidden_dimr:r;top_bnr r=top_activation)r@rnum_base_blocks num_blockscurr_block_numrirdr5r/r rjrrurblockrrAs` @rr4EfficientNetEncoder.__init__`s  !'!9!9 Df001L63K3KLL 'A"6+=+=a+@AF#F,?,?,BCG^^A&F --a0K!//2L=)A)A!)DEFq&!e$%Ev!/7O7O!O"44~E R )!!#! +!-'##1  e$!#'G(8mmF+  &vt4   nn**0E0EPVPjPj  %V%6%67rrmoutput_hidden_states return_dictrDcU(aU4OSnURHnU"U5nU(dMXA4- nM URU5nURU5nURU5nU(d[ SX455$[ UUS9$)Nc3,# UH ocMUv M g7frFr_)rvs rr.EfficientNetEncoder.forward..sX$Fq$Fs )last_hidden_staterm)rrrrtupler )r@rmrrall_hidden_statesrs rrHEfficientNetEncoder.forwards 1E],$[[E!-0M##!%55!!  m4  M2 ++M: X]$FXX X-++  r)rrrrrr)FT)rJrKrLrMrNrr4rOrqrrr rHrQrRrSs@rrrWs\58158t05&*  (( 'tn d^  (   rrcN\rSrSr%\\S'SrSr/rS\ R4Sjr Sr g) EfficientNetPreTrainedModelir efficientnetrCmodulecV[U[R[R[R45(akUR R RSURRS9 URb%URR R5 ggg)zInitialize the weightsg)meanstdN) r#rLinearr7r9weightdatanormal_rinitializer_ranger0zero_)r@rs r _init_weights)EfficientNetPreTrainedModel._init_weightssv fryy"))R^^D E E MM   & &CT[[5R5R & S{{&   &&(' Frr_N) rJrKrLrMr__annotations__base_model_prefixmain_input_name_no_split_modulesrModulerrQr_rrrrs) &$O)BII)rrc ^\rSrSrS\4U4Sjjr\S S\\RS\\ S\\ S\ \ \ 44Sjj5rS rU=r$) EfficientNetModelirc>[TU]U5 Xl[U5Ul[ U5UlURS:Xa%[R"URSS9Ul OMURS:Xa%[R"URSS9Ul O[SUR35eUR5 g)NrT) ceil_moderz2config.pooling must be one of ['mean', 'max'] got )r3r4rr'rrencoder pooling_typer AvgPool2drpooler MaxPool2d ValueErrorpooling post_initr?s rr4EfficientNetModel.__init__s   08*62    & (,,v'8'8DIDK  E ),,v'8'8DIDKQRXR`R`Qabc c rrCrrrDcUbUOURRnUbUOURRnUc [S5eUR U5nUR UUUS9nUSnUR U5nURURSS5nU(d Xg4USS-$[UUURS9$)Nz You have to specify pixel_valuesrrrrr)r pooler_outputrm) rruse_return_dictrrrrreshapeshaper rm)r@rCrrembedding_outputencoder_outputsr pooled_outputs rrHEfficientNetModel.forwards%9$D $++JjJj &1%> ? " " rrz EfficientNet Model with an image classification head on top (a linear layer on top of the pooled features), e.g. for ImageNet. ) custom_introc^\rSrSrU4Sjr\S S\\RS\\RS\\ S\\ S\ \ \ 44 Sjj5rS rU=r$) "EfficientNetForImageClassificationic>[TU]U5 URUlXl[ U5Ul[ R"URS9Ul URS:a+[ R"URUR5O[ R"5Ul UR5 g)Nrr)r3r4 num_labelsrrrrr dropout_raterrrIdentity classifierrr?s rr4+EfficientNetForImageClassification.__init__s   ++ -f5zzF$7$78 KO??]^K^"))F$5$5tGdfdododq rrClabelsrrrDcTUbUOURRnURXUS9nU(a UROUSnUR U5nUR U5nSnUGbURR cURS:XaSURlOoURS:aNUR[R:XdUR[R:XaSURlOSURlURR S:XaI[5n URS:Xa&U "UR5UR55nOU "Xr5nOURR S:Xa=[5n U "URSUR5URS55nO,URR S:Xa[!5n U "Xr5nU(dU4USS-n UbU4U -$U $[#UUUR$S 9$) ab labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the image classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Nrr regressionsingle_label_classificationmulti_label_classificationr)losslogitsrm)rr rrrr problem_typerdtyperOlongrrrrviewrr rm) r@rCrrroutputsrr rloss_fctoutputs rrH*EfficientNetForImageClassification.forwards&1%r5so" " AA! .,:   H % , C @U3:.@@(RYY2 ")) 44$$N$RYY$N"))BN NbW "))W t )/ ) ) 5 35 5 p B )DB  B J er