S\#S\#S\\#S\#4Sjjr$"SS\RJ5r&"SS\RJ5r'"SS\RJ5r("SS\RJ5r)"SS\RJ5r*"SS\RJ5r+"S S!\RJ5r,"S"S#\RJ5r-"S$S%\RJ5r."S&S'\RJ5r/"S(S)\5r0"S*S+\RJ5r1\"S,S-\55r2\"S.S/\255r3\"S0S19"S2S3\255r4"S4S5\RJ5r5"S6S7\RJ5r6"S8S9\RJ5r7\"S:S19"S;S<\255r8/S=Qr9g)?zPyTorch MobileViT model.N)OptionalUnion)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)ACT2FN)GradientCheckpointingLayer)BaseModelOutputWithNoAttention(BaseModelOutputWithPoolingAndNoAttention$ImageClassifierOutputWithNoAttentionSemanticSegmenterOutput)PreTrainedModel) find_pruneable_heads_and_indicesprune_linear_layer)auto_docstringlogging torch_int)MobileViTConfigvaluedivisor min_valuereturnc|UcUn[U[XS- -5U-U-5nUSU-:aX1- n[U5$)z Ensure that all layers have a channel count that is divisible by `divisor`. This function is taken from the original TensorFlow repo. It can be seen here: https://github.com/tensorflow/models/blob/master/research/slim/nets/mobilenet/mobilenet.py g?)maxint)rrr new_values h/var/www/html/shao/venv/lib/python3.13/site-packages/transformers/models/mobilevit/modeling_mobilevit.pymake_divisibler",sO  Is5Q;#677BWLMI3; y>c^\rSrSrSS\S\S\S\S\S\S\S \S \S \\\4S S 4U4Sjjjr S\ RS \ R4Sjr Sr U=r$)MobileViTConvLayer;config in_channels out_channels kernel_sizestridegroupsbiasdilationuse_normalizationuse_activationrNc D>[T U]5 [US- S- 5U-n X&-S:wa[SUSUS35eX6-S:wa[SUSUS35e[R "UUUUU UUUSS 9 UlU (a[R"US S S S S 9UlOSUlU (an[U [5(a[U Ul g[UR[5(a[URUl gURUl gSUl g)NrrrzInput channels (z) are not divisible by z groups.zOutput channels (zeros) r(r)r*r+paddingr.r,r- padding_modegh㈵>g?T) num_featuresepsmomentumaffinetrack_running_stats)super__init__r ValueErrorrConv2d convolution BatchNorm2d normalization isinstancestrr activation hidden_act) selfr'r(r)r*r+r,r-r.r/r0r3 __class__s r!r;MobileViTConvLayer.__init__<s, {Q!+,x7  1 $/ }UV\U]]efg g99#%#    !#)$( "D "&D  .#.."("8F--s33"():):";"("3"3"DOr#featurescURU5nURbURU5nURbURU5nU$N)r>r@rC)rErHs r!forwardMobileViTConvLayer.forwardrsK##H-    )))(3H ?? &x0Hr#)rCr>r@)rrFrTT)__name__ __module__ __qualname____firstlineno__rrboolrrBr;torchTensorrK__static_attributes__ __classcell__rFs@r!r%r%;s"&+/4#4#4# 4#  4#  4#4#4#4# 4#dCi(4# 4#4#l r#r%c ^\rSrSrSrSS\S\S\S\S\SS 4 U4S jjjrS \RS\R4S jr S r U=r $)MobileViTInvertedResidual{zQ Inverted residual block (MobileNetv2): https://huggingface.co/papers/1801.04381 r'r(r)r+r.rNc 6>[TU]5 [[[ X!R -55S5nUS;a[ SUS35eUS:H=(a X#:HUl[XUSS9Ul [UUUSUUUS9Ul [UUUSS S 9Ul g) N)rrzInvalid stride .rr(r)r*r )r(r)r*r+r,r.Fr(r)r*r0) r:r;r"rround expand_ratior< use_residualr% expand_1x1conv_3x3 reduce_1x1)rEr'r(r)r+r.expanded_channelsrFs r!r;"MobileViTInvertedResidual.__init__s *3u[CVCV5V/W+XZ[\  vha89 9#q[K{/J, :KYZ + )*$  - )%  r#rHcUnURU5nURU5nURU5nUR(aX!-$U$rJ)rbrcrdra)rErHresiduals r!rK!MobileViTInvertedResidual.forwardsG??8,==*??8,&*&7&7x"EXEr#)rcrbrdrar) rMrNrOrP__doc__rrr;rRrSrKrTrUrVs@r!rXrX{sn jk % 47 GJ TW cf   BF FFFr#rXc ^\rSrSrS S\S\S\S\S\SS4 U4S jjjrS \RS\R4S jr S r U=r $)MobileViTMobileNetLayerr'r(r)r+ num_stagesrNc>[TU]5 [R"5Ul[ U5H4n[ UUUUS:XaUOSS9nURRU5 UnM6 g)Nrr)r(r)r+)r:r;r ModuleListlayerrangerXappend) rEr'r(r)r+roirrrFs r!r; MobileViTMobileNetLayer.__init__sc ]]_ z"A-')!"avQ E JJ  e $&K#r#rHc<URH nU"U5nM U$rJrr)rErH layer_modules r!rKMobileViTMobileNetLayer.forwards JJL#H-H'r#rx)rr rMrNrOrPrrr;rRrSrKrTrUrVs@r!rmrms`op'%'47'GJ'TW'il' ''  r#rmcr^\rSrSrS\S\SS4U4SjjrS\RS\R4Sjr S r U=r $) MobileViTSelfAttentionr' hidden_sizerNcr>[TU]5 X!R-S:wa[SUSURS35eURUl[ X!R- 5UlURUR -Ul[R"X R URS9Ul [R"X R URS9Ul [R"X R URS9Ul [R"UR5Ulg)NrzThe hidden size z4 is not a multiple of the number of attention heads r\)r-)r:r;num_attention_headsr<rattention_head_size all_head_sizerLinearqkv_biasquerykeyrDropoutattention_probs_dropout_probdropoutrEr'rrFs r!r;MobileViTSelfAttention.__init__s  33 3q 8";-0334A7  $*#=#= #&{5O5O'O#P !558P8PPYY{,>,>V__U 99[*<*<6??SYY{,>,>V__U zz&"E"EF r# hidden_statescURup#nURU5RUSURUR5R SS5nUR U5RUSURUR5R SS5nURU5RUSURUR5R SS5n[R"XVR SS55nU[R"UR5- n[RRUSS9n URU 5n [R"X5n U R!SSSS5R#5n U R%5SSUR&4-n U R"U 6n U $)Nrrdimrr )shaperviewrr transposerrrRmatmulmathsqrtr functionalsoftmaxrpermute contiguoussizer) rEr batch_size seq_length_ query_layer key_layer value_layerattention_scoresattention_probs context_layernew_context_layer_shapes r!rKMobileViTSelfAttention.forwards$1$7$7!  JJ} % T*b$":":DS"S%**,CD r#)rrrrrrrr{rVs@r!r}r}sAGGSGTG&"U\\"ell""r#r}cr^\rSrSrS\S\SS4U4SjjrS\RS\R4Sjr S r U=r $) MobileViTSelfOutputr'rrNc>[TU]5 [R"X"5Ul[R "UR 5UlgrJr:r;rrdenserhidden_dropout_probrrs r!r;MobileViTSelfOutput.__init__s4 YY{8 zz&"<"<= r#rcJURU5nURU5nU$rJrrrErs r!rKMobileViTSelfOutput.forwards$ =1  ]3 r#rr{rVs@r!rrs=>>S>T> U\\ellr#rc^\rSrSrS\S\SS4U4SjjrS\\SS4SjrS \ RS\ R4S jr S r U=r $) MobileViTAttentionir'rrNc>[TU]5 [X5Ul[ X5Ul[ 5UlgrJ)r:r;r} attentionroutputset pruned_headsrs r!r;MobileViTAttention.__init__s0 /D)&> Er#headsc6[U5S:Xag[XRRURRUR 5up[ URRU5URl[ URRU5URl[ URRU5URl [ URRUSS9URl URR[U5- URlURRURR-URl UR RU5Ulg)Nrrr)lenrrrrrrrrrrrrunion)rErindexs r! prune_headsMobileViTAttention.prune_headss u:? 7 >>55t~~7Y7Y[_[l[l  2$..2F2FN/0B0BEJ1$..2F2FN.t{{/@/@%QO .2^^-O-ORUV[R\-\*'+~~'I'IDNNLnLn'n$ --33E:r#rcJURU5nURU5nU$rJ)rr)rEr self_outputsattention_outputs r!rKMobileViTAttention.forward s%~~m4 ;;|4r#)rrr)rMrNrOrPrrr;rrrRrSrKrTrUrVs@r!rrsT""S"T" ;S;d;$ U\\ ell  r#rcv^\rSrSrS\S\S\SS4U4SjjrS\RS\R4S jr S r U=r $) MobileViTIntermediatei&r'rintermediate_sizerNc>[TU]5 [R"X#5Ul[ UR [5(a[UR Ul gUR Ul grJ) r:r;rrrrArDrBr intermediate_act_fnrEr'rrrFs r!r;MobileViTIntermediate.__init__'sR YY{> f'' - -'-f.?.?'@D $'-'8'8D $r#rcJURU5nURU5nU$rJrrrs r!rKMobileViTIntermediate.forward/s& =1 00? r#rr{rVs@r!rr&sF99S9UX9]a9U\\ellr#rc^\rSrSrS\S\S\SS4U4SjjrS\RS \RS\R4S jr S r U=r $) MobileViTOutputi5r'rrrNc>[TU]5 [R"X25Ul[R "UR 5UlgrJrrs r!r;MobileViTOutput.__init__6s5 YY0> zz&"<"<= r#r input_tensorcRURU5nURU5nX-nU$rJr)rErrs r!rKMobileViTOutput.forward;s, =1  ]3 %4 r#rr{rVs@r!rr5sT>>S>UX>]a> U\\RWR^R^r#rcv^\rSrSrS\S\S\SS4U4SjjrS\RS\R4S jr S r U=r $) MobileViTTransformerLayeriBr'rrrNc>[TU]5 [X5Ul[ XU5Ul[ XU5Ul[R"X!RS9Ul [R"X!RS9Ul g)Nr6) r:r;rrr intermediaterrr LayerNormlayer_norm_epslayernorm_beforelayernorm_afterrs r!r;"MobileViTTransformerLayer.__init__Csg +F@1&GXY%f;LM " [>S>S T!||K=R=RSr#rcURURU55nX!-nURU5nURU5nUR X15nU$rJ)rrrrr)rErr layer_outputs r!rK!MobileViTTransformerLayer.forwardKsX>>$*?*? *NO(8 ++M: ((6 {{<? r#)rrrrrr{rVs@r!rrBsKTTSTUXT]aTU\\ellr#rcv^\rSrSrS\S\S\SS4U4SjjrS\RS\R4S jr S r U=r $) MobileViTTransformeriUr'rrorNc >[TU]5 [R"5Ul[ U5H>n[ UU[X!R-5S9nURRU5 M@ g)N)rr) r:r;rrqrrrsrr mlp_ratiort)rEr'rrortransformer_layerrFs r!r;MobileViTTransformer.__init__Vsa ]]_ z"A 9'"%k4D4D&D"E!  JJ  / 0 #r#rc<URH nU"U5nM U$rJrx)rErrys r!rKMobileViTTransformer.forwardbs  JJL(7M'r#rxr{rVs@r!rrUsE 1 1S 1c 1VZ 1U\\ellr#rc^\rSrSrSrSS\S\S\S\S\S\S \S S 4U4S jjjrS \RS \ \R\ 44Sjr S\RS\ S \R4Sjr S \RS \R4SjrSrU=r$)MobileViTLayerihz; MobileViT block: https://huggingface.co/papers/2110.02178 r'r(r)r+rror.rNc >[TU]5 URUlURUlUS:Xa([ UUUUS:XaUOSUS:aUS-OSS9UlUnOSUl[UUUURS9Ul [UUUSSSS9Ul [UUUS9Ul [R"XQRS9Ul[XUSS9Ul[USU-X!RS9Ulg) Nrr)r(r)r+r.r]F)r(r)r*r/r0)rror)r:r; patch_size patch_width patch_heightrXdownsampling_layerr%conv_kernel_sizeconv_kxkconv_1x1r transformerrrr layernormconv_projectionfusion) rEr'r(r)r+rror.rFs r!r;MobileViTLayer.__init__ms !,,"-- Q;&?')!)QvA*2Q,QA 'D #'K&*D #* #$//   + #$#   0 #!  k7L7LM1 +ST ) KkWnWn  r#rHcnURURp2[X#-5nURupVpx[R R 5(a$[[R"Xs- 5U-5O#[[R"Xs- 5U-5n [R R 5(a$[[R"X- 5U-5O#[[R"X- 5U-5n Sn X:wdX:wa#[RRXU 4SSS9nSn X-n X-n X-nURXV-U -X[T U]5 Xl[R"5UlSUlS=p#URS:XaSnSnOURS:XaSnSn[UURSURSSSS9nUR RU5 [UURSURSSS S9nUR RU5 [UURSURS SURSSS 9nUR RU5 U(aUS-n[UURS URS SURSS US 9nUR RU5 U(aUS-n[UURS URS SURSS US 9n UR RU 5 g)NFr[Trr)r(r)r+rorr )r(r)r+rro)r(r)r+rror.) r:r;r'rrqrrgradient_checkpointing output_stridermneck_hidden_sizesrtr hidden_sizes) rEr'dilate_layer_4dilate_layer_5r.layer_1layer_2layer_3layer_4layer_5rFs r!r;MobileViTEncoder.__init__s  ]]_ &+#+0/   1 $!N!N  ! !R '!N) 00311!4   '") 00311!4   '" 00311!4++A.   '"  MH 00311!4++A.  '"  MH 00311!4++A.  '"r#routput_hidden_states return_dictcU(aSOSn[UR5HupVU"U5nU(dMXA4-nM U(d[SX455$[XS9$)Nc3,# UH ocMUv M g7frJr5).0vs r! +MobileViTEncoder.forward..ksX$Fq$Fs )last_hidden_stater) enumeraterrrr )rErr2r3all_hidden_statesrurys r!rKMobileViTEncoder.forward\sc #7BD(4OA(7M##$58H$H! 5 X]$FXX X- oor#)r'r&rr)FT)rMrNrOrPrr;rRrSrQrrr rKrTrUrVs@r!r!r!sgH#H#4H#Z&+ p||p#p p u44 5 ppr#r!cX\rSrSr%\\S'SrSrSrS/r S\ RSS 4S jr S r g ) MobileViTPreTrainedModelipr' mobilevit pixel_valuesTrmodulerNc([U[R[R[R45(akUR R RSURRS9 URb%URR R5 gg[U[R5(aJURR R5 UR R RS5 gg)zInitialize the weightsg)meanstdNg?)rArrr=r?weightdatanormal_r'initializer_ranger-zero_rfill_)rErCs r! _init_weights&MobileViTPreTrainedModel._init_weightsxs fryy"))R^^D E E MM   & &CT[[5R5R & S{{&   &&('  - - KK   " " $ MM   $ $S ).r#r5)rMrNrOrPr__annotations__base_model_prefixmain_input_namesupports_gradient_checkpointing_no_split_modulesrModulerMrTr5r#r!r@r@ps9 #$O&*#)* *BII *$ *r#r@c ^\rSrSrS S\S\4U4SjjjrSr\S S\ \ RS\ \S\ \S \ \ \44S jj5rS rU=r$)MobileViTModelir' expand_outputcF>[TU]U5 XlX l[ UUR UR SSSS9Ul[U5Ul UR(a+[ UUR SUR SSS9Ul UR5 g ) a% expand_output (`bool`, *optional*, defaults to `True`): Whether to expand the output of the model using a 1x1 convolution. If `True`, the model will apply an additional 1x1 convolution to expand the output channels from `config.neck_hidden_sizes[5]` to `config.neck_hidden_sizes[6]`. rr r)r(r)r*r+r%rr]N) r:r;r'rWr% num_channelsr( conv_stemr!encoder conv_1x1_exp post_init)rEr'rWrFs r!r;MobileViTModel.__init__s   *+ ++11!4  (/    2"44Q7#55a8 !D  r#cUR5Hmup#URRUn[U[5(dM5UR RHnUR RU5 M Mo g)zPrunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base class PreTrainedModel N)itemsr\rrrArrrr)rEheads_to_prune layer_indexrmobilevit_layerrs r! _prune_headsMobileViTModel._prune_headssg#1"6"6"8 K"ll00=O/>::)8)D)D)J)J%%//;;EB*K#9r#rBr2r3rcUbUOURRnUbUOURRnUc [S5eUR U5nUR UUUS9nUR (a-URUS5n[R"USS/SS9nOUSnSnU(dUbXg4OU4nXSS-$[UUURS 9$) Nz You have to specify pixel_valuesr2r3rrrF)rkeepdimr)r; pooler_outputr) r'r2use_return_dictr<r[r\rWr]rRrEr r) rErBr2r3embedding_outputencoder_outputsr; pooled_outputrs r!rKMobileViTModel.forwards%9$D $++JjJj &1%>,7,, !5#'     $ 1 1/!2D E "JJ'8r2hPUVM / 2  M;H;T'7[lZnFAB// /7/')77  r#)r'r]r[r\rW)T)NNN)rMrNrOrPrrQr;rerrrRrSrrr rKrTrUrVs@r!rVrVst>C04/3&* ' u||,' 'tn' d^ ' u>> ? ' ' r#rVz MobileViT model with an image classification head on top (a linear layer on top of the pooled features), e.g. for ImageNet. ) custom_introc^\rSrSrS\SS4U4Sjjr\S S\\RS\\ S\\RS \\ S\ \ \ 44 S jj5rS rU=r$) MobileViTForImageClassificationir'rNc~>[TU]U5 URUl[U5Ul[ R "URSS9UlURS:a.[ R"URSUR5O[ R"5Ul UR5 g)NT)inplacerr)r:r; num_labelsrVrArrclassifier_dropout_probrrr(Identity classifierr^rEr'rFs r!r;(MobileViTForImageClassification.__init__s   ++'/zz&"@"@$O JPJ[J[^_J_BIIf..r2F4E4E Fegepeper  r#rBr2labelsr3cPUbUOURRnURXUS9nU(a UROUSnUR UR U55nSnUGbURR cURS:XaSURlOoURS:aNUR[R:XdUR[R:XaSURlOSURlURR S:XaI[5n URS:Xa&U "UR5UR55nOU "Xs5nOURR S:Xa=[5n U "URSUR5URS55nO,URR S:Xa[!5n U "Xs5nU(dU4USS-n UbU4U -$U $[#UUUR$S 9$) ab labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the image classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss). If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Nrhr regressionsingle_label_classificationmulti_label_classificationrr)losslogitsr)r'rkrArjrxr problem_typerudtyperRlongrrsqueezerrrrr) rErBr2r{r3outputsrnrrloss_fctrs r!rK'MobileViTForImageClassification.forwards&1%[TU]5 [R"SS9Ul[ UUUSSSSS9Ulg)Nr) output_sizeTrelu)r(r)r*r+r/r0)r:r;rAdaptiveAvgPool2d global_poolr%r)rEr'r(r)rFs r!r;MobileViTASPPPooling.__init__*sB //A>* #%"!  r#rHcURSSnURU5nURU5n[RR XSSS9nU$)NrrFr)rrrrrr)rErH spatial_sizes r!rKMobileViTASPPPooling.forward9sQ~~bc* ##H-==*==,,Xzin,or#)rrr{rVs@r!rr)sF    S  PS  X\   r#rcr^\rSrSrSrS\SS4U4SjjrS\RS\R4Sjr S r U=r $) MobileViTASPPiAz{ ASPP module defined in DeepLab papers: https://huggingface.co/papers/1606.00915, https://huggingface.co/papers/1706.05587 r'rNcn>[TU]5 URSnURn[ UR 5S:wa [ S5e[R"5Ul [UUUSSS9nURRU5 URRUR Vs/sHn[UUUSUSS9PM sn5 [XU5nURRU5 [USU-USSS9Ul[R"UR S 9Ulgs snf) Nrr z"Expected 3 values for atrous_ratesrrr^)r(r)r*r.r0r%)p)r:r;r(aspp_out_channelsr atrous_ratesr<rrqconvsr%rtextendrprojectraspp_dropout_probr)rEr'r(r) in_projectionrate pool_layerrFs r!r;MobileViTASPP.__init__Fs- ..r2 // v"" #q (AB B]]_ * #%!   -( #// 0D# +!- !!#) 0  *&|L  *%) L 0|YZkq  zzF$<$<= ) s4D2rHc/nURHnURU"U55 M [R"USS9nUR U5nUR U5nU$r)rrtrRrrr)rErHpyramidconvpooled_featuress r!rKMobileViTASPP.forwardqsWJJD NN4> *))G+,,w/,,7r#)rrr rMrNrOrPrkrr;rRrSrKrTrUrVs@r!rrAs<)>)>4)>V r#rcr^\rSrSrSrS\SS4U4SjjrS\RS\R4Sjr S r U=r $) MobileViTDeepLabV3i|zB DeepLabv3 architecture: https://huggingface.co/papers/1706.05587 r'rNc >[TU]5 [U5Ul[R "UR 5Ul[UURURSSSSS9Ul g)NrFT)r(r)r*r/r0r-) r:r;rasppr Dropout2drvrr%rrurxrys r!r;MobileViTDeepLabV3.__init__s] !&) ||F$B$BC , 00**#  r#rcrURUS5nURU5nURU5nU$)Nr)rrrx)rErrHs r!rKMobileViTDeepLabV3.forwards699]2./<<)??8,r#)rrxrrrVs@r!rr|s;  4 U\\ellr#rzX MobileViT model with a semantic segmentation head on top, e.g. for Pascal VOC. c^\rSrSrS\SS4U4Sjjr\S S\\RS\\RS\\ S \\ S\ \ \ 44 S jj5rS rU=r$) MobileViTForSemanticSegmentationir'rNc>[TU]U5 URUl[USS9Ul[ U5UlUR5 g)NF)rW)r:r;rurVrArsegmentation_headr^rys r!r;)MobileViTForSemanticSegmentation.__init__sD   ++'eD!3F!; r#rBr{r2r3czUbUOURRnUbUOURRnUb%URRS:Xa [ S5eUR USUS9nU(a UR OUSnURU5nSnUbQ[RRXrRSSSSS 9n [URRS 9n U "X5nU(d%U(a U4USS-n O U4US S-n UbU4U -$U $[UUU(aUR SS 9$SSS 9$) a labels (`torch.LongTensor` of shape `(batch_size, height, width)`, *optional*): Ground truth semantic segmentation maps for computing the loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels > 1`, a classification loss is computed (Cross-Entropy). Examples: ```python >>> import requests >>> import torch >>> from PIL import Image >>> from transformers import AutoImageProcessor, MobileViTForSemanticSegmentation >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> image_processor = AutoImageProcessor.from_pretrained("apple/deeplabv3-mobilevit-small") >>> model = MobileViTForSemanticSegmentation.from_pretrained("apple/deeplabv3-mobilevit-small") >>> inputs = image_processor(images=image, return_tensors="pt") >>> with torch.no_grad(): ... outputs = model(**inputs) >>> # logits are of shape (batch_size, num_labels, height, width) >>> logits = outputs.logits ```Nrz/The number of labels should be greater than oneTrhrrFr) ignore_indexr)rrr attentions)r'r2rkrur<rArrrrrrrsemantic_loss_ignore_indexr) rErBr{r2r3rencoder_hidden_statesrrupsampled_logitsrrs r!rK(MobileViTForSemanticSegmentation.forwardsmH%9$D $++JjJj &1%  !}}88\\"#.Zu 9  (T[[5[5[\H,5D# WQR[0 WQR[0)-)9TGf$ Ev E&3G'//   NR   r#)rArurr)rMrNrOrPrr;rrrRrSrQrrrrKrTrUrVs@r!rrs 404)-/3&* I u||,I &I 'tn I d^ I u-- . I I r#r)rrrrVr@)r[N):rkrtypingrrrRtorch.utils.checkpointrtorch.nnrrr activationsr modeling_layersr modeling_outputsr r rrmodeling_utilsr pytorch_utilsrrutilsrrrconfiguration_mobilevitr get_loggerrMloggerrr"rTr%rXrmr}rrrrrrrr!r@rVrrrrrr__all__r5r#r!rs " " AA!9 .Q774   H % #  HSM UX ==@-F -F`bii.6RYY6r "))   > BII  bii  &299&f/fR\pryy\p~***(R -R R j E &>E  E P29908BII8v8 U '?U  U p r#