SJS9\?\@\@4S:\5S;\@S$\\RS<\@S=\R4 S>jjrC\""S?S@\>55rDSrE\""SASB9"SCSD\>55rF\""SESB9"SFSG\>55rG/SHQrHg)KN)CallableOptionalUnion)CrossEntropyLoss)ACT2FN)is_deepspeed_zero3_enabled)is_fsdp_managed_module)_prepare_4d_attention_mask#_prepare_4d_attention_mask_for_sdpa)FlashAttentionKwargs)GradientCheckpointingLayer)BaseModelOutputCausalLMOutputSequenceClassifierOutput)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack)auto_docstringis_torch_flex_attn_availablelogging) HubertConfig)make_flex_block_causal_maskc.^\rSrSrU4SjrSrSrU=r$)HubertPositionalConvEmbedding2c2>[TU]5 [R"URURUR UR S-UR S9UlSUlUR(a'[R"UR5UlGO[RRn[[RRS5(a$[RRRn[5(Ga%SSKnUR"R%URR&SS9 U"URSSS9UlSSS5 [URS5(aUURRR&R(nURRR&R*nO,URR,nURR.nUR"R1X5 UR"R1X5 OU"URSSS9Ul[3UR 5Ul[6UR8Ulg!,(df  GN,=f) N) kernel_sizepaddinggroups weight_normr modifier_rankweight)namedimparametrizations)super__init__nnConv1d hidden_sizenum_conv_pos_embeddingsnum_conv_pos_embedding_groupsconv batch_normconv_pos_batch_norm BatchNorm1dutilsr#hasattrr)r deepspeedzeroGatheredParametersr& original0 original1weight_gweight_vregister_external_parameterHubertSamePadLayerr!rfeat_extract_activation activation)selfconfigr#r7r<r= __class__s b/var/www/html/shao/venv/lib/python3.13/site-packages/transformers/models/hubert/modeling_hubert.pyr+&HubertPositionalConvEmbedding.__init__3s II      6622a777     % % nnV-?-?@DO((..Krxx00-@@ hh77CC )++ ^^66tyy7G7GWX6Y +DIIH! LDIZ499&899#yy99@@JJH#yy99@@JJH#yy11H#yy11H::4J::4J' aH )&*H*HI  !?!?@ZYs J JcURSS5nURbURU5nURU5nURU5nUR U5nURSS5nU$)Nrr) transposer2r1r!rArB hidden_statess rEforward%HubertPositionalConvEmbedding.forwardXsn%//15 ?? & OOM:M -0  ]3  6 %//15 )rAr2r1r!__name__ __module__ __qualname____firstlineno__r+rK__static_attributes__ __classcell__rDs@rErr2s#AJ  rMrc.^\rSrSrU4SjrSrSrU=r$)r?dcR>[TU]5 US-S:XaSUlgSUlg)Nrrr)r*r+num_pad_remove)rBr/rDs rEr+HubertSamePadLayer.__init__es) #:Q#>!#CarMcXURS:aUSS2SS2SUR*24nU$NrrYrIs rErKHubertSamePadLayer.forwardis6    ")!Q0F43F3F2F0F*FGMrMr]rNrUs@rEr?r?dsKrMr?c2^\rSrSrSU4SjjrSrSrU=r$)HubertNoLayerNormConvLayerocb>[TU]5 US:aURUS- OSUlURUUl[ R "URURURUURUURS9Ul [URUl g)Nrrr stridebias)r*r+conv_dim in_conv_dim out_conv_dimr,r- conv_kernel conv_stride conv_biasr1rr@rArBrClayer_idrDs rEr+#HubertNoLayerNormConvLayer.__init__ps [TU]5 US:aURUS- OSUlURUUl[ R "URURURUURUURS9Ul [ R"URSS9Ul [URUlg)NrrrcT)elementwise_affine)r*r+rfrgrhr,r-rirjrkr1 LayerNorm layer_normrr@rArls rEr+!HubertLayerNormConvLayer.__init__s [TU]5 US:aURUS- OSUlURUUl[ R "URURURUURUURS9Ul [URUl [ R"URURSS9Ulg)NrrrcT) num_groups num_channelsaffine)r*r+rfrgrhr,r-rirjrkr1rr@rA GroupNormryrls rEr+!HubertGroupNormConvLayer.__init__s [TU]5 URS:Xa?[USS9/[ UR S- 5Vs/sHn[ XS-S9PM sn-nOUURS:Xa,[ UR 5Vs/sH n[XS9PM nnO[SURS35e[R"U5Ul SUl S Ul gs snfs snf) Ngroupr)rmrlayerz`config.feat_extract_norm` is z), but has to be one of ['group', 'layer']FT)r*r+feat_extract_normrrangenum_feat_extract_layersr`rt ValueErrorr, ModuleList conv_layersgradient_checkpointing_requires_grad)rBrCirrDs rEr+HubertFeatureEncoder.__init__s   # #w .3FQGHLQRXRpRpstRtLuLLuq*6EBLuLK % % 0QVW]WuWuQvwQvA3FGQvKwK01I1I0JJst ==5&+#"Lxs CC#cNUR5H nSUlM SUlgNF) parameters requires_gradrrBparams rE_freeze_parameters'HubertFeatureEncoder._freeze_parameterss#__&E"'E '#rMcUSS2S4nUR(aUR(aSUlURH nU"U5nM U$)NT)rtrainingrr)rB input_valuesrJ conv_layers rErKHubertFeatureEncoder.forwardsK$QW-    4==*.M '**J&}5M+rM)rrr) rOrPrQrR__doc__r+rrKrSrTrUs@rErrs8#"$  rMrc.^\rSrSrU4SjrSrSrU=r$)HubertFeatureProjectioncx>[TU]5 URUlUR(a1[R"UR SUR S9Ul[R"UR SUR5Ul [R"UR5Ul g)Nr}eps)r*r+feat_proj_layer_normr,rxrflayer_norm_epsryLinearr. projectionDropoutfeat_proj_dropoutdropoutrBrCrDs rEr+ HubertFeatureProjection.__init__s} $*$?$?!  $ $ ll6??2+>FDYDYZDO))FOOB$79K9KLzz&":":; rMcUR(aURU5nURU5nURU5nU$rp)rryrrrIs rErKHubertFeatureProjection.forwards;  $ $ OOM:M 6  ]3 rM)rrryrrNrUs@rErrs<rMrmodulequerykeyvalueattention_maskscalingr head_maskc UcURS5S-n[R"XRSS55U-n UbX-n [R R U SS9n UbXRSSSS5-n [R RXURS9n [R"X5n U RSS5R5n X4$)Nr}rrr(r)pr) sizetorchmatmulrHr, functionalsoftmaxviewrr contiguous) rrrrrrrrkwargs attn_weights attn_outputs rEeager_attention_forwardrs**R.D(<<}}Q':;gEL!#4 ==((2(>L#nnQAq&AA ==((6??([L,,|3K''1-88:K  $$rMcZ^\rSrSrSrSS\S\S\S\S\S\S \\ 4U4S jjjr SS \ RS \\ RS \\ RS\\ RS\\S\ \S\\ R\\ R\\\ R44SjjrSrU=r$)HubertAttentioni z=Multi-headed attention from 'Attention Is All You Need' paper embed_dim num_headsr is_decoderre is_causalrCc>[TU]5 XlX lX0lX-UlXplUR U-UR:wa[SURSUS35eUR S-UlX@l X`l [R"XUS9Ul [R"XUS9Ul[R"XUS9Ul[R"XUS9Ulg)Nz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).r)re)r*r+rrrhead_dimrCrrrrr,rk_projv_projq_projout_proj) rBrrrrrerrCrDs rEr+HubertAttention.__init__s "" !.  MMI %$.. 8MdnnM]$YKr3 }}d* $"ii 4@ ii 4@ ii 4@  )TB rMrJkey_value_statesrlayer_head_maskoutput_attentionsrreturnc USLnURSSupU(aURSOU n XSUR4n XSUR4n URU5R"U 6R SS5n U(aUOUnUR U5R"U 6R SS5nUR U5R"U 6R SS5n[nURRS:wa[URRnU"UU UUU4UR(dSO URURUUS.UD6unnURXS5R5nUR!U5nUUS4$)z#Input shape: Batch x Time x ChannelNr}rreager)rrrr)shaperrrrHrrrrC_attn_implementationrrrrreshaperr)rBrJrrrrris_cross_attentionbsztgt_lensrc_len q_input_shapekv_input_shape query_statescurrent_states key_states value_statesattention_interfacerrs rErKHubertAttention.forward/s.T9%**3B/ /A"((+wr4==9 DMM:{{=166 FPPQRTUV -?)][[055~FPPQRTUV {{>277HRRSTVWX (? ;; + +w 6"9$++:Z:Z"[ $7      % $}}C$,,LL/% %  % ! \"))#;FFH mmK0 L$..rM) rCrrrrrrrrrrr)rFTFN)NNNF)rOrPrQrRrintfloatboolrrr+rTensorrr tuplerKrSrTrUs@rErr s G  )-CCC C  C  CC&CCD481526,1 3/||3/#5<<03/!. 3/ "%,,/ 3/ $D> 3/-.3/ u||Xell3XeELL>Q5RR S3/3/rMrc.^\rSrSrU4SjrSrSrU=r$)HubertFeedForwardiec>[TU]5 [R"UR5Ul[R "URUR5Ul [UR[5(a[URUlOURUl[R "URUR5Ul[R"UR 5Ulgrp)r*r+r,ractivation_dropoutintermediate_dropoutrr.intermediate_sizeintermediate_dense isinstance hidden_actstrrintermediate_act_fn output_densehidden_dropoutoutput_dropoutrs rEr+HubertFeedForward.__init__fs $&JJv/H/H$I!"$))F,>,>@X@X"Y f'' - -'-f.?.?'@D $'-'8'8D $IIf&>&>@R@RS jj)>)>?rMcURU5nURU5nURU5nURU5nUR U5nU$rp)rrrrrrIs rErKHubertFeedForward.forwardssX// > 00? 11-@ ))-8 ++M: rM)rrrrrrNrUs@rErres @rMrc2^\rSrSrU4SjrSSjrSrU=r$)HubertEncoderLayeri}c>[TU]5 [URURUR SUS9Ul[R"UR5Ul [R"URURS9Ul [U5Ul[R"URURS9Ulg)NFrrrrrCr)r*r+rr.num_attention_headsattention_dropout attentionr,rrrrxrryr feed_forwardfinal_layer_normrs rEr+HubertEncoderLayer.__init__~s (((00,,  zz&"7"78 ,,v'9'9v?T?TU-f5 " V-?-?VEZEZ [rMcUnURXUS9upnURU5nXA-nURU5nXRU5-nUR U5nU4nU(aXu4- nU$Nrr)rrryrrrBrJrr attn_residualr_outputss rErKHubertEncoderLayer.forwards% )- L]*8* & Q ]3 %5  6 %(9(9-(HH --m<  "   &GrM)rrrrryrrNrUs@rEr r }s \rMr c ^\rSrSrU4SjrS S\R S\\RS\ S\ S\ 4 S jjr S\ \RS4S \R4S jr S r U=r$) HubertEncoderic>[TU]5 Xl[U5Ul[ R "URURS9Ul [ R"UR5Ul [ R"[UR5Vs/sHn[!U5PM sn5UlSUlgs snfNrF)r*r+rCrpos_conv_embedr,rxr.rryrrrrrnum_hidden_layersr layersrrBrCrrDs rEr+HubertEncoder.__init__s  ;FC,,v'9'9v?T?TUzz&"7"78 mmvOgOgIh$iIhA%7%?Ih$ij &+#%j CNrJrroutput_hidden_states return_dictcU(aSOSnU(aSOSnUb4URS5RSSURS5nSX)'URUU5nUR U5n X-nUR U5nUR U5n[5=(d [U5n URHn U(aXa4-n[R"/5n UR=(a XRR:n U (aU (a U "XUS9nUSnU (aSnU(dM|UWS4-nM U(aXa4-nU(d[SXU455$[!UUUS 9$) Nr}rrrrNNc3,# UH ocMUv M g7frpr(.0vs rE (HubertEncoder.forward..m$[q$[ last_hidden_staterJ attentions) unsqueezerepeatr_update_full_maskrryrr r r!rrandrrC layerdroprrrBrJrrr%r&all_hidden_statesall_self_attentionsexpand_attention_maskposition_embeddings synced_gpusrdropout_probabilityskip_the_layer layer_outputss rErKHubertEncoder.forwards#7BD$5b4  %$2$<$[TU]5 URUlURUl[ R"UR 5Ul[ R"UR UR5Ul [ R"5Ul [ R"URUR 5Ul g)z Implements adapter modules directly with 3D tensor weight as parameters and without using ModuleList to speed up training throughput. N)r*r+adapter_attn_dim input_dimr. hidden_dimr,rxnormrlinear_1ReLUact_fnlinear_2rs rEr+HubertAttnAdapterLayer.__init__s 00 ,,LL1  $//4>>B ggi  $..$//B rMrJcURU5nURU5nURU5nURU5nU$rp)rXrYr[r\rIs rErKHubertAttnAdapterLayer.forward s@ -0  m4  M2  m4 rM)r[rWrVrYr\rX) rOrPrQrRr+r FloatTensorrKrSrTrUs@rErSrSs  CU%6%6rMrSct^\rSrSrU4SjrSS\R S\\R S\4Sjjr Sr U=r $) !HubertEncoderLayerStableLayerNormic>[TU]5 [URURUR SUS9Ul[R"UR5Ul [R"URURS9Ul [U5Ul[R"URURS9Ul[#USS5b[%U5UlgSUlg)NFr rrU)r*r+rr.r rrr,rrrrxrryrrrgetattrrS adapter_layerrs rEr+*HubertEncoderLayerStableLayerNorm.__init__s (((00,,  zz&"7"78 ,,v'9'9v?T?TU-f5 " V-?-?VEZEZ [ 6-t 4 @!7!?D !%D rMrJrrcUnURU5nURXUS9upnURU5nXA-nXRUR U55-nUR bXR U5-nU4nU(aXu4- nU$r)ryrrrrrers rErK)HubertEncoderLayerStableLayerNorm.forward+s &  6 )- L]*8* & Q ]3 %5 %(9(9$:O:OP]:^(__    )),>,>},MMM "   &GrM)rerrrrryr) rOrPrQrRr+rrrrrKrSrTrUs@rErbrbsC&,26"' ||!. rMrbc~^\rSrSrU4SjrS SjrS\\RS4S\R4Sjr Sr U=r $) HubertEncoderStableLayerNormiEc>[TU]5 Xl[U5Ul[ R "URURS9Ul [ R"UR5Ul [ R"[UR5Vs/sHn[!U5PM sn5UlSUlgs snfr)r*r+rCrrr,rxr.rryrrrrrr rbr!rr"s rEr+%HubertEncoderStableLayerNorm.__init__Fs  ;FC,,v'9'9v?T?TUzz&"7"78 mm@EfF^F^@_ `@_1 .v 6@_ `  ',# ar$NcU(aSOSnU(aSOSnUb4URS5RSSURS5nSX)'URUU5nUR U5n X-nUR U5n[ 5=(d [U5n URHn U(aXa4-n[R"/5n UR=(a XRR:n U (aU (a U "XUS9nUSnU (aSnU(dM|UWS4-nM URU5nU(aXa4-nU(d[SXU455$[!UUUS 9$) Nr(r}rrrrr)c3,# UH ocMUv M g7frpr(r+s rEr.7HubertEncoderStableLayerNorm.forward..r0r1r2)r5r6rr7rrr r r!rr8rrCr9ryrrr:s rErK$HubertEncoderStableLayerNorm.forwardQs#7BD$5b4  %$2$<$._conv_out_lengths 99\7wWZ[[ [rM)ziprCrirj)rBrrr rds rE _get_feat_extract_output_lengths6HubertPreTrainedModel._get_feat_extract_output_lengthssG  \ $'t{{'>'> @W@W#X K,]PM$YrMfeature_vector_lengthrcURURS55R[R5nUR Sn[R "XA4URURS9nSU[R"UR SURS9US- 4'URS/5RS5RS/5R5nU$)Nr}r)rKdevicer)r) rsumtorlongrzerosrKrarangeflipcumsumr)rBrroutput_lengths batch_sizes rE"_get_feature_vector_attention_mask8HubertPreTrainedModel._get_feature_vector_attention_masks>>~?Q?QRT?UVYYZ_ZdZde#))!,   /~7K7KTbTiTi uv ^%9%9!%<^EZEZ[]kno]opq',,bT299"=BBB4HMMOrMr(N)rOrPrQrRr__annotations__base_model_prefixmain_input_namesupports_gradient_checkpointing_supports_flash_attn_supports_sdpa_supports_flex_attnrrr LongTensorrrrrSr(rMrErtrtsh  $O&*#N[B eEDTDTVYDY>Z   ]b]m]m rMrtr mask_prob mask_length min_masksrc ^^^^^UunmTS:a [S5eTT:a[STSTS35e[RRS5R 5mUUUUU4SjnUb-UR 5R S5R5O[U5Vs/sHnTPM snn[R"UT4[S 9n /n U"T5n U S :XaU $UHn U"U 5n [RR[R"U TS- - 5U S S 9n[U5S :XaTS- nOUS n[R"U[R"X- [R S 9U-/5nU R#U5 M [R$"U 5n [R&"U SS2SS2S4X[T45n U R)X[T-5n [R"T5SSSS24n[R&"UX[T45R)X[T-5nU U-n U R+5TS- :a TS- XTS- :'[R,"XSS5 U $s snf) a2 Computes random mask spans for a given shape. Used to implement [SpecAugment: A Simple Data Augmentation Method for ASR](https://huggingface.co/papers/1904.08779). Note that this method is not optimized to run on TPU and should be run on CPU as part of the preprocessing during training. Args: shape: The shape for which to compute masks. This should be of a tuple of size 2 where the first element is the batch size and the second element is the length of the axis to span. mask_prob: The percentage of the whole axis (between 0 and 1) which will be masked. The number of independently generated mask spans of length `mask_length` is computed by `mask_prob*shape[1]/mask_length`. Note that due to overlaps, `mask_prob` is an upper bound and the actual percentage will be smaller. mask_length: size of the mask min_masks: minimum number of masked spans attention_mask: A (right-padded) attention mask which independently shortens the feature axis of each batch dimension. rz&`mask_length` has to be bigger than 0.zO`mask_length` has to be smaller than `sequence_length`, but got `mask_length`: z and `sequence_length`: `c>[TU-T- T-5n[UT5nUT-T:aTT-nUTS- - U:a[UTS- - S5nU$)z;Given input length, compute how many spans should be maskedrr)rmax)rnum_masked_spanepsilonrrrsequence_lengths rEcompute_num_masked_span6_compute_mask_indices..compute_num_masked_spansoi,6DwNOoy9 [ (? :-^\rSrSrS\4U4SjjrSS\RS\\RS\\R4Sjjr \ SS\\RS\\RS\\RS \\ S \\ S \\ S \\\44S jj5rSrU=r$)ridrCc>[TU]U5 Xl[U5Ul[ U5UlURS:dURS:aG[R"[R"UR5R55UlUR (a[#U5UlO['U5UlUR)5 g)Nr)r*r+rCrfeature_extractorrfeature_projectionmask_time_probmask_feature_probr, Parameterrrr.rrydo_stable_layer_normrjencoderr post_initrs rEr+HubertModel.__init__fs   !5f!="9&"A  3 &&*B*BS*H%'\\%,,v?Q?Q2R2[2[2]%^D "  & &7?DL(0DL rMrJmask_time_indicesrc[URSS5(dU$UR5upEnUb(URR UR 5X'OURR S:aUR(a[XE4URR URRUURRS9n[R"X!R[RS9nURR UR 5X'URRS:aUR(a[XF4URRURR URR"S9n[R"XqR[RS9nUSS2S4R%SUS5nSX'U$) z Masks extracted features along time axis and/or along feature axis according to [SpecAugment](https://huggingface.co/papers/1904.08779). apply_spec_augmentTNr)rrrr)rrK)rrrr})rdrCrryrrKrrrmask_time_lengthmask_time_min_masksrrQrrrmask_feature_lengthmask_feature_min_masksexpand)rBrJrrrrr.mask_feature_indicess rE_mask_hidden_statesHubertModel._mask_hidden_statesxst{{$8$?? 4A3E3E3G0 [  (/3/E/E/H/HI\I\/]M , [[ ' '! + 5-++44 KK88-++99 ! !& ->G[G[chcmcm n /3/E/E/H/HI\I\/]M , ;; ( (1 ,#8)++77 KK;;++<< $ $)<<0DMaMainisis#t #74#@#G#GO]_#` 23M /rMrrr%r&rcUbUOURRnUbUOURRnUbUOURRnUR U5nUR SS5nUbUR URSU5nURU5nURXS9nURUUUUUS9n U SnU(d U4U SS-$[UU RU RS9$)a mask_time_indices (`torch.BoolTensor` of shape `(batch_size, sequence_length)`, *optional*): Indices to mask extracted features for contrastive loss. When in training mode, model learns to predict masked extracted features in *config.proj_codevector_dim* space. Example: ```python >>> from transformers import AutoProcessor, HubertModel >>> from datasets import load_dataset >>> processor = AutoProcessor.from_pretrained("facebook/hubert-large-ls960-ft") >>> model = HubertModel.from_pretrained("facebook/hubert-large-ls960-ft") >>> def map_to_array(example): ... example["speech"] = example["audio"]["array"] ... return example >>> ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation") >>> ds = ds.map(map_to_array) >>> input_values = processor(ds["speech"][0], return_tensors="pt").input_values # Batch size 1 >>> hidden_states = model(input_values).last_hidden_state ```Nrr)rrrr%r&rr2)rCrr%use_return_dictrrHrrrrrrrJr4) rBrrrrr%r&extract_featuresrJencoder_outputss rErKHubertModel.forwards)H2C1N-TXT_T_TqTq$8$D $++JjJj &1%59 ,((,$E$5$56,!!1!12 ,\269=,0/3&*D u||,D !.D $E$5$56 D $D> D 'tn D d^D  uo% &D D rMrzn Hubert Model with a `language modeling` head on top for Connectionist Temporal Classification (CTC). ) custom_introc^\rSrSrSS\\4U4SjjjrSrSrSr Sr \ SS\\ RS \\ RS \\S \\S \\S \\ RS\\\44Sjj5rSrU=r$) HubertForCTCi target_langc>[TU]U5 [U5Ul[R "UR 5UlX lURc[SURS35e[US5(aUR(a URO URn[R "X1R5UlUR%5 g)a target_lang (`str`, *optional*): Language id of adapter weights. Adapter weights are stored in the format adapter..safetensors or adapter..bin. Only relevant when using an instance of [`HubertForCTC`] with adapters. Uses 'eng' by default. NzYou are trying to instantiate z with a configuration that does not define the vocabulary size of the language model head. Please instantiate the model as follows: `HubertForCTC.from_pretrained(..., vocab_size=vocab_size)`. or define `vocab_size` of your model's configuration. add_adapter)r*r+rrur,r final_dropoutrr vocab_sizerrDr6routput_hidden_sizer.rlm_headr)rBrCrrrDs rEr+HubertForCTC.__init__s  !&) zz&"6"67 &    $00@AHH *1)G)GFL^L^F % %djdvdv yy!35F5FG  rMcURnUb'[URSS5c[SUS35eUc.[URSS5b[R S5 gUbUR USS9 gg)a This method overwrites [`~PreTrainedModel.tie_weights`] so that adapter weights can be correctly loaded when passing `target_lang=...` to `from_pretrained(...)`. This method is **not** supposed to be called by the user and is prone to be changed in the future. NrUzCannot pass `target_lang`: z- if `config.adapter_attn_dim` is not defined.z)By default `target_lang` is set to 'eng'.T) force_load)rrdrCrloggerinfo load_adapter)rBrs rE tie_weightsHubertForCTC.tie_weightss&&  "wt{{ D 'tn D d^ D &D  un$ %D D rMrz Hubert Model with a sequence classification head on top (a linear layer over the pooled output) for tasks like SUPERB Keyword Spotting. c^\rSrSrU4SjrSrSrSr\SS\ \ RS\ \ RS\ \ S \ \ S \ \ S \ \ RS \ \\44S jj5rSrU=r$)ric">[TU]U5 [US5(aUR(a [ S5e[ U5UlURS-nUR(a2[R"[R"U5U- 5Ul [R"URUR 5Ul[R"UR UR$5UlUR)5 g)Nrz]Sequence classification does not support the use of Hubert adapters (config.add_adapter=True)r)r*r+r6rrrrur use_weighted_layer_sumr,rrrrzrr.classifier_proj_size projector num_labels classifierr)rBrC num_layersrDs rEr+(HubertForSequenceClassification.__init__s  6= ) )f.@.@o "&) --1  ( (!#ejj.Dz.Q!RD 6#5#5v7R7RS))F$?$?ARARS rMcZ[R"S[5 UR5 g)z Calling this function will disable the gradient computation for the feature encoder so that its parameters will not be updated during training. rNrrs rEr8HubertForSequenceClassification.freeze_feature_extractorrrMcLURRR5 grr rs rEr6HubertForSequenceClassification.freeze_feature_encoderr rMcTURR5H nSUlM gr rrs rEr1HubertForSequenceClassification.freeze_base_modelrrMrrrr%r&rrc0UbUOURRnURR(aSOUnURUUUUUS9nURR(aiU[n[ R "USS9n[RRURSS9n XRSSS5-RSS9nOUSnURU5nUcURSS9n OURUR SU5n U R#S5R%SSUR S5n S X)'URSS9U RSS9RSS5- n UR'U 5n SnUbF[)5nU"U RSURR*5URS55nU(dU 4U[S-nUbU4U-$U$[-UU UR.UR0S 9$) a input_values (`torch.FloatTensor` of shape `(batch_size, sequence_length)`): Float values of input raw speech waveform. Values can be obtained by loading a `.flac` or `.wav` audio file into an array of type `list[float]`, a `numpy.ndarray` or a `torch.Tensor`, *e.g.* via the torchcodec library (`pip install torchcodec`) or the soundfile library (`pip install soundfile`). To prepare the array into `input_values`, the [`AutoProcessor`] should be used for padding and conversion into a tensor of type `torch.FloatTensor`. See [`HubertProcessor.__call__`] for details. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). NTrrrr}rrrr)rCrr/rur&rstackr,rrrzrrr1rwrrr5r6r3rr2rrJr4)rBrrrr%r&rrrJ norm_weights pooled_output padding_maskexpand_padding_maskrrloss_fctr+s rErK'HubertForSequenceClassification.forwards.&1%->r1a-HHMMRSMTM#AJM}5  !)..1.5MBB=CVCVWXCY[ijL"."8"8"<"C"CAq-J]J]^_J`"a 25M. /)--!-4|7G7GA7G7N7S7STVXY7ZZM/  ')HFKKDKK,B,BCV[[QS_UDY)F)G!HHF)-)9TGf$ Ev E'!//))   rM)r3rurzr1r)rOrPrQrRr+rrrrrrrrrrrrKrSrTrUs@rErrs" &;(26,0/3&*)-B u||,B !.B $D> B 'tn B d^ B &B  u.. /B B rMr)rrrrt)NrNr\)Irtypingrrrnumpyrrtorch.nnr,r activationsrintegrations.deepspeedr integrations.fsdpr modeling_attn_mask_utilsr r modeling_flash_attention_utilsr modeling_layersrmodeling_outputsrrrmodeling_utilsrrprocessing_utilsrr5rrrconfiguration_hubertrintegrations.flex_attentionr get_loggerrOrModulerr?r`rtrrrrrrrrr rrSrbrjrtrrrndarrayrrr&rr__all__r(rMrErVs,,, %!@7gB9YYF&JJ. !!J   H %/BII/d!;*9690#299#Lbii0 $(,% II% <<% % << % U\\* % e_ %% %%