S \\R~S!\4S"jr@S#\AS!\4S$jrB"S%S&\5rC"S'S(\C5rD"S)S*\R5rF"S+S,\R5rG\$"S-S.\.55rHS/\\RS0\R~S1\\AS!\R~4S2jrJ"S3S4\H5rK"S5S6\K5rL\$"S7S8\H55rM\$"S9S:\H55rN"S;S<\H\5rO\$"S=S>\H55rP\$"S?S@\H55rQ/SAQrRg)B)AnyCallableOptionalUnionN)OutputRecordercheck_model_inputs)Cache DynamicCacheEncoderDecoderCache)PretrainedConfig)GenerationMixin)FlashAttentionKwargs)GradientCheckpointingLayer)BaseModelOutput)BaseModelOutputWithPastAndCrossAttentionsSeq2SeqLMOutputSeq2SeqModelOutputSequenceClassifierOutputTokenClassifierOutput)ALL_ATTENTION_FUNCTIONS)Unpack)TransformersKwargsauto_docstringcan_return_tupleis_torch_flex_attn_availableis_torchdynamo_compilinglogging) Gemma2Config)Gemma2Attention Gemma2MLPGemma2PreTrainedModel Gemma2RMSNormGemma2RotaryEmbeddingcreate_causal_mask!create_sliding_window_causal_maskeager_attention_forwardz google/t5gemma-2b-2b-prefixlm-itc\rSrSrSrg)T5GemmaModuleConfigEN__name__ __module__ __qualname____firstlineno____static_attributes__r,c/var/www/html/shao/venv/lib/python3.13/site-packages/transformers/models/t5gemma/modular_t5gemma.pyr*r*Er3r*cp^\rSrSrSrSrS/r0SS_SS_SS_S S _S S_S S_S S _SS_SS_SS_SS _SS_SS_SS_SS _SS_SS_SS 0ErS/S/4SS/S/4S/S/4S/S/4SS/S/4S/S/4S.rS*S\ \ \ \ \ \ 44S\ \ \ \ \ \ 44S \S!\S"\S#\S$\S%\4U4S&jjjrU4S'jrS+S(jrS)rU=r$), T5GemmaConfigIaP This is the configuration class to store the configuration of a [`T5GemmaModel`]. It is used to instantiate an T5Gemma model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to a hypothetical balanced Gemma2 encoder-decoder model. e.g. [google/t5gemma-2b-2b-prefixlm-it](https://huggingface.co/google/t5gemma-2b-2b-prefixlm-it) ```python >>> from transformers import T5GemmaConfig, T5GemmaModel >>> t5gemma_config = T5GemmaConfig.from_pretrained("google/t5gemma-2b-2b-prefixlm-it") >>> model = T5GemmaModel(t5gemma_config) ``` Configuration objects inherit from [PretrainedConfig] and can be used to control the model outputs. Read the documentation from [PretrainedConfig] for more information. Args: encoder (`Union[T5GemmaModuleConfig, dict]`, optional, *optional*): Configuration for the encoder. decoder (`Union[T5GemmaModuleConfig, dict]`, optional, *optional*): Configuration for the decoder. is_encoder_decoder (bool, optional, *optional*, defaults to `True`): Whether the model is used as an encoder/decoder or not. dropout_rate (`float`, *optional*, defaults to 0.0): The ratio for all dropout layers (following T5). classifier_dropout_rate (`float`, *optional*, defaults to 0.0): The dropout ratio for classifier (following T5). attention_dropout (`float`, *optional*, defaults to 0.0): The dropout ratio for attention. tie_word_embeddings (`bool`, *optional*, defaults to `True`): Whether tie input and output embeddings. vocab_size (`int`, *optional*, defaults to 256000): Vocabulary size of the T5Gemma model (the same as Gemma 2). kwargs (additional keyword arguments, optional, *optional*): Will be passed to the PretrainedConfig base class. t5gemmapast_key_valuesz!encoder.layers.*.self_attn.q_projcolwisez!encoder.layers.*.self_attn.k_projz!encoder.layers.*.self_attn.v_projz!encoder.layers.*.self_attn.o_projrowwisezencoder.layers.*.mlp.gate_projzencoder.layers.*.mlp.up_projzencoder.layers.*.mlp.down_projz!decoder.layers.*.self_attn.q_projz!decoder.layers.*.self_attn.k_projz!decoder.layers.*.self_attn.v_projz!decoder.layers.*.self_attn.o_projz"decoder.layers.*.cross_attn.q_projz"decoder.layers.*.cross_attn.k_projz"decoder.layers.*.cross_attn.v_projz"decoder.layers.*.cross_attn.o_projzdecoder.layers.*.mlp.gate_projzdecoder.layers.*.mlp.up_projzdecoder.layers.*.mlp.down_proj input_ids inputs_embeds hidden_statesattention_mask)zencoder.embed_tokenszencoder.layersz encoder.normzdecoder.embed_tokenszdecoder.layersz decoder.normencoderdecoderis_encoder_decoder dropout_rateclassifier_dropout_rateattention_dropouttie_word_embeddings vocab_sizec  p>[U[5(a [S0UD6nO6Uc [5nO([U[5(d[U5S35e[U[5(a [S0UD6nO.UcUnO([U[5(d[U5S35e[S0UR 5D6n[S0UR 5D6nSUlXAlXalXlSUlSUl XBlXblURUl X l SHn X;dM [X*5X'M [T U]<"S0U D6 X0lU R#SUR5Ul U R#SUR$5UlX@lX`lXPlXplXlg)Nz is not supported.FT) bos_token_id pad_token_id eos_token_id use_cacheinitializer_ranger,) isinstancedictr*typeto_dict is_decoderrDrFrArM hidden_sizecross_attention_hidden_sizerBgetattrsuper__init__rCgetrNrErGrH) selfrArBrCrDrErFrGrHkwargsspecial_token_key __class__s r4rXT5GemmaConfig.__init__s gt $ $)4G4G _)+Gg':;; aW N`=a a; gt $ $)4G4G _Gg':;; aW N`=a a;%:(9:%:(9:"+$5! ! +$5!.5.A.A+ !Q  .,3G,O)"R "6""4K1B1BC!',?AZAZ![(!2'>$#6 %r3c>/SQnX;a,[URX5 [URX5 [TU]X5 g)N)output_hidden_statesoutput_attentions_attn_implementationrDrFrH)setattrrArBrW __setattr__)rZkeyvalueshared_attr_with_submodulesr]s r4rdT5GemmaConfig.__setattr__s<' #  - DLL# - DLL# - C'r3cAU$Nr,)rZrBs r4get_text_configT5GemmaConfig.get_text_configs  r3) rFrErBrDrArNrCrGrMrH)NNTrmrmTiF)r.r/r0r1__doc__ model_typekeys_to_ignore_at_inferencebase_model_tp_planbase_model_pp_planrrr*rPrboolfloatintrXrdrkr2 __classcell__r]s@r4r7r7Is BJ#4"5+Y ,Y ,Y  ,Y  ))  '  )) ,Y ,Y ,Y ,Y -i -i -i!" -i#$ ))%& ' '( )))0#.0A B+-=>@QR)*_,=>"-0A B+-=>@QR)*_,=> IMHL#'!),#&$( 8%% 3T#s(^ CDE8%% 3T#s(^ CDE8%! 8%  8% "' 8%!8%"8%8%8%t (r3r7c\rSrSrSrg)T5GemmaRMSNormr,Nr-r,r3r4rzrzr5r3rzc.^\rSrSrU4SjrSrSrU=r$) T5GemmaMLPcn>[TU]U5 [R"UR5Ulgrj)rWrXnnDropoutrDdropoutrZconfigr]s r4rXT5GemmaMLP.__init__s&  zz&"5"56 r3cURURU55URU5-nURU5nUR U5nU$rj)act_fn gate_projup_projr down_proj)rZxr?rs r4forwardT5GemmaMLP.forwardsH DNN1$56aH  ]3 NN=1 r3)r)r.r/r0r1rXrr2rwrxs@r4r}r}s7r3r}c,^\rSrSrSU4SjjrSrU=r$)T5GemmaRotaryEmbeddingc$>[TU]X5 grj)rWrX)rZrdevicer]s r4rXT5GemmaRotaryEmbedding.__init__s (r3r,rj)r.r/r0r1rXr2rwrxs@r4rrs ))r3rc4^\rSrSrS\S\4U4SjjrSrU=r$)T5GemmaSelfAttentionr layer_idxcF>[TU]X5 URUlgrj)rWrXrS is_causalrZrrr]s r4rXT5GemmaSelfAttention.__init__s +**r3)r) r.r/r0r1r*rvrXr2rwrxs@r4rrs+2+s++r3rc ^\rSrSrS\S\4U4SjjrS S\RS\ \RS\ \RS\ \ S \ \ S \ \R\ \R\ \ \R44 S jjrS rU=r$)T5GemmaCrossAttentionrrc~>[TU]X5 U?SUlURc [ S5e[ R"URURUR-URS9Ul [ R"URURUR-URS9Ul g)NFzBCross-attention needs cross_attention_hidden_size to be specified.bias) rWrXsliding_windowrrU ValueErrorrLinearnum_key_value_headshead_dimattention_biask_projv_projrs r4rXT5GemmaCrossAttention.__init__s +    - - 5ab bii  . .0J0JT]]0Zagavav  ii  . .0J0JT]]0Zagavav  r3r?r@encoder_hidden_statespast_key_valuer[returnc Uc [S5eURSSn/UQSPURP7nURU5R U5R SS5nUb1UR RUR5n URn UbW (dURSSn /U QSPURP7n URU5R U 5R SS5n URU5R U 5R SS5nUb7W RXUR5upSUR UR'OFW RURRn U RURRn[ nUR"R$S:wa[&UR"R$nU"UUU UU4UR((a UR*OSUR,SUR.S.UD6unnUR0"/UQSP76R35nUR5U5nUU4$) Nz5Encoder hidden state is required for cross attention.rTeagerrm)rscalingrsoftcap)rshaperq_projview transpose is_updatedrYrcross_attention_cacherrupdatelayerskeysvaluesr(rrbrtrainingrFrattn_logit_softcappingreshape contiguouso_proj)rZr?r@rrr[ input_shape hidden_shape query_statesrcurr_past_key_valueencoder_input_shapeencoder_hidden_shape key_states value_statesattention_interface attn_output attn_weightss r4rT5GemmaCrossAttention.forward sC ! (TU U#))#2. 88b8$--8 {{=166|DNNqRST  %'2266t~~FJ"0"F"F   !"7"="=cr"B #L%8#L"#Ldmm#L %:;@@AUV``abdefJ;;'<=BBCWXbbcdfghL)+>+E+Ej`d`n`n+o( <@))$..9,33DNNCHHJ.55dnnELLL(? ;; + +w 6"9$++:Z:Z"[ $7      % /3mmD**LL// %  % ! \"));;;;FFH kk+. L((r3)rrrrj)r.r/r0r1r*rvrXtorchTensorrr rrtuplerr2rwrxs@r4rrs  2  s  (+/ 3)||3)!.3) ( 5 3) ! 3) -. 3) u||Xell3XeELL>Q5RR S3)3)r3rr@rc T^S[S[S[S[S[4 U4SjjnU$)z, This creates bidirectional attention mask. batch_idxhead_idxq_idxkv_idxrc>Tc#[R"S[RS9$TX4R[R5$)Nr,dtype)ronesrtto)rrrrr@s r4 inner_mask/bidirectional_mask_function..inner_maskDs;  !::b 3 3i/033EJJ??r3rvrt)r@rs` r4bidirectional_mask_functionr?s9 @c@S@@c@d@ r3rc T^S[S[S[S[S[4 U4SjjnU$)z@ This creates bidirectional attention mask with sliding window. rrrrrc$>UT- U:X2T-:-$rjr,)rrrrrs r4r>sliding_window_bidirectional_mask_function..inner_maskQs &/F^=S4STTr3r)rrs` r4*sliding_window_bidirectional_mask_functionrLs9 UcUSUUcUdU r3c ^\rSrSrSrS\4U4SjjrS S\RS\ \R\R4S\ \RS\ \RS \ \R44 S jjr S rU=r$) T5GemmaEncoderLayeriWzEncoder sub-layer.rc$>[TU]5 URUlXlX lUR UUl[UUS9Ul[URURS9Ul [URURS9Ul [U5Ul[URURS9Ul[URURS9Ul["R$"UR&5UlgN)rreps)rWrXrTrr layer_typesattention_typer self_attnrz rms_norm_epspre_self_attn_layernormpost_self_attn_layernormr}mlppre_feedforward_layernormpost_feedforward_layernormrrrDrrs r4rXT5GemmaEncoderLayer.__init__Zs !-- "$00;- (6f6H6HfNaNa'b$(6v7I7IvObOb(c%f%)78J8JPVPcPc)d&*89K9KQWQdQd*e'zz&"5"56 r3r?position_embeddingsr@ position_idsrc 8UnURU5nUR"SUUUUSS.UD6upURU5nX`RU5-nUnUR U5nUR U5nUR U5nX`RU5-nU$)N)r?rr@rrr,)rrrrrrr)rZr?rr@rr[residual_s r4rT5GemmaEncoderLayer.forwardns!44]C >> ' 3)%     55mD  << #>>  66}E / 77 F  << #>> r3) rrrrTrrrrrrr)NN)r.r/r0r1rorvrXrrrr LongTensor FloatTensorrr2rwrxs@r4rrWs7#702637 ||#5<<#=>!.  u//0  u  ! "r3rc`^\rSrSrSrS\4U4SjjrSS\RS\ \R\R4S\ \RS\ \RS \ \ S \ \ S \ \RS \ \RS \ \RS\R4SjjrSrU=r$)T5GemmaDecoderLayeriz2Decoder sub-layer: an extra cross-attention layer.rc>[TU]X5 [XS9Ul[ UR UR S9Ul[ UR UR S9Ulgr) rWrXr cross_attnrzrTrpre_cross_attn_layernormpost_cross_attn_layernormrs r4rXT5GemmaDecoderLayer.__init__sS +/vS(6v7I7IvObOb(c%)78J8JPVPcPc)d&r3r?rr@rrrMcache_positionrencoder_attention_maskrc Un URU5nUR"SUUUUUb UROSUUS.U D6upURU5nXR U5-nUn UR U5nUR "SUUU UUS.U D6upURU5nXR U5-nUn URU5nURU5nURU5nXR U5-nU$)N)r?rr@rrrMr)r?rr@rrMr,) rrself_attention_cacherrrrrrrr) rZr?rr@rrrMrrrr[rrs r4rT5GemmaDecoderLayer.forwards0!44]C >>  ' 3)%BPB\>>>bf)     55mD  << #>>  55mD ?? '"71)     66}E  << #>>  66}E / 77 F  << #>> r3)rrr)NNNFNNN)r.r/r0r1rorvrXrrrrrr rtrrr2rwrxs@r4rrs<e#e26378<$)598<9=.||.#5<<#=>.!. . u//0 . !!45 .D>.!!1!12. ( 5.!) 6.   ..r3rcz^\rSrSrSrS S\S\S\4U4SjjjrS\RS\R4S jr S r U=r $) T5GemmaClassificationHeadiz-Head for sentence-level classification tasks.rT num_labelsrEc>[TU]5 [R"US9Ul[R "X5Ulg)N)p)rWrXrrrrout_proj)rZrTr rEr]s r4rX"T5GemmaClassificationHead.__init__s/ zz$;<  +: r3r?rcJURU5nURU5nU$rjrr )rZr?s r4r!T5GemmaClassificationHead.forwards$ ]3  m4 r3r)rm) r.r/r0r1rorvrurXrrrr2rwrxs@r4r r sF7;C;S;SX;; U\\ellr3r cz^\rSrSrSrS S\S\S\4U4SjjjrS\RS\R4S jr S r U=r $) T5GemmaLMHeadiz.Head for language modeling (generation) tasks.rTrHrcV>[TU]5 [R"XUS9Ulg)Nr)rWrXrrr )rZrTrHrr]s r4rXT5GemmaLMHead.__init__s   +E r3r?rc(URU5nU$rjr )rZr?logitss r4rT5GemmaLMHead.forwards}- r3rrn) r.r/r0r1rorvrtrXrrrr2rwrxs@r4rrsJ8FCFSFFFU\\ellr3rc:\rSrSr%\\S'SrSrS/rSr Sr Sr g ) T5GemmaPreTrainedModelirmodelT T5GemmaBlockc[R"U5 URRn[ U[ 5(aUR RRSS-nUR RRRSX#-S9 [UR S5(aGUR Rb/UR RRR5 ggg[ U[5(asURR(dWUR RRSS-nUR RRRSX#-S9 ggg)Nrgrm)meanstdr)r# _init_weightsrrNrOr r weightrdatanormal_hasattrrzero_rrG)rZmoduler scales r4r!$T5GemmaPreTrainedModel._init_weightss++F3kk++ f7 8 8OO**003t;E OO " " ' ' / /Sck / Jv//FOO4H4H4T$$))//15U/  . .;;22..44Q74?&&++33#+3N3/r3cbURRRnURRRnUc [ S5eUR UR 5nUSSS24R5USSS24'X$S'Uc [ S5eURUS:HU5 U$) z Shifts input_ids to the right, prepends the decoder_start_token_id, and handles pad_token_id replacement for labels that were -100. This is a common preparation step for decoder inputs in sequence-to-sequence models. Nz:self.model.config.decoder.bos_token_id has to be defined. .rr).rz9self.model.config.decoder.pad_token_id has to be defined.i) rrBrJrKr new_zerosrclone masked_fill_)rZr=decoder_start_token_idrKshifted_input_idss r4 _shift_right#T5GemmaPreTrainedModel._shift_rights "&!4!4!A!A{{**77 ! )YZ Z&// @%.sCRCx%8%>%>%@#qr'"$:&!  XY Y &&'8D'@,O  r3r,N) r.r/r0r1r7__annotations__base_model_prefixsupports_gradient_checkpointing_no_split_modulesr!r0r2r,r3r4rrs( &*#'( O!r3r token_idsr?rKcUb<Uc [S5eX:gRUR[R5nU$[R "UR SUR S4UR[RS9nU$)z%Construct the default attention mask.z3`pad_token_id` is required for padding information.rrrr)rrrrlongrr)r6r?rKr@s r4make_default_2d_attention_maskr:s   RS S#377 8L8LejjY    #]%8%8%; <]EYEYafakak  r3c^\rSrSr\\S.rU4Sjr\S S\ \ RS\ \ RS\ \ RS\ \ RS\\S \4 S jj5rS rU=r$) T5GemmaEncoderi ) attentionsr?c P>[TU]U5 URUlURUl[ R "URURUR5Ul[URURS9Ul [US9Ul SUl[ R"[!UR"5Vs/sHn[%X5PM sn5Ul[ R("UR*5UlUR/5 gs snf)Nr)rF)rWrXrK padding_idxrHr EmbeddingrT embed_tokensrzrnormr rotary_embgradient_checkpointing ModuleListrangenum_hidden_layersrrrrDr post_initrs r4rXT5GemmaEncoder.__init__&s  !.. ++LL):):Fr[rc USLUSL- (a [S5eUcURU5n[R"SURSUR S9nUcUR S5nUc [XURR5n[U=n[5(dbURUUUSUS.n[S 0UDS[U50D6[S 0UD[URR 5[U5S.D6S.nUn UR#X5n [R$"URR&S -U R(S 9n X-n UR+U 5n UR,SURR.Hn U "U U X|R0U40UD6n M UR3U 5n UR+U 5n [5U S 9$) N:You must specify exactly one of input_ids or inputs_embedsrrrr input_embedsr@rr:ror_mask_function)rOand_mask_functionfull_attentionsliding_attention?r)last_hidden_stater,)rrArarangerr unsqueezer:rrKrOrPr&rr'rrrCtensorrTrrrrGrrBr) rZr=r@rr>r[rself_attn_mask_mapping mask_kwargsr?r normalizer layer_modules r4rT5GemmaEncoder.forward8s  -t"; <YZ Z   --i8Ma)<)mFYFYZ %2  ]3  KK(G$++*G*GHL(#&'B'BC   MI -0  ]3 +  r3)rrArDrrBr?rCrHNNNN)r.r/r0r1rr_can_record_outputsrXrrrrrrrrrrr2rwrxs@r4r<r< s*, $15153759 > E,,-> !.> u//0 >   1 12 > +, >  > > r3r<cf^\rSrSr\"\SS9\"\SS9\S.rU4Sjr \ SS\ \ RS\ \ RS\ \ RS \ \S \ \ R S \ \S \ \ RS \ \ RS\ \ RS\\S\4Sjj5rSrU=r$)T5GemmaDecoderizr)index)r=cross_attentionsr?c >[TU]U5 [R"[ UR 5Vs/sHn[ X5PM sn5UlUR5 gs snfrj) rWrXrrErFrGrrrHrs r4rXT5GemmaDecoder.__init__sW  mmEJ6KcKcEd eEd  3Ed e    fsA)r=r@rr:r>rMrrrr[rc USLUSL- (a [S5eUc [S5eUcURU5nUR(d$U(aUc[[ 5[ 5S9nUcDUbUR 5OSn [ R"XURS-URS9nUcURS5nUc#Uc [XURR5n[U=n [5(d9URUUUUb UR OSUS.n [#S0U D6[%S0U D6S.n [U =n[5(d-URUU USSS.n S [#S0U DS ['U 50D60nUnUR)X5n[ R*"URR,S -UR.S 9nUU-nUR1U5nUR2SURR4H$nU"UUU UR6UUUUUUS 4 0U D6nM& UR9U5nUR1U5n[;UUS 9$)NrKz0`encoder_hidden_states` must be given in decoder)rrrrrLrMrQrRrOrTr)rUr:r,)rrArr r get_seq_lengthrrVrrrWr:rrKrOrPrr&r'rrCrXrTrrrrGrrBr)rZr=r@rr:r>rMrrrr[past_seen_tokensrYrZcross_attn_mask_mappingr?rr[r\s r4rT5GemmaDecoder.forwards  -t"; <YZ Z (OP P   --i8M}}/F1%1^&2nO  !CRC^==?de "\\ ]5H5H5K"KTaThThN  )33A6L  !o&=;IVZVaVaVnVnoNNB0DII++ -"0"0KZKf?#G#Glp , K#5"C{"C%F%U%U& " 5KK1TRR++ 5"8"0#' $ K!"4#!#%@AW%X#' #& "oomJ\\$++"9"93">mFYFYZ % 2  ]3  KK(G$++*G*GHL(#&|'B'BC%'(89  MI -0  ]3 8++  r3)r) NNNNNNNNN)r.r/r0r1rrrrr_rXrrrrrr rrtrrrrr2rwrxs@r4rarazs*$%9C*+@J, 1515379=59$(598<9=] E,,-] !.] u//0 ] ""56 ]   1 12 ] D>] !!1!12]  ( 5] !) 6] +,]  3] ] r3rac^\rSrSrS\4U4SjjrSrSrSrSr \ \ SS\ \ RS \ \ RS \ \ RS \ \ RS \ \ R S \ \ RS\ \S\ \S\ \ R&S\ \ R&S\ \S\ \ RS\\S\4Sjj55rSrU=r$) T5GemmaModelirc>[TU]U5 UR(d [S5e[ UR 5Ul[ UR5UlUR5 g)NzVT5GemmaModel only support encoder-decoder modeling. Use `T5GemmaEncoderModel` instead.) rWrXrCrr<rArarBrHrs r4rXT5GemmaModel.__init__sO  ((uv v%fnn5 %fnn5  r3cUR$rjrArZs r4 get_encoderT5GemmaModel.get_encoder ||r3cUR$rj)rBrqs r4 get_decoderT5GemmaModel.get_decoderrtr3c6URR5$rjrAget_input_embeddingsrqs r4rz!T5GemmaModel.get_input_embeddings||0022r3c8URRU5$rjrAset_input_embeddingsrZnew_embeddingss r4r!T5GemmaModel.set_input_embeddings||00@@r3r=r@rdecoder_input_idsdecoder_attention_maskdecoder_position_idsencoder_outputsr:r>decoder_inputs_embedsrMrr[rc UcUR"SUUUU S.U D6nURnUR"SUUUU UUUU U S. U D6n[URURU R SS5(a UR O UR4URURURUR URS9$)a8 decoder_position_ids (`torch.LongTensor` of shape `(batch_size, decoder_sequence_length)`, *optional*): Indices of positions of each decoder input sequence tokens in the position embeddings. Selected in the range `[0, config.decoder.n_positions - 1]`. [What are position IDs?](../glossary#position-ids) r=r@rr>) r=r@rr>r:rrrMrr`F)rUr:decoder_hidden_statesdecoder_attentionsrcencoder_last_hidden_staterencoder_attentionsr,) rArUrBrr:rYr?r=rc)rZr=r@rrrrrr:r>rrMrr[rdecoder_outputss r4rT5GemmaModel.forwards.  ""ll#-)+   O!0 A A,,  '1-/+"7#1)    "-??+;;zz0%88#2"?"?!335.99,==&5&G&G"1"?"?.99  r3)rBrA) NNNNNNNNNNNN)r.r/r0r1r7rXrrrvrzrrrrrrr BoolTensorrr rrtrrrrr2rwrxs@r4rlrlsd } 3A156:378<=A;?599=048<$(598 E,,-8 !!2!238 u//0 8 $E$4$45 8 !))9)9 : 8 'u'7'788 "/28 ""568   -8  ( 58 D>8 !!1!128 +,8  8 8 r3rlc^\rSrSrS\4U4SjjrSrSr\\ SS\ \ RS\ \ RS\ \ RS \ \ RS \\S \4 S jj55rS rU=r$)T5GemmaEncoderModeli@rc>[TU]U5 UR(a [S5e[ UR 5UlUR 5 g)NzQT5GemmaEncoderModel only supports encoder-only model. Use `T5GemmaModel` instead.)rWrXrCrr<rArHrs r4rXT5GemmaEncoderModel.__init__Bs?   $ $pq q%fnn5  r3c6URR5$rjryrqs r4rz(T5GemmaEncoderModel.get_input_embeddingsKr|r3c8URRU5$rjr~rs r4r(T5GemmaEncoderModel.set_input_embeddingsNrr3r=r@rr>r[rc 4UR"SUUUUS.UD6nU$)Nrr,rp)rZr=r@rr>r[rs r4rT5GemmaEncoderModel.forwardQs5,, )%'    r3rpr^)r.r/r0r1r7rXrzrrrrrrrrrrrrr2rwrxs@r4rr@s}3A156:3704 E,,-!!2!23u//0    -  +,  r3rc%l^\rSrSrSS/rSS0rSS/S/40rS\4U4S jjrS r S r S r S r Sr \\S"S\\R$S\\R&S\\R$S\\R$S\\R(S\\R$S\\S\\S\\R&S\\R&S\\R$S\\S\\R$S\\\R44S\\S\\\R&\44 Sjj55rS\R44S jr S!r!U=r"$)#T5GemmaForConditionalGenerationiez!model.decoder.embed_tokens.weightzlm_head.out_proj.weightzlm_head.out_proj colwise_repr?rrc>SUl[TU] U5 [U5UlUR R Ul[UR RUR 5Ul SUl UR5 g)NT ForMaskedLM) rCrWrXrlrrBrHrrTlm_head loss_typerHrs r4rX(T5GemmaForConditionalGeneration.__init__jsb$(!  !&)  ..33$V^^%?%?Q & r3c$XRlgrjrr rs r4set_output_embeddings5T5GemmaForConditionalGeneration.set_output_embeddingsus . r3c.URR$rjrrqs r4get_output_embeddings5T5GemmaForConditionalGeneration.get_output_embeddingsxs||$$$r3cURR(aCURURRUR 5R 55 ggrj)rrG_tie_or_clone_weightsrr rvrzrqs r4 _tie_weights,T5GemmaForConditionalGeneration._tie_weights{s@ ;; * *  & &t||'<'N>N>P>e>e>g h +r3c.URR$rj)rrArqs r4rr+T5GemmaForConditionalGeneration.get_encoderzz!!!r3c.URR$rj)rrBrqs r4rv+T5GemmaForConditionalGeneration.get_decoderrr3r=r@rrrrrr:r>rlabelsrMrlogits_to_keepr[rc UR(acURRS:waISURRS3n[5(a [ U5e[ R U5 U bUcU cURU 5nUR"SUUUUUUUUU U U U S. UD6nURn[U[5(a [U*S5OUnURUSS2USS245nUR5RnURb4UUR- n[ R""U5nUUR-nSnU bUR$"UXR&40UD6n[)UUUR*UR,UR.UR0UR2UR4UR6S9 $)a decoder_position_ids (`torch.LongTensor` of shape `(batch_size, decoder_sequence_length)`, *optional*): Indices of positions of each decoder input sequence tokens in the position embeddings. Selected in the range `[0, config.decoder.n_positions - 1]`. [What are position IDs?](../glossary#position-ids) labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. rziIt is strongly recommended to train T5Gemma models with the `eager` attention implementation instead of `zp`. Use `eager` with `AutoModelForCausalLM.from_pretrained('', attn_implementation='eager')`.N) r=r@rrrrrr:r>rrMr) lossrr:rrrcrrrr,)rrrbrrlogger warning_oncer0rrUrOrvslicerrvfinal_logit_softcappingrtanh loss_functionrHrr:rrrcrrr)rZr=r@rrrrrr:r>rrrMrrr[msgrr? slice_indicesrdecoder_configrs r4r'T5GemmaForConditionalGeneration.forwards: ==T[[==H#{{??@Aqr ()) o%##C(  "3";@U@] $ 1 1& 9 .2jj/ )%/#9!5++'"7)/ /  (99 8B>SV8W8W~ot4]k mA}a,?@A))+22  1 1 =nDDDFZZ'FnDDDF  %%ffooPPD+;;"1"G"G.AA,==&5&O&O"1"G"G.AA  r3c$URU5$rj)r0)rZrs r4%prepare_decoder_input_ids_from_labelsET5GemmaForConditionalGeneration.prepare_decoder_input_ids_from_labelss  ((r3)rrrrH)NNNNNNNNNNNNNr)#r.r/r0r1_tied_weights_keys_tp_plan_pp_planr7rXrrrrrrvrrrrrrrrr rtrrvrrrrrrrr2rwrxs@r4rres=?XY"M2H"o%6 $CDH } /%i ""156:378<=A;?599=59=A-1$(5934R E,,-R !!2!23R u//0 R $E$4$45 R !))9)9 : R 'u'7'78R "/2R ""56R   1 12R  ((9(9:R ))*R D>R !!1!12R c5<</0R +,!R " uU&&'8 9#R R h)ELL))r3rc^\rSrSrSS\S\\4U4SjjjrSrSr \ \ SS\\ RS\\ RS \\ RS \\ RS \\ RS \\ RS \\S\\ R S\\ R S\\ RS\\S\4Sjj55rSrU=r$) T5GemmaForSequenceClassificationirrCc>UbX!l[TU] U5 URUlUR(a[ U5UlO[ U5UlURRnUR(aURRn[USS5n[X0RU5Ul UR5 g)z is_encoder_decoder (`Optional`, *optional*): Whether use encoder_decoder for sequence classification. When set to False, only encoder is used. NrE皙?rCrWrXr rlrrrArTrBrVr scorerHrZrrCrTclassifier_dropoutr]s r4rX)T5GemmaForSequenceClassification.__init__s  )(: %   ++  $ $%f-DJ,V4DJnn00  $ $ ..44K$V-FL.{OOM_`  r3c6URR5$rjrrzrqs r4rz5T5GemmaForSequenceClassification.get_input_embeddingszz..00r3c:URRU5 grjrrrZrfs r4r5T5GemmaForSequenceClassification.set_input_embeddings ''.r3r=r@rrrrrr>rrr[rc URR(a)Uc&Ub#[SURRS35eURR(a%Uc"U cUc [ S5eUR U5nURR(aBUR"U4UUUUUUUU SS. U D6n U Rn U RnU RnO;UR"U4UUUS.U D6n U Rn U RnU RnURU 5nUbURSnOURSnURRcUS :wa [ S 5eURRcS nGOUbXRR:gR!UR"[$R&5n[$R("URS UR"[$R&S 9nUU-R+S 5nURR(a*US - n[$R,"UURS S - S 9nO.S n[.R1URRS35 U[$R("UUR"S9U4nSnU bUR3UU UURS9n[5UUUUS9$) decoder_position_ids (`torch.LongTensor` of shape `(batch_size, decoder_sequence_length)`, *optional*): Indices of positions of each decoder input sequence tokens in the position embeddings. Selected in the range `[0, config.decoder.n_positions - 1]`. [What are position IDs?](../glossary#position-ids) labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). N8Passing input embeddings is currently not supported for  in encoder-decoder mode.If no `decoder_input_ids` or `decoder_inputs_embeds` are passed, `input_ids` cannot be `None`. Please pass either `input_ids` or `decoder_input_ids` or `decoder_inputs_embeds`.F r@rrrrrr>rrMr@rr>rrz=Cannot handle batch sizes > 1 if no padding token is defined.rr8)maxz will not detect padding tokens in `inputs_embeds`. Results may be unexpected if using padding tokens in conjunction with `inputs_embeds.`rL)rr pooled_logitsrrrr?r=)rrCNotImplementedErrorr]r.rr0rrUrrr?r=rrrKrrrint32rVargmaxclamprrrr)rZr=r@rrrrrr>rrr[outputsrUr?r=r batch_sizelast_non_pad_token non_pad_mask token_indicesrrs r4r(T5GemmaForSequenceClassification.forwards2 ;; ) )y/@]E^%J4>>KbKbJcc|}  ;; ) )/@/HMbMj  U !% 1 1) <  ;; ) )*.** +-)"3'=%9 /+&; + +G!( 9 9 #99M 33J'+zz(-)+ (  (G!( 9 9 #11M ++J-.  "+J&,,Q/J ;; # # + a\] ] ;; # # +!#   "%)A)AAEEfmmUZU`U`aL!LL)!F!Fr!J {{--"a'"%*[[1CIZI`I`acIdghIh%i"!#    >>**+,ZZ  u||Jv}}MOaab   %%VFR_hlhshs%tD' '!   r3rr rrj NNNNNNNNNN)r.r/r0r1r7rrtrXrzrrrrrrrrrrrrr2rwrxs@r4rrsS}(4..1/1515378<9=;?5959=A-1i E,,-i !.i u//0 i $E$4$45 i !) 6 i 'u'7'78i "/2i   1 12i  ((9(9:i ))*i +,i  "i i r3rc^\rSrSrSS\S\\4U4SjjjrSrSr \ \ SS\\ RS\\ RS \\ RS \\ RS \\ RS \\ RS \\S\\ R S\\ R S\\ RS\\S\4Sjj55rSrU=r$)T5GemmaForTokenClassificationimrrCc>UbX!l[TU] U5 URUlUR(a[ U5UlO[ U5UlURRnUR(aURRn[USS5n[X0RU5Ul UR5 g)z is_encoder_decoder (`Optional`, *optional*): Whether use encoder_decoder for token classification. When set to False, only encoder is used. NrErrrs r4rX&T5GemmaForTokenClassification.__init__os  )(: %   ++  $ $%f-DJ,V4DJnn00  $ $ ..44K$V-FL.{OOM_`  r3c6URR5$rjrrqs r4rz2T5GemmaForTokenClassification.get_input_embeddingsrr3c:URRU5 grjrrs r4r2T5GemmaForTokenClassification.set_input_embeddingsrr3r=r@rrrrrr>rrr[rc URR(a)Uc&Ub#[SURRS35eURR(a%Uc"U cUc [ S5eUR U5nURR(aBUR"U4UUUUUUUU SS. U D6n U Rn U RnU RnO;UR"U4UUUS.U D6n U Rn U RnU RnURU 5nSnU bURUXR5n[UUUUS9$) rNrrrFrrr)rrCrr]r.rr0rrUrrr?r=rrr)rZr=r@rrrrrr>rrr[rrUr?r=rrs r4r%T5GemmaForTokenClassification.forwards4 ;; ) )y/@]E^%J4>>KbKbJcc|}  ;; ) )/@/HMbMj  U !% 1 1) <  ;; ) )*.** +-)"3'=%9 /+&; + +G!( 9 9 #99M 33J'+zz(-)+ (  (G!( 9 9 #11M ++J-.  %%ffkkBD$'!   r3rrjr)r.r/r0r1r7rrtrXrzrrrrrrrrrrrrr2rwrxs@r4rrmsS}(4.01/1515378<9=;?5959=A-1N E,,-N !.N u//0 N $E$4$45 N !) 6 N 'u'7'78N "/2N   1 12N  ((9(9:N ))*N +,N  N N r3r)r7r*rrlrrrr)Stypingrrrrrtorch.nnrtransformers.utils.genericrr cache_utilsr r r configuration_utilsr generationrmodeling_flash_attention_utilsrmodeling_layersrmodeling_outputsrrrrrrmodeling_utilsrprocessing_utilsrutilsrrrrrrgemma2.configuration_gemma2r gemma2.modeling_gemma2r!r"r#r$r%r&r'r(_CHECKPOINT_FOR_DOC get_loggerr.rr*r7rzr}rrrrrrvrrrModuler rrrr:r<rarlrrrr__all__r,r3r4r sY 21 ICC3)B96&7   9 !!   H % , Q$Qh ]   )2) +?+C)OC)L 0F 8 sx141h7-7t   BII ,!2,!,!^(()<<3- \\ "W +W tm ^m `R )R R j!0!!Hx)&