ó <±h¹ãó6•SSKJr SSKrS\RS\S\R4SjrSS\RRS\RS \RS \RS\\RS\S \\S\\ S\ \RS44Sjjrg)é)ÚOptionalNÚ hidden_statesÚn_repÚreturncó •URup#pEUS:XaU$USS2SS2SSS2SS24RX#XU5nURX#U-XE5$)zÈ This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, head_dim) to (batch, num_attention_heads, seqlen, head_dim) éN)ÚshapeÚexpandÚreshape)rrÚbatchÚnum_key_value_headsÚslenÚhead_dims Ú\/var/www/html/shao/venv/lib/python3.13/site-packages/transformers/integrations/sdpa_paged.pyÚ repeat_kvrs_€ð 2?×1DÑ1DÑ.€E ØƒzØÐØ!¢!¢Q¨ªa²Ð"2Ñ3×:Ñ:¸5ÐW\ÐdlÓm€MØ× Ñ ¸eÑ(CÀTÓTÐTóÚmoduleÚqueryÚkeyÚvalueÚattention_maskÚdropoutÚscalingÚ is_causalc óè•URSS5n U b U R"X#UR40UD6up#[US5(a*[ X R 5n[ X0R 5nUn UR 5nUR 5nUR 5n[RRRUUUU UUSS9nURSS5R 5nUS4$)NÚcacheÚnum_key_value_groupsF)Ú attn_maskÚ dropout_pÚscalerré)ÚpopÚupdateÚ layer_idxÚhasattrrrÚ contiguousÚtorchÚnnÚ functionalÚscaled_dot_product_attentionÚ transpose)rrrrrrrrÚkwargsrÚcausal_maskÚattn_outputs rÚsdpa_attention_paged_forwardr/sï€ð J‰Jw Ó%€EØÑØ—\’\ #¨f×.>Ñ.>ÑIÀ&ÑI‰ ˆÜˆvÐ-×.Ñ.Ü˜×8Ñ8Ó9ˆÜ˜%×!<Ñ!<Ó=ˆà €KØ×ÑÓ€EØ .‰.Ó €CØ×ÑÓ€EÜ—(‘(×%Ñ%×BÑBØ ØØ ØØØØðCð€Kð×'Ñ'¨¨1Ó-×8Ñ8Ó:€Kà˜ÐÐr)gNN)Útypingrr'ÚTensorÚintrr(ÚModuleÚfloatÚboolÚtupler/©rrÚr8sÊðÝãð U˜UŸ\™\ð U°#ð U¸%¿,¹,ô Uð$Ø#Ø $ñ!ØH‰HO‰Oð!à<‰<ð!ð ‰ð!ð<‰<ð !ð ˜UŸ\™\Ñ*ð!ðð !ðe‰_ð!ð˜‰~ð!ðˆ5<‰<˜ÐÑö!r