"S4S5\ RZ5r?"S6S7\ RZ5r@"S8S9\ RZ5rA"S:S;\ RZ5rB"S<S=\ RZ5rC"S>S?5rD\$"S@SA\55rE\$"SBSC9"SDSE\E55rF\$"SFSG\E55rG\$"SHSC9"SISJ\E\55rH/SKQrIg)NzPyTorch Chameleon model.)cached_property)CallableOptionalUnionN)nn)ACT2FN)Cache DynamicCache)GenerationMixin)create_causal_mask)FlashAttentionKwargs)GradientCheckpointingLayer)BaseModelOutputWithPastCausalLMOutputWithPast)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack)TransformersKwargsauto_docstringcan_return_tuplelogging)ChameleonConfigChameleonVQVAEConfigc8^\rSrSrSU4SjjrSrSrSrU=r$)ChameleonRMSNorm/c>[TU]5 [R"[R "U55UlX lg)z/ ChameleonRMSNorm is equivalent to T5LayerNorm N)super__init__r Parametertorchonesweightvariance_epsilon)self hidden_sizeeps __class__s h/var/www/html/shao/venv/lib/python3.13/site-packages/transformers/models/chameleon/modeling_chameleon.pyr!ChameleonRMSNorm.__init__0s/ ll5::k#:; #cURnUR[R5nUR S5R SSS9nU[R "X0R-5-nURURU5-$)NT)keepdim) dtypetor#float32powmeanrsqrtr&r%)r' hidden_states input_dtypevariances r+forwardChameleonRMSNorm.forward8sw#)) %((7  $$Q',,R,>% H?T?T4T(UU {{]--k:::r-c^[URR5SUR3$)Nz, eps=)tupler%shaper&r's r+ extra_reprChameleonRMSNorm.extra_repr?s* ))*+6$2G2G1HIIr-)r&r%)ư>) __name__ __module__ __qualname____firstlineno__r!r;rA__static_attributes__ __classcell__r*s@r+rr/s$;JJr-rcZ^\rSrSrSU4Sjjr\R "5S5rSrU=r $)ChameleonRotaryEmbeddingEc P>[TU]5 XPlXlX lX0lSUR [ R"SURS[ RS9RU[ RS9UR- -- nURSUSS9 X l g) N?rr/r2devicer2inv_freqF persistent) r r!scaling_factordimmax_position_embeddingsbaser#arangeint64r3floatregister_buffermax_seq_len_cached)r'rWrXrYrRrVrSr*s r+r!!ChameleonRotaryEmbedding.__init__Fs ,'>$  II Q!5;;?BB&X]XcXcBdgkgogoo q  ZeD"9r-cURSSS2S4R5RURSSS5nUSS2SSS24R5nURR nUS:waUOSn[ R"USS9 UR5UR5-RSS5n[ R"Xf4SS 9nUR5nUR5n SSS5 WRURS 9W RURS 94$!,(df  N@=f) Nrr0rmpscpuF) device_typeenabledr/rWrP)rSr\expandr?rRtyper#autocast transposecatcossinr3r2) r'x position_idsinv_freq_expandedposition_ids_expandedrcfreqsembrkrls r+r; ChameleonRotaryEmbedding.forwardTs!MM$4-8>>@GG HZHZ[\H]_acde ,QaZ 8 > > @hhmm %0E%9ku ^^ U C&,,.1F1L1L1NNYYZ[]^_E))UN3C'')C'')C D vvAGGv$cff177f&;;; D Cs A(D22 E)rYrWrXr^rV)ii'NrO) rDrErFrGr!r#no_gradr;rHrIrJs@r+rLrLEs" : ]]_ < UR5UR- n[TU] X5up4X44$N)r\rVr r;)r'rmrnrkrlr*s r+r;-ChameleonLinearScalingRotaryEmbedding.forwardhs3#))+d.A.AA 7?13xr-rDrErFrG__doc__r;rHrIrJs@r+rvrvesir-rvc,^\rSrSrSrU4SjrSrU=r$))ChameleonDynamicNTKScalingRotaryEmbeddingozqChameleonRotaryEmbedding extended with Dynamic NTK scaling. Credits to the Reddit users /u/bloc97 and /u/emozillac >[R"U5S-nX0R:aURURU-UR- URS- - UR UR S- - --nSU[R "SUR S[RS9RUR[RS9UR - -- nURSUSS 9 [TU]5X5upgXg4$) Nrr/rOrrPrQrSFrT)r#maxrXrYrVrWrZr[r3rRr\r]r r;) r'rmrnseq_lenrYrSrkrlr*s r+r;1ChameleonDynamicNTKScalingRotaryEmbedding.forwardrs))L)A- 11 199$$w.1M1MMRVReRehiRij((dhhl+ --DLLDHHau{{CFFahh^c^i^iFjmqmumuuwH  X% H7?13xr-r{r|rJs@r+rros{r-rcUSSURSS-24nUSURSS-S24n[R"U*U4SS9$)z*Rotates half the hidden dims of the input..Nr0r/re)r?r#rj)rmx1x2s r+ rotate_halfrsZ 3"!''"+"" " #B 3 q " " #B 99rc2YB ''r-cURU5nURU5nX-[U5U--nX-[U5U--nXg4$)aApplies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): The key tensor. cos (`torch.Tensor`): The cosine part of the rotary embedding. sin (`torch.Tensor`): The sine part of the rotary embedding. position_ids (`torch.Tensor`, *optional*): Deprecated and unused. unsqueeze_dim (`int`, *optional*, defaults to 1): The 'unsqueeze_dim' argument specifies the dimension along which to unsqueeze cos[position_ids] and sin[position_ids] so that they can be properly broadcasted to the dimensions of q and k. For example, note that cos[position_ids] and sin[position_ids] have the shape [batch_size, seq_len, head_dim]. Then, if q and k have the shape [batch_size, heads, seq_len, head_dim], then setting unsqueeze_dim=1 makes cos[position_ids] and sin[position_ids] broadcastable to the shapes of q and k. Similarly, if q and k have the shape [batch_size, seq_len, heads, head_dim], then set unsqueeze_dim=2. Returns: `tuple(torch.Tensor)` comprising of the query and key tensors rotated using the Rotary Position Embedding. ) unsqueezer)qkrkrlrn unsqueeze_dimq_embedk_embeds r+apply_rotary_pos_embrsS( -- &C -- &Cw;q>C/0Gw;q>C/0G  r-c.^\rSrSrU4SjrSrSrU=r$) ChameleonMLPc>[TU]5 XlURUlURUl[ R "URURURS9Ul[ R "URURURS9Ul [ R "URURURS9Ul [URUl g)Nbias)r r!configr(intermediate_sizerLinearmlp_bias gate_projup_proj down_projr hidden_actact_fnr'rr*s r+r!ChameleonMLP.__init__s  !--!'!9!94#3#3T5K5KRXRaRabyy!1!143I3IPVP_P_` 4#9#94;K;KRXRaRabV../ r-cURURURU55URU5-5nU$ry)rrrr)r'rmrs r+r;ChameleonMLP.forwards6NN4;;t~~a/@#ADLLQRO#ST r-)rrrrr(rrrDrErFrGr!r;rHrIrJs@r+rrs0r-rc2^\rSrSrSrU4SjrSrSrU=r$)ChameleonLayerNormar LayerNorm but computes stats only over the last dim because Chameleon applies gamma and beta from each shard separately to each head, instead of reducing. We can apply each head's own gamma/beta by repeat-interleaving weights from each shard, but the stats have to be computed in the last dimension. This module applies gamma/beta manually to fulfill this requirement. cD>[TU]"U/UQ70UD6 US4Ulg)Nr0)r r!normalized_shape)r'r(argskwargsr*s r+r!ChameleonLayerNorm.__init__s) 6t6v6!,R 2r-c~[R"XRSSSS9nXR-UR-nU$)Ngh㈵>r))F layer_normrr%rr'r8s r+r;ChameleonLayerNorm.forwards9 ]4I4I4QU[_` % 3dii? r-)r) rDrErFrGr}r!r;rHrIrJs@r+rrs3r-rr8n_repreturncURup#pEUS:XaU$USS2SS2SSS2SS24RX#XU5nURX#U-XE5$)z This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, head_dim) to (batch, num_attention_heads, seqlen, head_dim) rN)r?rfreshape)r8rbatchnum_key_value_headsslenhead_dims r+ repeat_kvrs_ 2?1D1D.E z!!Qa"23::5W\dlmM  e(CT TTr-modulequerykeyvalueattention_maskscalingdropoutrc @[X R5n[X0R5n [R"XR SS55U-n Ub"USS2SS2SS2SUR S24n X-n [ RRU S[RS9RUR5n [ RRXURS9n [R"X5n U R SS5R5n X4$)Nr/rr0)rWr2)ptrainingr)rnum_key_value_groupsr#matmulrir?r functionalsoftmaxr4r3r2rr contiguous) rrrrrrrr key_states value_states attn_weights causal_mask attn_outputs r+eager_attention_forwardrs3 ; ;[TU]5 XlX lUc-[R SUR RS35 URUlURUl URUl URUR-Ul URUl URUR-UlURUlUR UlSUlUR$UlURS-UlURUR-UR:wa&[)SURSURS35e[*R,"URURUR-UR.S9Ul[*R,"URURUR-UR.S9Ul[*R,"URURUR-UR.S9Ul[*R,"URURUR.S9UlRUR45Ul[9URUR45UlUR?5 g) NzInstantiating z without passing a `layer_idx` is not recommended and will lead to errors during the forward call if caching is used. Please make sure to provide a `layer_idx` when creating this class.Tz?hidden_size must be divisible by num_heads (got `hidden_size`: z and `num_heads`: z).r) r r!rrlogger warning_oncer*rDattention_dropoutr(num_attention_heads num_headsrrrrX rope_theta is_causalmodel_parallel_sizer ValueErrorrrattention_biasq_projk_projv_projo_projrq_normk_norm _init_roper'rrr*s r+r!ChameleonAttention.__init__s,  "      !8!8 9:,,  "(!9!9!--33((DNN: #)#=#= $(NNd6N6N$N!'-'E'E$ ++#)#=#= }}d* MMDNN *t/?/? ?QRVRbRbQc$T^^$4B8  ii 0 0$..4==2PW]WlWlm ii 0 0$2J2JT]]2Zagavavw ii 0 0$2J2JT]]2Zagavavw ii 0 0$2B2BI^I^_ ($..$--)HI ($*B*BDMM)RS  r-cURRc/[URURUR S9UlgURRSnURRSnUS:Xa0[URURUUR S9UlgUS:Xa0[URURUUR S9Ulg[SU35e)N)rXrYrgfactorlinear)rXrVrYdynamiczUnknown RoPE scaling type ) r rope_scalingrLrrXr rotary_embrvrr)r' scaling_typerVs r+rChameleonAttention._init_ropes ;; # # +6 (,(D(D__DO  ;;33F;L![[55h?Nx'"GMM,0,H,H#1 # *"KMM,0,H,H#1 #!#=l^!LMMr-r8rrnpast_key_valueoutput_attentions use_cachecache_positionrc UR5upn URU5n URU5n URU5nU R SUR UR 5n URU 5n U R SURUR 5n URU 5n U R XUR UR 5RSS5n U R XURUR 5RSS5n URXURUR 5RSS5nURX5unn[XUU5upUb$UXS.nURXURU5up[ nUR"R$S:wa[&UR"R$nU"UU U UU4UR((dSO UR*UR,S.UD6unnUR XS5R/5nUR1U5nUU4$)Nr0rr/)rlrkreager)rr)sizerrrrrrrrrriviewrrupdaterrr_attn_implementationrrrrrr)r'r8rrnrrrrrbszq_len_ query_statesrrrkrl cache_kwargsattention_interfacerrs r+r;ChameleonAttention.forward8s&**, A{{=1 [[/ {{=1 #++B N {{<0 ''D,D,DdmmT [[, #++C V``abdef ''D4L4Ldmm\ffghjkl #((T5M5Mt}}]gghiklm ??<>S#7 RUWZ#[  %#&sUL'5'<'r;rHrIrJs@r+rrsG##8C=##NN<2637*."'595)||5)!.5)u//0 5) ! 5)  5)5)!!1!125) u||Xell3XeELL>Q5RR S5)5)r-rc8^\rSrSrS\S\4U4SjjrSS\RS\ \RS\ \RS\ \ S \ \ S \ \ S \ \RS \ \R\ \ \R\R444S jjrSrU=r$)ChameleonDecoderLayeriqrrc >[TU]5 URUl[XS9Ul[ U5Ul[URURS9Ul [URURS9Ul gN)rrr r r!r(r self_attnrmlpr rms_norm_epsinput_layernormpost_attention_layernormrs r+r!ChameleonDecoderLayer.__init__rk !--+6O'/0B0BH[H[\(89K9KQWQdQd(e%r-r8rrnrrrrrc Un URU5nUR"SUUUUUUUS.UD6upX-nUn URU5nURU5nX-nU4n U(aX4- n U $)at Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`, *optional*): attention mask of size `(batch_size, sequence_length)` if flash attention is used or `(batch_size, 1, query_sequence_length, key_sequence_length)` if default attention is used. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). past_key_value (`Tuple(torch.FloatTensor)`, *optional*): cached past key and value projection states cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence kwargs (`dict`, *optional*): Arbitrary kwargs to be ignored, used for FSDP and other methods that injects code into the model r8rrnrrrrr{)rrrr r'r8rrnrrrrrresidualself_attn_weightsoutputss r+r;ChameleonDecoderLayer.forward|s<!,,]; ,0>> , ')%)/) ,  , ( !0 !55mD /  0  "  + +Gr-r(rrrrrrDrErFrGrrr!r#rrrr r r> FloatTensorr;rHrIrJs@r+r r qsff3f2637*.,1$)59:||:!.:u//0 : ! : $D> :D>:!!1!12: u  (51B1BEDUDU1U+V"WW X::r-r c8^\rSrSrS\S\4U4SjjrSS\RS\ \RS\ \RS\ \ S \ \ S \ \ S \ \RS \ \R\ \ \R\R444S jjrSrU=r$)ChameleonSwinDecoderLayerirrc >[TU]5 URUl[XS9Ul[ U5Ul[URURS9Ul [URURS9Ul gr rrs r+r!"ChameleonSwinDecoderLayer.__init__rr-r8rrnrrrrrc Un UR"SUUUUUUUS.UD6upURU5nX-nUn URU5nURU5nX-nU4n U(aX4- n U $)a Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`, *optional*): attention mask of size `(batch_size, sequence_length)` if flash attention is used or `(batch_size, 1, query_sequence_length, key_sequence_length)` if default attention is used. position_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Indices of positions of each input sequence tokens in the position embeddings past_key_value (`Tuple(torch.FloatTensor)`, *optional*): cached past key and value projection states output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence. rr{)rrrrrs r+r;!ChameleonSwinDecoderLayer.forwards>!,0>> , ')%)/) ,  , ( ,,];  0  / 55mD  0  "  + +Gr-rrrrJs@r+r!r!sff3f2637*.,1$)598||8!.8u//0 8 ! 8 $D> 8D>8!!1!128 u  (51B1BEDUDU1U+V"WW X88r-r!cN^\rSrSrSrU4SjrS\R4SjrSr U=r $)ChameleonVQVAEVectorQuantizeria A module for vector quantization using learned embedding vectors. This module implements the quantization process similar to te one described in the VQ-VAE (Vector Quantized Variational AutoEncoder) paper. It quantizes continuous input vectors into discrete codebook vectors, which are learned during training. Current implementation improves over previous ones by avoiding costly matrix multiplications and allowing for post-hoc remapping of indices. c>[TU]5 URUlURUl[ USS5Ul[R"URUR5Ul g)Nbetag?) r r!num_embeddings embed_dim embedding_dimgetattrr)r Embedding embeddingrs r+r!&ChameleonVQVAEVectorQuantizer.__init__ sX $33#--FFD1 d&9&94;M;MNr- hidden_statec >URSSSS5R5nURSUR5n[R "US-SSS9[R "UR RS-SS9-S[R"S X R RRSS55-- n[R"USS9nUR U5RUR5n[R"UR5U- S-5UR[R"XQR5- S-5--nXU- R5-nURSSSS5R5nXVU4$) Nrr/rrr0T)rWr1rez bd,dn->bn)permuterrr,r#sumr/r%einsumriargminr?r6detachr))r'r1hidden_state_flattened distancesmin_encoding_indiceshidden_state_quantlosss r+r;%ChameleonVQVAEVectorQuantizer.forwards#++Aq!Q7BBD !-!2!22t7I7I!J II,a/Q Eii--q0a8 9%,,{,BNNDYDYDcDcdeghDijj k  %||I1=!^^,@AFF|GYGYZzz-446E!KLtyy[`[e[e "5"5"7 7A =\ P   *,-N,V,V,XX0771aCNNP!)===r-)r)r/r,r*) rDrErFrGr}r!r#rr;rHrIrJs@r+r'r's#O>ELL>>r-r'c.^\rSrSrU4SjrSrSrU=r$)#ChameleonVQVAEEncoderConvDownsamplei.cZ>[TU]5 [R"XSSSS9Ulg)Nrr/r kernel_sizestridepadding)r r!rConv2dconvr' in_channelsr*s r+r!,ChameleonVQVAEEncoderConvDownsample.__init__/s% IIkAaYZ[ r-cV[R"USSSS9nURU5nU$)N)rrrrconstantr)padmoder)rrLrFrs r+r;+ChameleonVQVAEEncoderConvDownsample.forward3s+mJVWX  -0 r-)rFrrJs@r+r?r?.s\r-r?c6^\rSrSrSU4SjjrSrSrU=r$) ChameleonVQVAEEncoderResnetBlocki:c>[TU]5 X lUcUOUUlX@l[ R RSUSSS9Ul[ R RX#SSSS9Ul [ R RSUSSS9Ul [ R RUR5Ul [ R RX3SSSS9UlURUR:wa]UR(a&[ R RX#SSSS9Ulg[ R RX#SSSS9Ulgg) N rCT num_groups num_channelsr)affinerrrAr)r r!rH out_channelsuse_conv_shortcutr#r GroupNormnorm1rEconv1norm2Dropoutrconv2 conv_shortcut nin_shortcut)r'rrHrWr_r*s r+r!)ChameleonVQVAEEncoderResnetBlock.__init__;s% &+7+?K\!.XX''2KUYbf'g XX__[AVWab_c XX''2LVZcg'h xx''7 XX__\QWXbc_d   t00 0%%%*XX__[\]fgqr_%s"$)HHOOK[\efpqO$r! 1r-cUnURU5nU[R"U5-nURU5nUR U5nU[R"U5-nUR U5nUR U5nURUR:wa7UR(aURU5nX!-$URU5nX!-$ry) rZr#sigmoidr[r\rr^rHrWrXr_r`)r'r8rs r+r;(ChameleonVQVAEEncoderResnetBlock.forwardRs  =1 }55  =1  =1 }55  ]3  =1   t00 0%%--h7'' ,,X6''r-) r[r^r_rrHr`rZr\rWrX)NFrrJs@r+rPrP:s  s.((r-rPc.^\rSrSrU4SjrSrSrU=r$)ChameleonVQVAEEncoderAttnBlockifc>[TU]5 Xl[RR SUSSS9Ul[RRXSSSS9Ul[RRXSSSS9Ul [RRXSSSS9Ul [RRXSSSS9Ul g)NrRrCTrSrrrA) r r!rHr#rrYnormrErrvproj_outrGs r+r!'ChameleonVQVAEEncoderAttnBlock.__init__gs &HH&&";TXae&f qQR\]^qQR\]^qQR\]^ aXYcde r-cZUnURU5nURU5nURU5nURU5nURupgpUR XgX-5R SSS5nUR XgX-5n[R"X45n U [U5S--n [R"U SS9n UR XgX-5nU R SSS5n [R"XZ5R XgX5n URU 5n X+-$)Nrr/rrre) rhrrrir?rr3r#bmmrrrrj) r'r8rrrr batch_sizechannelsheightwidthrrs r+r;&ChameleonVQVAEEncoderAttnBlock.forwardqs  -0 vvm, VVM* vvm, /;.@.@+ f#++J&.QYYZ[]^`ab '' fnM yy: #s8}'>? yy15 $++J&.Q #++Aq!4 ii ;CCJZ`h mmK0 %%r-)rHrrhrjrrirrJs@r+rfrffsf&&r-rfcJ^\rSrSrU4SjrS\R 4SjrSrU=r $)ChameleonVQVAEEncoderic >[TU]5 [UR5UlUR UlUR nURnURnURnURnURn[RRXBSSSS9UlUnS[U5-n Xl[R""5Ul['UR5GH%n [R""5n [R""5n X)U -n X'U -n['UR 5HknU R)[+UU US95 Un UR,cM.XR,;dM?UR.S:XdMQU R)[1U 55 Mm [R2"5nU UlU UlXRS- :wa[9U 5UlUS-nUR$R)U5 GM( [R2"5Ul[+UW U S9UR<lUR.S:Xa [1U 5O[R@"5UR<l![+UU U S9UR<l"[RRGSU S S S 9Ul$[RRU U(aSU-OUSSSS9Ul%g) NrrrA)r)rrHrWvanillar/rRrCTrS)&r r!lenchannel_multipliernum_resolutionsnum_res_blocks base_channels resolutionrH double_latentlatent_channelsr#rrEconv_inr>in_channel_multiplier ModuleListdownrangeappendrPattn_resolutions attn_typerfModuleblockattnr? downsamplemidblock_1Identityattn_1block_2rYnorm_outconv_out)r'rr{r|rHr}r~rxcurr_resri_levelrrblock_in block_outi_blockrr*s r+r!ChameleonVQVAEEncoder.__init__s "6#<#<=$33,, && (( ,,  00#66xx{qYZdef  $u-?'@ @%:"MMO T112GMMOE==?D$W'EEH%7(CCI !4!45 4%$,%.%++7 $;$;;((I5KK >x HI6 99;DDJDI..22"Eh"O#q= II  T "73:99;; !  GMFVFVZcFc8Bikititiv; !  **bxUYbf*g  #0A o (  r- pixel_valuesc@URU5/n[UR5Hn[UR5HnURUR U"US5n[ URUR5S:a"URURU"U5nURU5 M X0RS- :wdMURURURUS55 M USnURRU5nURRU5nURRU5nURU5nU[R "U5-nUR#U5nU$)Nr0rr)rrryrzrrrwrrrrrrrrr#rcr)r'rr8rrr1last_hidden_states r+r;ChameleonVQVAEEncoder.forwardsrl34 T112G !4!45#yy177@!"%  tyy)../!3#'99W#5#:#:7#CL#QL$$\2 6..22$$TYYw%7%B%B=QSCT%UV3*"- HH,,->? HHOO,=> HH,,->?!MM*;<U]]+<== MM*;<  r-)rrrrrrrzry) rDrErFrGr!r#rr;rHrIrJs@r+rtrts!C J!E$4$4!!r-rtc\rSrSrSrSr\S5r\S5r\S5r \S5r \S5r \S 5r S \ RS \ R4S jrS rg)ChameleonImageVocabularyMappingizE A class for mapping discrete image tokens from VQGAN to BPE tokens. c<XlURS5Ulg)Nz) vocab_mapgetimage_token_id)r'rs r+r!(ChameleonImageVocabularyMapping.__init__s"'mmI6r-cjURR5VVs0sHupX!_M snn$s snnfry)ritemsr'rris r+val2name(ChameleonImageVocabularyMapping.val2names-!%!5!5!78!7!7888 /c [URR5VVs/sHupURS5(dMUPM! snn5$s snnf)NIMGIMG)sortedrr startswith)r'namevals r+ image_tokens,ChameleonImageVocabularyMapping.image_tokenss<DNN,@,@,B`,BytdooV^F_s,B`aa`s A A c $^[S5Vs0sH$n[[S5U-5[U5_M& snmS[S[4U4SjjnURVs0sH!o3[ U"UR U55_M# sn$s snfs snf)N Aold_namercR>SRU4SjU[S5S55$)Nc3D># UHnTRX5v M g7fry)r).0cimg_tkn_chr_mappings r+ IChameleonImageVocabularyMapping.bpe2img..remap..s"_B^Q.22188B^s rr0)joinrw)rrs r+remap6ChameleonImageVocabularyMapping.bpe2img..remaps$77_(3x=[]B^__ _r-)rchrordstrrrr)r'irtokrs @r+bpe2img'ChameleonImageVocabularyMapping.bpe2imgsBG)L)Qs3s8a<0#a&8)L `C `C `@D?P?PQ?PSt}}S1233?PQQ M Rs *B'B cjURR5VVs0sHupX!_M snn$s snnfry)rrrs r+img2bpe'ChameleonImageVocabularyMapping.img2bpes-!%!3!3!56!5!5666rc[R"[URR 555[R"[URR 5554$ry)r#tensorrrkeysvaluesr@s r+bpe2img_search_tensors6ChameleonImageVocabularyMapping.bpe2img_search_tensorssC||F4<<#4#4#678%,,vdllNaNaNcGd:eeer-c[R"[URR 55S-[R S9nURR 5H up#X1U'M U$)NrrP)r#zerosrrrrr)r'mappingrris r+img2bpe_mapping_tensor6ChameleonImageVocabularyMapping.img2bpe_mapping_tensor sR++c$,,"3"3"56:%))LLL&&(DAAJ)r- img_batchrcxURnURURS5nURU5$)Nrb)rRrr3)r'rrR img_tokenss r+convert_img2bpe/ChameleonImageVocabularyMapping.convert_img2bpes5!!00e1DE }}V$$r-)rrN)rDrErFrGr}r!rrrrrrrr#rrrHr{r-r+rrs799bbRR77ff %%%,,%r-rcP\rSrSr%\\S'SrSrSS/rSS/r Sr Sr Sr S r SrSrS rg ) ChameleonPreTrainedModelirmodelTr r!past_key_valuesrFr{N)rDrErFrGr__annotations__base_model_prefixsupports_gradient_checkpointing_no_split_modules_skip_keys_device_placement_supports_flash_attn_supports_sdpa_can_compile_fullgraph!_supports_param_buffer_assignment_supports_flex_attn_supports_attention_backendrHr{r-r+rrsN &*#02MN#4m"DN!(-%"&r-raW The VQ-VAE model used in Chameleon for encoding/decoding images into discrete tokens. This model follows the "Make-a-scene: Scene-based text-to-image generation with human priors" paper from [ Oran Gafni, Adam Polyak, Oron Ashual, Shelly Sheynin, Devi Parikh, and Yaniv Taigman](https://huggingface.co/papers/2203.13131). ) custom_introcf^\rSrSr%\\S'/SQrS\4U4SjjrS\R4Sjr Sr U=r $)ChameleonVQVAEi)r)r'rfrPcl>[TU]U5 [U5Ul[ U5Ul[ RRURURS5Ul [ RRURURS5Ul UR5 gNr)r r!rtencoderr'quantizer#rrEr~r+ quant_convpost_quant_convevalrs r+r!ChameleonVQVAE.__init__9s|  ,V4 5f= ((//&*@*@&BRBRTUV$xxv/?/?AWAWYZ[ r-rcvURU5nURU5nURU5up4nX4U4$ry)rrr)r'rr8quantemb_lossindicess r+encodeChameleonVQVAE.encodeBs< \2  6 #'==#? ''r-)rrrr) rDrErFrGrrrr!r#rrrHrIrJs@r+rr)s7 !  3(5#3#3((r-rc^\rSrSrS\4U4SjjrS\R4SjrS\R4Sjr S\RS\RS \R4S jr \ SS\ \RS\ \RS \ \RS \ \RS \ \S\ \RS\ \S\ \S\ \S\ \S\ \RS\\S\\\44Sjj5rSrU=r$)ChameleonModeliIrc >[TU]U5 URUlURUl[ R "URURUR5Ul[UR5Ul URR(d[O[n[ R "[#UR$5Vs/sH o2"X5PM sn5Ul[)URUR*S9Ul[.R1UR25UlSUlUR95 gs snf)NrF)r r! pad_token_id padding_idx vocab_sizerr.r( embed_tokensrvocabulary_mapvocabulary_mappingr swin_normr r!rrnum_hidden_layerslayersrrrhr _from_config vq_configvqmodelgradient_checkpointing post_init)r'r decoder_layerrr*s r+r!ChameleonModel.__init__Ks  !.. ++LL):):F.s^a^s )rrr8 attentions)rrr$ruse_return_dictr rrrrrrr"masked_scatterr#jit is_tracingr get_seq_lengthrZr?rRrr rrhr>r)r'rrrrnrrrrr$r%rr image_embedsrpast_seen_tokensrr8all_hidden_statesall_self_attnsr  layer_outputss r+r;ChameleonModel.forwards 2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKKrr;rHrIrJs@r+rrIs$U->-> !u/@/@ !"))":?:K:K"]b]n]n"015481537+/59$(,0/3&*59j E,,-j u001j !. j u//0 j "% j   1 12j D>j $D>j 'tnj d^j !!1!12j -.j  u-- .j j r-rzb Chameleon Model with a head on top used for outputting logits for next token prediction. c^\rSrSrS/rU4SjrSrSrSrSr \ \ SS\ \ RS \ \ RS \ \ R S \ \ RS \ \S \ \ RS\ \ RS\ \S\ \S\ \S\ \ RS\\S\\\44Sjj55rSU4SjjrSrU=r$)!ChameleonForConditionalGenerationizlm_head.weightc>[TU]U5 [U5UlURUl[ R "URURSS9UlUR5 g)NFr) r r!rrrrrr(lm_headr rs r+r!*ChameleonForConditionalGeneration.__init__sU  #F+  ++yy!3!3V5F5FUS  r-cXlgryr)r'decoders r+ set_decoder-ChameleonForConditionalGeneration.set_decoders r-cUR$ryr=r@s r+ get_decoder-ChameleonForConditionalGeneration.get_decoders zzr-c8URRU5$ry)rrr'rs r+r2ChameleonForConditionalGeneration.get_image_tokensszz**<88r-c8URRU5$ry)rrrEs r+r4ChameleonForConditionalGeneration.get_image_featuresszz,,\::r-rrrrnrrlabelsrrr$rrrc  0U bU OURRn U bU OURRn UR"SUUUUUUUU U SU S. U D6n U SnUR U5nURR R n[R"UR5RUSS2SS2U4'SnUb)UR"SXURRS.U D6n[UUU RU RU R S9$)aE labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import ChameleonProcessor, ChameleonForConditionalGeneration >>> import torch >>> import requests >>> from PIL import Image >>> model = ChameleonForConditionalGeneration.from_pretrained("facebook/chameleon-7b", torch_dtype=torch.bfloat16) >>> processor = ChameleonProcessor.from_pretrained("facebook/chameleon-7b") >>> prompt = "I used to know a lot about constellations when I was younger, but as I grew older, I forgot most of what I knew. These are the only two constellations that I really remember now.I would like for you to tell me about 3 more constellations and give me a little bit of history about the constellation." >>> image = Image.open(requests.get("https://nineplanets.org/wp-content/uploads/2020/12/the-big-dipper-1.jpg", stream=True).raw) >>> image_2 = Image.open(requests.get("https://www.kxan.com/wp-content/uploads/sites/40/2020/10/ORION.jpg", stream=True).raw) >>> inputs = processor(images=[image, image_2], text=prompt, return_tensors="pt").to(model.device, torch.bfloat16) >>> generated_ids = model.generate(**inputs, max_new_tokens=100, do_sample=False) >>> processor.batch_decode(generated_ids, skip_special_tokens=True)[0] ```NT) rrrrnrrrrr$r%rr)logitsrIr)r<rKrr8r*r{)rrr$rr:rrr#finfor2min loss_functionrrrr8r*)r'rrrrnrrrIrrr$rrrr8rKrr<s r+r;)ChameleonForConditionalGeneration.forwards-V2C1N-TXT_T_TqTq$8$D $++JjJj  **  %)%+'/!5)      m,zz44AA %*[[%>%B%Bq!\!"  %%pVt{{OeOepiopD%#33!//))   r-c V>[T U]"U4UUUUUUUS.U D6n USS:waSU S'U $)N)rrrrrrnrrr)r prepare_inputs_for_generation) r'rrrrrrrnrr model_inputsr*s r+rQ?ChameleonForConditionalGeneration.prepare_inputs_for_generationps\w<   %+)')%     !  !,0L (r-)r:rrr6)NNNNNNT)rDrErFrG_tied_weights_keysr!r?rBrrrrrr#rrrr r rrrr>rr;rQrHrIrJs@r+r8r8s ++9;15481537+/59-1$(,0/359O E,,-O u001O !. O u//0 O "% O   1 12O ))*O D>O $D>O 'tnO !!1!12O +,O  u,, -O O hr-r8)r8rrrr)r)Jr} functoolsrtypingrrrr#torch.nn.functionalrrrtorch.utils.checkpoint activationsr cache_utilsr r generationr masking_utilsr modeling_flash_attention_utilsrmodeling_layersrmodeling_outputsrrmodeling_utilsrrprocessing_utilsrutilsrrrrconfiguration_chameleonrr get_loggerrDrrrrLrvrrrr LayerNormrrrrr\rrr r!r'r?rPrfrtrrrrr8__all__r{r-r+rgs%,, !.)/B9OF& K   H %JryyJ,BII,>^ ")) )(ryy)(X &RYY &F^!BII^!B,%,%^ ' ' '(-((0s -s s l J(@/J  JZ pr-