tj}<dZddlZddlZddlZddlmZddlmZddl m Z ddl m Z mZddlmZdd lmZmZmZdd lmZdd lmZmZdd lmZdd lmZmZddlm Z ddl!m"Z"m#Z#m$Z$m%Z%ddl&m'Z'ddl(m)Z)m*Z*m+Z+m,Z,m-Z-ddl.m/Z/m0Z0m1Z1er ddl2m3Z3ddlm4Z4e%j5e6Z7Gddej8Z9Gdde9Z:e9e:dZ;Gddej8Z<GddeZ=e"Gd d!e Z>Gd"d#e>eZ?e"d$%Gd&d'e?Z@e"d(%Gd)d*e?ZAe"d+%Gd,d-e>ZBe"d.%Gd/d0e>eZCgd1ZDdS)2zPyTorch BARK model.N)nn) functional)initialization)Cache DynamicCache)GenerationMixin)#AlternatingCodebooksLogitsProcessor!BarkEosPrioritizerLogitsProcessorSuppressTokensLogitsProcessor)create_bidirectional_mask)!flash_attn_supports_top_left_maskis_flash_attn_available)GradientCheckpointingLayer)CausalLMOutputWithPastMaskedLMOutput)PreTrainedModel)auto_docstringis_accelerate_availableis_torch_accelerator_availablelogging) AutoModel)BarkCoarseConfig BarkConfigBarkFineConfigBarkSemanticConfigBarkSubModelConfig)BarkCoarseGenerationConfigBarkFineGenerationConfigBarkSemanticGenerationConfig)get_target_dtype)_flash_attention_forwardcDeZdZdfd ZdZdZd dZ d dZxZS) BarkSelfAttentionFNc>t|j|_tj|j|_tj|j|_|j|_|j |_ |j|j z|_ ||_ |j|j zdkr td|jd|j dtj |jd|jz|j|_tj |j|j|j|_||_||_|rd|j}t)jt)j||ft.dd||}|d |dSdS) Nrz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).rbiasdtyperr))super__init__dropoutrDropout attn_dropout resid_dropout hidden_size embed_dim num_headshead_dimconfig ValueErrorLinearr)att_projout_proj is_causal layer_idx block_sizetorchtrilonesboolviewregister_buffer)selfr6r;r<r=r) __class__s d/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/models/bark/modeling_bark.pyr-zBarkSelfAttention.__init__Es ~ Jv~66Z77+)$.8    0 0A 5 5'dn''N'''   &"4a&:L6LSYS^___  &"4f6Hv{[[[ ""  /*J:ej*j)ANNNOOTTUVXY[egqrrD   . . . . . / /c|dd||fz}||}|ddddS)J Splits hidden_size dim into attn_head_size and num_heads Nrrrr)sizerBpermuterDtensorr4attn_head_size new_shapes rF _split_headszBarkSelfAttention._split_headsesMKKMM#2#&)^)DD Y''~~aAq)))rGc|dd}||dd||zfz}|S)S Merges attn_head_size dim and num_attn_heads dim into hidden_size rrN) transpose contiguousrBrKrDrNr4rOs rF _merge_headszBarkSelfAttention._merge_headsmsY!!!Q''2244V[[]]3B3/9~3M2OOPP rGcptj||dddtj|jz z}|jrz|d|d}}||j dddd||z |d|fdktj |j j }|||z}tj|d}||j }||}tj||}||fS)NrJrT?rdim)r>matmulrUmathsqrtr5r;rK masked_fillr)finfor+minrrsoftmaxtor0) rDquerykeyvalueattention_mask attn_weights query_length key_length attn_outputs rF_attnzBarkSelfAttention._attnys)|E3==R+@+@AAS49UYUbKcKcEcd > ',zz"~~sxx||*L(33 !!!QQQ \ 9J F  STXYY L.//3L  %'.8L},,\r,BB #u{33 ((66 l<77 L((rGc D|||jd\}}} |||j|j}|||j|j}|| |j|j} |||| |j\}} |||| |\} } | | |j|j} | | } | | } | | fS)Nrr[) r9splitr3rQr4r5updater<rmrXr:r1) rD hidden_statesrhpast_key_values use_cacheoutput_attentionskwargsrerfrgrlris rFforwardzBarkSelfAttention.forwards !MM-88>>t~ST>UUsE!!%GGT^T]CC!!%GG  &(//UDNKKJC$(JJuc5.$Q$Q! \'' T^T]SS mmK00 ((55 L((rGFNNNNFF) __name__ __module__ __qualname__r-rQrXrmrv __classcell__rEs@rFr&r&As//////@***   ))))< ))))))))rGr&c>eZdZdZfdZdZdZ ddZxZS) BarkSelfFlashAttention2aH Bark flash attention module. This module inherits from `BarkSelfAttention` as the weights of the module stays untouched. The only required change would be on the forward pass where it needs to correctly call the public API of flash attention and deal with padding tokens in case the input contains any of them. c`tj|i|t|_dSrx)r,r-r_flash_attn_uses_top_left_mask)rDargsrurEs rFr-z BarkSelfFlashAttention2.__init__s6$)&))) /P.Q.Q+++rGcr|dd||fz}||}|S)rINrJ)rKrBrMs rFrQz$BarkSelfFlashAttention2._split_headss<KKMM#2#&)^)DD Y'' rGcr||dd||zfz}|S)rSNrT)rBrKrWs rFrXz$BarkSelfFlashAttention2._merge_headss8 V[[]]3B3/9~3M2OOPP rGNFc |\}}} |||jd\} } } || |j|j} || |j|j} || |j|j} ||| | |j\} } t| |} t| | | |||j r|j nd|j |j|  }|||j|j}||}||}|dfS)Nrr[g)r.use_top_left_maskr; target_dtype)rKr9ror3rQr4r5rpr<r#r$trainingr.rr;rXr:r1)rDrqrhrrrsrtru batch_size query_len_rerfrgrrls rFrvzBarkSelfFlashAttention2.forwards\$1#5#5#7#7 Iq!MM-88>>t~ST>UUsE!!%GGT^T]CC!!%GG  &(//UDNKKJC't44 .     $(M:DLLs"An%    '' T^T]SS mmK00 ((55 D  rGry) rzr{r|__doc__r-rQrXrvr}r~s@rFrrs RRRRR '!'!'!'!'!'!'!'!rGr)eagerflash_attention_2c$eZdZfdZdZxZS)BarkMLPclttj|jd|jz|j|_tjd|jz|j|j|_tj|j |_ tj |_ dS)Nr() r,r-rr8r2r)in_projr:r/r.GELUgelurDr6rEs rFr-zBarkMLP.__init__s y!3Q9K5KRXR]^^^  !f&8"8&:LSYS^___ z&.11 GII rGc||}||}||}||}|Srx)rrr:r.)rDrqs rFrvzBarkMLP.forward sL ]33  -00  m44  ]33 rGrzr{r|r-rvr}r~s@rFrrsGrGrc0eZdZdfd Z ddZxZS) BarkBlockFNct|rKtj|j|j|_tj|j|j|_nV[QQQD !|F,>V[QQQD  !|F,>??D !|F,>??D *6+FG i9    6??rGc ||}||||||}|d} |dd} || z}||||z}|f| zS)Nrrrhrsrtrr)rrrr) rDrqrrrhrsrtruintermediary_hidden_states attn_outputsrloutputss rFrvzBarkBlock.forward%s&*%5%5m%D%D"yy &+)/ !  #1o qrr"%2[%@"%?$((   7 8 8C C & "+,w66rGrwryrr~s@rFrrs_######* 77777777rGrc\eZdZUeed<dZdZedej ffd Z fdZ xZ S)BarkPreTrainedModelr6FTreturnc@t|dstjS|D]S}t|drAt|jdr,|jj t j|jjcSTtjS) `torch.device`: The device on which the module is (assuming that all the module parameters are on the same device). _hf_hookexecution_device)hasattrr,devicemodulesrrr>rDmodulerEs rFrzBarkPreTrainedModel.deviceIstZ(( "77> !llnn F FF ++ FFO-?@@ FO4@|FO$DEEEEEww~rGc`t|t|trt|jro|jj}tjtj ||ft dd||}tj |j|dSdSdS)Nr*r)r, _init_weights isinstancer&r;r6r=r>r?r@rArBinitcopy_r))rDrr=r)rEs rFrz!BarkPreTrainedModel._init_weights]s f%%% f/ 0 0 . .#]5 z%*j*-ET"R"R"RSSXXYZ\]_ikuvv 6;-----  . . . .rG) rzr{r|r__annotations__supports_gradient_checkpointing_supports_flash_attnpropertyr>rrr}r~s@rFrrCs &+#  X&.........rGrc$eZdZUeed<dZfdZdZdZdZ e dde j dzd e dzd e j dzd e j dzd e jdzd e j dzdedzdedzdedzdedzdee j ezfdZxZS)BarkCausalModelr6)audioc`t|_tjjj|_tjjj|_ tj j |_ tj fdtjD|_tjjj|_tjjjd|_d|_|dS)Nc4g|]}td|S)Trr.0ir6s rF z,BarkCausalModel.__init__..us*$v$v$vXYYvQR%S%S%S$v$v$vrGr(F)r,r-r6r Embeddinginput_vocab_sizer2input_embeds_layerr=position_embeds_layerr/r.drop ModuleListrange num_layerslayersrr)layernorm_finalr8output_vocab_sizelm_headgradient_checkpointing post_initrs `rFr-zBarkCausalModel.__init__ks     #%,v/FHZ"["[%'\&2CVEW%X%X"Jv~.. m$v$v$v$v]bcict]u]u$v$v$vww !|F,>V[QQQy!3V5MTYZZZ &+# rGcdSrxrDs rFget_output_embeddingsz%BarkCausalModel.get_output_embeddingss trGc|jSrxrrs rFget_input_embeddingsz$BarkCausalModel.get_input_embeddingss &&rGc||_dSrxrrDnew_embeddingss rFset_input_embeddingsz$BarkCausalModel.set_input_embeddingss"0rGN input_idsrrrh position_idslabels inputs_embedsrsrtoutput_hidden_states return_dictrc  ||n |jj}| | n |jj} ||n |jj}| | n |jj} d} |t d||t d||n*|||}n|nt d|dd} | d}|j r%|j r|rt dd}|r|t|j}||nd}||jjj}|It'j|||zt&j|jjj }|d}||jjj}||}t/|j|| }|||z}| |dfz}|rd nd}| rd nd}t3|jD]3\}}| r||fz}|||||| }|d}|r ||d fz}4||}||}| r||fz}||}| st=dd||||fDSt?| ||||S)NzXTraining is not implemented yet for Bark - ensure you do not pass `labels` to the model.DYou cannot specify both input_ids and inputs_embeds at the same time5You have to specify either input_ids or inputs_embedsrJzZ`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...F)r6rr+rr6rrhrrrc3K|]}||V dSrxrrvs rF z*BarkCausalModel.forward..s1fgfsfsfsfsfsrG)losslogitsrrrq attentions) r6rtrrsrNotImplementedErrorr7rrKrrlogger warning_oncerget_seq_lengthrdrweightrr>arangelong unsqueezer r enumeraterrrBrtupler)rDrrrrhrrrrsrtrrrur input_shape seq_length past_lengthposition_embedsrq output_shapeall_self_attentionsall_hidden_statesrblockrrs rFrvzBarkCausalModel.forwards2C1N--TXT_Tq$8$D $+Jj "+!6IIDKcdd d  &?+B   " 33I>>MM  & TUU U#((**3B3/  _  & "4= " "##p"  ?0*$+>>>O:I:Uo44666[\ %(()C)J)QRR   <[(j18? L (11!44L#t'A'H'OPP 44\BB2;')     -/"ABB "m&8&8&<&<%>> $5?bb4"6@BBD!$+.. J JHAu# I$58H$H!e /-#"3 G$AJM  J&9WQZM&I#,,];; %**<88   E 1]4D D m,,  &/;LNab &++*     rG) NNNNNNNNNN)rzr{r|rroutput_modalitiesr-rrrrr>Tensorr LongTensorrArrrvr}r~s@rFrrgs "( '''111*.(,.2,0*.-1!%)-,0#'x x <$&x x  t+ x lT) x  4' x |d*x $;x  $;x #Tkx D[x  u| 5 5x x x ^x x x x x rGrz Bark semantic (or text) model. It shares the same architecture as the coarse model. It is a GPT-2 like autoregressive model with a language modeling head on top. ) custom_introc eZdZUdZeed< d dejdedzde e ejfdzdejdzdej f fd Z xZ S) BarkSemanticModelsemanticr6Nrsemantic_generation_confighistory_promptrhrc |td|jd}|j}||jz}|0|d|z |j}|L|d| d}tj |d|t|z f|j d}n-tj |f|j |jtj}tj|d|d }tj|jgg|ztj |j} tj||ddd|f||ddd|dzfz|| gd } t-t/|j|j } | t-t/|j dz|jjt9| |j } |d |j} t?|j | |j }tCj"tj#||dzftj|jf| | |g|d|}|ddr|j$dd|dzdf}n|dd|dzdf}|S)a Generates text semantic tokens from an input prompt and an additional optional `Bark` speaker prompt. Args: input_ids (`Optional[torch.Tensor]` of shape (batch_size, seq_len), *optional*): Input ids, i.e tokenized input sentences. Will be truncated up to semantic_generation_config.max_input_semantic_length tokens. Note that the output audios will be as long as the longest generation among the batch. semantic_generation_config (`BarkSemanticGenerationConfig`): Generation config indicating how to generate the semantic tokens. history_prompt (`Optional[dict[str,torch.Tensor]]`, *optional*): Optional `Bark` speaker prompt. attention_mask (`Optional[torch.Tensor]`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) Returns: torch.LongTensor: Output semantic tokens. N/`semantic_generation_config` has to be providedrrsemantic_promptconstant)rgmode)rr+r[r*r min_eos_p) eos_token_idrrr)rlogits_processorgeneration_configreturn_dict_in_generateF)%r7shapemax_input_semantic_lengthtext_encoding_offsetr`rAtext_pad_tokenrrpadlensemantic_pad_tokenr>fullrintrepeat_interleaverNsemantic_infer_tokenrdcatrlistrsemantic_vocab_sizeextendr6rr getrr rr,generater@ sequences)rDrr rrhrurrsemantic_history infer_arrayrtokens_to_suppress suppress_tokens_logits_processorrearly_stopping_logits_processorsemantic_outputrEs rFr*zBarkSemanticModel.generatesi< & -NOO O_Q' $>$X! : OO  %!--q>/A.G.G.I.IKeKtuuI  %-.?@B[A[A\A\] !}00 -4D0E0EEF0C 1     %z*,*={i     !23CD3I:[\]]]l(= > ?* LTYT]   "T[//  '' !!!5O6O5O2O(PQQ))*:111>]@Y\]@]>];]*^__`'' 44      " ,@B\Bo p p   !! 1DqH$+Jghh i i   ,II[dmdt+u+u+u(JJ{,F,PQQ *K3@I^g^n+ + + ' +%''* J $=$AB%)\`\g h h h '>@_`8       ::/ 7 7 R-7;TWX;X;Z;Z8Z[OO-aaa1JQ1N1P1P.PQOrGNNN)rzr{r|base_model_prefixrrr>rr"dictstrrr*r}r~s@rFr r s#  KO9=.2 ff<f%A4$GfS%,./$6 f  t+ f  ffffffffffrGr z Bark coarse acoustics model. It shares the same architecture as the semantic (or text) model. It is a GPT-2 like autoregressive model with a language modeling head on top. c eZdZUdZeed< ddededededed eee j fdzf d Z dd e j de dzd e dzded eee j fdzdedzde jee je jfzffd ZxZS)BarkCoarseModelcoarse_acousticsr6Nmax_coarse_historysemantic_to_coarse_ratiorr  codebook_sizerc |tj|dd|d}|d}|5td|jdD]} || ddfxx|| zz cc<tj|ddd}||jz}tj|d|d}ttj ||z } t| |jd|jddzz ttj |jd|z g} tt| |z} |dd| df}|dd| df}|dddd f}nTtj gg|ztj|j }tj gg|ztj|j }||fS) a Preprocess the optional `Bark` speaker prompts before `self.generate`. Args: max_coarse_history (`int`): Maximum size of coarse tokens used. semantic_to_coarse_ratio (`int`): Ratio of semantic to coarse frequency batch_size (`int`): Batch size, i.e the number of samples. semantic_generation_config (`BarkSemanticGenerationConfig`): Generation config indicating how to generate the semantic tokens. codebook_size (`int`): Codebook channel size, i.e. the size of the output vocabulary per codebook channel. history_prompt (`Optional[dict[str,torch.Tensor]]`): Optional `Bark` speaker prompt. Returns: Returns: `tuple(torch.FloatTensor)`: - **x_semantic_history** (`torch.FloatTensor` -- Processed semantic speaker prompt. - **x_coarse_history** (`torch.FloatTensor`) -- Processed coarse speaker prompt. Nrrr[ coarse_promptrrJrrTr)r>r#clonerrrUreshaper'r"npfloorrbroundrNr) rDr9r:rr r;rx_semantic_historyx_coarse_historynmax_semantic_historyn_semantic_hist_providedn_coarse_hist_provideds rFpreprocess_historiesz$BarkCoarseModel.preprocess_historiessQ<  %!&!8HY9Z[_9`blrs!t!t!t -o>DDFF (q"2"8";<<@@A$QT***ma.??**** %/?AFFNNrRR /2L2`` $67G7Mz_`aaa $'rx0BE]0]'^'^#_#_ '*(&,Q/2D2J12MPQ2QQ!1!7!:=U!UVVWW(( $&)/GJb/b)c)c%d%d "!3AAA8P7P7Q7Q4Q!R!V!V!X!X /4J3J3K3K0KLPPRR /3B37  "'rdZ.?uyY]Yd!e!e!e $|RD:,=UYW[Wbccc !#333rGr1coarse_generation_configreturn_output_lengthsrc |td|td|j}|j} |j} |||jk|j|j|jz |j z} ttj | | z } ||jk d} tj | | z|j z } tj| |j z } tj| }|jd}||| | |||\}}|jd}tj||g}ttj|| z }d}|jd}t-|D]}|tt|| z z}|ddtjd|| z gdf}|ddd|f}t/j|d||jdz fd|j}tj|tj|jgg|z|j |dd| dfg}t9|jd|j|}t=j|f|gtA| ||z |d |}|jd}|!d d r(tj||j"dd|dfg}n"tj||dd|dfg}|jd|z }~|dd|df}|r|| fS|S) aW Generates coarse acoustics tokens from input text semantic tokens and an additional optional `Bark` speaker prompt. Args: semantic_output (`torch.Tensor` of shape (batch_size, seq_len), *optional*): Input text semantic ids, i.e the output of `BarkSemanticModel.generate`. semantic_generation_config (`BarkSemanticGenerationConfig`): Generation config indicating how to generate the semantic tokens. coarse_generation_config (`BarkCoarseGenerationConfig`): Generation config indicating how to generate the coarse tokens. codebook_size (`int`, *optional*, defaults to 1024): Codebook channel size, i.e. the size of the output vocabulary per codebook channel. history_prompt (`Optional[dict[str,torch.Tensor]]`, *optional*): Optional `Bark` speaker prompt. return_output_lengths (`bool`, *optional*): Whether or not to return the output lengths. Useful when batching. Returns: By default: torch.LongTensor: Output coarse acoustics tokens. If `return_output_lengths=True`: `Tuple(torch.Tensor, torch.Tensor): The output coarse acoustics tokens, and the length of each sample of the batch. Nr-`coarse_generation_config` has to be providedrr)rr9r:rr r;rJrr)rmax_new_tokensrrF)#r7max_coarse_input_lengthr9sliding_window_len masked_fill_r coarse_semantic_pad_tokencoarse_rate_hzsemantic_rate_hzn_coarse_codebooksr"r@rAsumr>rBmaxitemrrIhstackceilrFrrNcoarse_infer_tokenrr r'r,r*rbr)r+)rDr1r rKr;rrLrurPr9rQr:rFoutput_lengthsmax_generated_lenrrCx_coarsebase_semantic_idxn_window_stepstotal_generated_lenlen_coarse_historyr semantic_idx input_coarsealternatingLogitsProcessor output_coarseinput_coarse_len coarse_outputrEs rFr*zBarkCoarseModel.generates$F & -NOO O # +LMM M":"R5H5H $$ 9L L $ >   % 3(9 :&9 : ! #28,>AY,Y#Z#Z[[)-E-__ddefgg 5 58P8c c  ^6N6a%abbffhh!In55::<<$*1- '+'@'@)1%=!'A' (A( ( $H/4Q7,(:O'LMMRW%69K%KLLMM%^A.~&&+ + A,s59LOg9g3h3h/i/iiL+111bfaH\9\5].^.^.`.`+`aL'+C,C+C(CDL5+l.@.DDE(B L!< L#;#N"O!PS]!]fjfqrrrQQQ!3 3 4 445L*M"1%*>** & -EGG,"rrIr"r rArrr*r}r~s@rFr7r7ysp+ :>H4H4H4#&H4 H4 %( H4  H4S%,./$6H4H4H4H4ZKOFJ!9=-1III%A4$GI#=t"C I  I S%,./$6 I $d{I  E%"2E4D"DE EIIIIIIIIIIrGr7z Bark fine acoustics model. It is a non-causal GPT-like model with `config.n_codes_total` embedding layers and language modeling heads, one for each codebook. ceZdZUdZeed<dZfdZdZdZ dZ dZ d"d Z d#d e d zd e d zdedejfdZe d$de dejd zdejd zdejd zdejd zdejd zded zded zded zdeejezfdZej d%dejded zded zdede d eeejfd zdejfd!ZxZ S)& BarkFineModelfine_acousticsr6 codebook_idxcHt|_i|_t |jj|jjz D]}d|dzd|jd|d<tjfdt jD|_ tj j j |_ tjj|_tjfdt jD|_tjj |_tjfdt jjD|_d|_j|_|dS) Nzinput_embeds_layers.rz.weightz lm_heads.cNg|]!}tjjj"Sr)rrrr2rrr6s rFrz*BarkFineModel.__init__..ps+ l l l1R\&163E F F l l lrGc4g|]}td|S)Frrrs rFrz*BarkFineModel.__init__..ws( _ _ _Yv! < < < _ _ _rGcRg|]#}tjjjd$S)Fr()rr8r2rrqs rFrz*BarkFineModel.__init__..}s@    &,f.FUSSS   rGF)r,r-r6_tied_weights_keysr n_codes_total n_codes_givenrrinput_embeds_layersrr=r2rr/r.rrrrrlm_headsrr)rDr6rrEs ` rFr-zBarkFineModel.__init__es     "$t{04;3LLMM d dA>cUVYZUZ>c>c>cD #$:$:$:$: ; ;$&= l l l lPUV\VjPkPk l l l$ $  &(\&2CVEW%X%X"Jv~.. m _ _ _ _eFL]F^F^ _ _ _    "|F,>??     v3V5IJJ      ',##1 rGc|jSrxrwrs rFrz"BarkFineModel.get_input_embeddingss ''rGc||_dSrxrzrs rFrz"BarkFineModel.set_input_embeddingss#1   rGc|jSrxrxrs rFrz#BarkFineModel.get_output_embeddingss }rGc||_dSrxr})rDnew_output_embeddingss rFset_output_embeddingsz#BarkFineModel.set_output_embeddingss - rGNTc}tjfd|D}||djjdVjjsJ}tjfd|D} |S)Nc@g|]}|Sr)_get_resized_embeddings)rold_embeddings mean_resizingnew_num_tokenspad_to_multiple_ofrDs rFrz:BarkFineModel._resize_token_embeddings..s?   ",,^^M_anoo   rGrc<g|]}|Sr)_get_resized_lm_head)r old_lm_headrrDs rFrz:BarkFineModel._resize_token_embeddings..s)lllK**;GGlllrG) rrrrrrrr6tie_word_embeddingsr)rDrrrold_embeddings_listnew_embeddings_listold_lm_head_listnew_lm_head_lists```` rF_resize_token_embeddingsz&BarkFineModel._resize_token_embeddingss"7799 m       &9      !!"5666,Q/6= 7.5` (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128. For more details about this, or help on choosing the correct value for resizing, refer to this guide: https://docs.nvidia.com/deeplearning/performance/dl-performance-matrix-multiplication/index.html#requirements-tc mean_resizing (`bool`): Whether to initialize the added embeddings from a multivariate normal distribution that has old embeddings' mean and covariance or to initialize them with a normal distribution that has a mean of zero and std equals `config.initializer_range`. Setting `mean_resizing` to `True` is useful when increasing the size of the embeddings of causal language models, where the generated tokens' probabilities won't be affected by the added embeddings because initializing the new embeddings with the old embeddings' mean will reduce the kl-divergence between the next token probability before and after adding the new embeddings. Refer to this article for more information: https://nlp.stanford.edu/~johnhew/vocab-expansion.html Return: `torch.nn.Embedding`: Pointer to the input tokens Embeddings Module of the model. Nr)rrrr6r vocab_size tie_weights)rDrrr model_embedss rFresize_token_embeddingsz%BarkFineModel.resize_token_embeddingssF44^EWYfgg  !&8&@ )5Q(>(DQ(G %!-a!7!=a!@ !-a!7!=a!@&q/06q9 rGrrhrrrrtrrc  ||n |jj}||n |jj}| | n |jj} d} |t d|dkrt d|t d|t dafdt |jD}tj |d }|ddddddd|d zf d }| dd} | d } | |j jj}|Ftjd| tj|j jj }|d}| |j jj}| |}t'|j|| }|||z}| | dfz}|rd nd}|rd nd}t |jD]1\}}|r||fz}||||}|d}|r ||d fz}2||}||}|r||fz}|j||jjz |}| st5dd|||fDSt7| |||S)z codebook_idx (`int`): Index of the codebook that will be predicted. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): NOT IMPLEMENTED YET. NzTraining is not implemented yetrzRCannot predict 0th codebook - 0th codebook should be predicted by the coarse modelrrclg|]0\}}|dddd|fd1S)NrJr)rrrrs rFrz)BarkFineModel.forward.. sZ)A)#"9QQQ1W#566@@DDrGrJr[rrrr)rhrtc3K|]}||V dSrxrrs rFrz(BarkFineModel.forward..Es(llq^_^k^k^k^k^kllrG)rrrqr)r6rtrrrr7rrwr>r%rWrKrdrrrrrrr rrrrBrxrvrr)rDrnrrhrrrrtrrrurrrrrqrrrrrrrs ` rFrvzBarkFineModel.forwards(2C1N--TXT_Tq$8$D $+Jj &1%cdd d  !6TUU U  -6t7O-P-PM"Im<<> $5?bb4"6@BBD!$+.. J JHAu# I$58H$H!e-"3G $AJM  J&9WQZM&I#,,];; %**<88   E 1]4D D H|dk.GGHWW mllT63DFY$Zlllll l+*     rGrJrjr rKfine_generation_configr;rc |td|td|td|d|j}|j} |j} ||jdd|j}tj ||j z |}|jd} |)tj |dj d| d } nd} |j} tj|d|j| z fd |}| Ftj| dd| dddf|gd }| dd| dddfjd }nd}d}|jd | kr,| |jd z }tj|ddd|fd | }|jd | |z z | z }t#t%j|}t)d|d z}t+|D]}t-|| z|jd | z g}t-||| zz|jd | z g}||z }|dd||| zddf}t+| |jD]}|||j}||d kr'|dd|dd|f}tj|d}nw|ddddd|f|z }tj|d dd|| f}|d|f}tj|d | d}|tj}||dd|d|f<~~t+| |jD]!}|dd|d|f|dd||| |z z|f<"~|d ddddd|df}|dkr|ddddd| f}|jd|jdkrtd|S)ap Generates fine acoustics tokens from input coarse acoustics tokens and an additional optional `Bark` speaker prompt. Args: coarse_output (`torch.Tensor` of shape (batch_size, seq_len)): Input coarse acoustics ids, i.e the output of `BarkCoarseModel.generate`. semantic_generation_config (`BarkSemanticGenerationConfig`): Generation config indicating how to generate the semantic tokens. coarse_generation_config (`BarkCoarseGenerationConfig`): Generation config indicating how to generate the coarse tokens. fine_generation_config (`BarkFineGenerationConfig`): Generation config indicating how to generate the fine tokens. codebook_size (`int`, *optional*, defaults to 1024): Codebook channel size, i.e. the size of the output vocabulary per codebook channel. history_prompt (`Optional[dict[str,torch.Tensor]]`, *optional*): Optional `Bark` speaker prompt. Returns: torch.LongTensor: Output fine acoustics tokens. NrrNz+`fine_generation_config` has to be provided temperaturerrJ fine_promptr[rr)rrgrZ) num_samplesrrTz-input and output should have the same seq_len) r7r)rmax_fine_history_lengthmax_fine_input_lengthrBrrVr> remainderr'r#Tr\rn_fine_codebooksr%r"r@r[rXrrbrvrargmaxrcr? multinomialrdint32rU)rDrjr rKrr;rrurrrrx_fine_historyn_coarse fine_input n_historyn_remove_from_endn_loopsn_outer start_idxstart_fill_idxrel_start_fill_idx input_buffern_innerrrelevant_logitscodebook_predsprobss rFr*zBarkFineModel.generateNs> & -NOO O # +LMM M ! )JKK K jj0F0RSS "8"P 6 L&**=+>q+A2G_Grss  8R8f(fhuvv "(+  %"4^M5R5TUY5Z\flmnnnNN"N+>U  &7(B C       %N1117N6N6O6OQRQRQR3R$SU_#`fghhhJ'qqq+B*B*C*CQQQ'FGMaPIII  A !6 6 6 5 8H8K K zAq!5F+Gj`mnnnJ!&q)-BY-NOSjjbgg&&''a//A%W~~  GW'>> @PQR@SVk@klmmI W'>>> @PQR@SVm@mnN"0)!; %aaaYAV5V)VXYXYXY&YZL +A+RSS + +g|<<C&+*<*<&,QQQ0B0C0C^m^-S&TO%*\/2%F%FNN&,QQQ>M>-A&B[&POIo2>>>qqqBTUjBj?jkE!MM2}*=>>E%*%6u!%L%L%L%Q%QR\^`%a%aN!/!2!25;!?!?@N QQQ 2 3 3W<=NN!+A+RSS B B!$6$7$7!@AAA~:ORd:d(eegnn ))!Q//111ijj0@A q #AAAqqq*=,=+=*=$=>J  B =#6r#: : :LMM MrGNT)NNT)NNNNNNNN)NNNrJN)!rzr{r|r3rrmain_input_namer-rrrrrr"rArrrrr>rrrrrvno_gradr"r r!r4r5r*r}r~s@rFrlrlZs) $O!!!!!F(((222...++++.&*)-" 00d 0 $J0 0  0000d*..2,0*.-1)-,0#'l l l <$&l  t+ l lT) l  4' l |d*l  $;l #Tkl D[l  u| ~ -l l l ^l \U]__KOFJ;?!9=FF|F%A4$GF#=t"C F !9 F  FS%,./$6F  FFF_FFFFFrGrla7 The full Bark model, a text-to-speech model composed of 4 sub-models: - [`BarkSemanticModel`] (also referred to as the 'text' model): a causal auto-regressive transformer model that takes as input tokenized text, and predicts semantic text tokens that capture the meaning of the text. - [`BarkCoarseModel`] (also referred to as the 'coarse acoustics' model), also a causal autoregressive transformer, that takes into input the results of the last model. It aims at regressing the first two audio codebooks necessary to `encodec`. - [`BarkFineModel`] (the 'fine acoustics' model), this time a non-causal autoencoder transformer, which iteratively predicts the last codebooks based on the sum of the previous codebooks embeddings. - having predicted all the codebook channels from the [`EncodecModel`], Bark uses it to decode the output audio array. It should be noted that each of the first three modules can support conditional speaker embeddings to condition the output sound according to specific predefined voice. c eZdZUeed<fdZedefdZe de j ffd Z dde dzfd Z dd Ze j dd e jdzd eee jfdzd edzde jfdZxZS) BarkModelr6cRt|t|j|_t |j|_t|j |_ tj |j |_||_|dSrx)r,r-r semantic_configr r7coarse_acoustics_configr8rlfine_acoustics_configrmr from_config codec_config codec_modelr6rrs rFr-zBarkModel.__init__s    )&*@AA /0N O O+F,HII$01DEE  rGrcdSrr)clss rF can_generatezBarkModel.can_generates trGc2t|jdstjS|jD]S}t|drAt|jdr,|jj tj|jjcSTdS)rrrN)rr r,rrrrr>rs rFrzBarkModel.devicest}j11 "77> !m++-- F FF ++ FFO-?@@ FO4@|FO$DEEEEE  F FrGraccelerator_idNc dtrddlm}ntdd}t r#t jj}t j |d|}tt |}|j jdkr)| d| ||j j|\|j _}d}|j |j|jfD]} || ||\}}||_||j||\}}||_dS) a Offloads all sub-models to CPU using accelerate, reducing memory usage with a low impact on performance. This method moves one whole sub-model at a time to the accelerator when it is used, and the sub-model remains in accelerator until the next sub-model runs. Args: accelerator_id (`int`, *optional*, defaults to 0): accelerator id on which the sub-models will be loaded and offloaded. r)cpu_offload_with_hookz1`enable_model_cpu_offload` requires `accelerate`.cuda:cpuN)prev_module_hook)r accelerater ImportErrorrr> acceleratorcurrent_acceleratortypergetattrrd empty_cacher rr8rmfine_acoustics_hookrcodec_model_hook) rDrrur device_typertorch_accelerator_modulerhookcpu_offloaded_models rFenable_cpu_offloadzBarkModel.enable_cpu_offloadsg # $ $ S 8 8 8 8 8 8 8QRR R ) + + G+??AAFK??~??@@#*5+#>#> ; u $ $ GGENNN $ 0 0 2 2 2/D.CDMDdfl.m.m+ (! M  !  $  ` `  ,+,?Z^___GAtt#' ''(8&SWXXX4!%rGc&|dd}jj|}|)dt ||D}fd|D}n/j|}|d}|S)z:Turn quantized audio codes into audio array using encodec.rrNcVg|]&\}}|ddd|fd'S)Nrr)rsamplels rFrz*BarkModel.codec_decode..Ls:ZZZ+616!!!RaR%=**1--ZZZrGchg|].}j|/Sr)rdecodersqueeze)rrrDs rFrz*BarkModel.codec_decode..Ms6VVV)11&99AACCVVVrG)rUr quantizerdecodeziprr)rD fine_outputr^embout audio_arrs` rF codec_decodezBarkModel.codec_decodeBs"++Aq11 (// <<  %[ZS.AYAYZZZCVVVVRUVVVII"**3//C AIrGrrrLc tdi|jj}tdi|jj}t di|jj}|dd|ddd}i} i} |D]\} } | dr| tdd} | || <7| dr| tdd} | | | <i| dr| tdd} | | | <| |vr| || <| | vr| | | <| | vr| | | <d|vr|d|j j |f||d |} d| vr| d|j j | f||||jj|d | }d}|r|\}}||jz}d| vr| d|jj |f|||||jjd | }t#|d d=|j|j|j|_|||}t#|d d|j|r7d|D}t2jj|dd}||fS|S)a^ Generates audio from an input prompt and an additional optional `Bark` speaker prompt. Args: input_ids (`Optional[torch.Tensor]` of shape (batch_size, seq_len), *optional*): Input ids. Will be truncated up to 256 tokens. Note that the output audios will be as long as the longest generation among the batch. history_prompt (`Optional[dict[str,torch.Tensor]]`, *optional*): Optional `Bark` speaker prompt. Note that for now, this model takes only one speaker prompt per batch. kwargs (*optional*): Remaining dictionary of keyword arguments. Keyword arguments are of two types: - Without a prefix, they will be entered as `**kwargs` for the `generate` method of each sub-model. - With a *semantic_*, *coarse_*, *fine_* prefix, they will be input for the `generate` method of the semantic, coarse and fine respectively. It has the priority over the keywords without a prefix. This means you can, for example, specify a generation strategy for all sub-models except one. return_output_lengths (`bool`, *optional*): Whether or not to return the waveform lengths. Useful when batching. Returns: By default: - **audio_waveform** (`torch.Tensor` of shape (batch_size, seq_len)): Generated audio waveform. When `return_output_lengths=True`: Returns a tuple made of: - **audio_waveform** (`torch.Tensor` of shape (batch_size, seq_len)): Generated audio waveform. - **output_lengths** (`torch.Tensor` of shape (batch_size)): The length of each waveform in the batch Example: ```python >>> from transformers import AutoProcessor, BarkModel >>> processor = AutoProcessor.from_pretrained("suno/bark-small") >>> model = BarkModel.from_pretrained("suno/bark-small") >>> # To add a voice preset, you can pass `voice_preset` to `BarkProcessor.__call__(...)` >>> voice_preset = "v2/en_speaker_6" >>> inputs = processor("Hello, my dog is cute, I need him in my life", voice_preset=voice_preset) >>> audio_array = model.generate(**inputs, semantic_max_new_tokens=100) >>> audio_array = audio_array.cpu().numpy().squeeze() ``` rhNr)rhr semantic_coarse_fine_r)rr )rr rKr;rL)rr rKrr;rrc,g|]}t|Sr)r)rrs rFrz&BarkModel.generate..s>>>fc&kk>>>rGTr) batch_first padding_valuer)r"rrr rr!rpopitems startswithrr r*r8r;rVrmrroffloadrrdrrrrutilsrnn pad_sequence)rDrrrLrur rKrkwargs_semantic kwargs_coarse kwargs_finerfrgr1rjr^outputrs rFr*zBarkModel.generateTsh&B%k%kDDZDj%k%k"#=#o#o@V@n#o#o !9!i!iD"7       [,9 )M>+/G/ZZN + - - OO/ 0 0 0-$-  )'A%=#90>      4. 5 5 A  $ , , . . .#/224;??D !!&.99 4+T 2 2 >  ! ) ) + + +  )>>>>>NHL--eUV-WWE.( ( rG)rrxr2)rzr{r|rrr- classmethodrArrr>rr"rrrrr4r5rr*r}r~s@rFrrsr&      T[F FFFFFXF&&',%,%d ,%,%,%,%\$U]__*.9=-1 OO<$&OS%,./$6O $d{ O   OOO_OOOOOrGr)rlr r7rrr)Err^numpyr@r>rtorch.nnrr\rr cache_utilsrr generationr generation.logits_processr r r masking_utilsr modeling_flash_attention_utilsrrmodeling_layersrmodeling_outputsrrmodeling_utilsrrrrrrautorconfiguration_barkrrrrrgeneration_configuration_barkr r!r"integrations.flash_attentionr#r$ get_loggerrzrModuler&rrrrrrr r7rlr__all__rrGrFrs  $$$$$$&&&&&&........)))))) 766666hhhhhhhh999999FFFFFFFF------ K@@@@@@JJJJJJ  H % %l)l)l)l)l) l)l)l)^I!I!I!I!I!/I!I!I!Z0     bi    .7.7.7.7.7*.7.7.7b...../...F\ \ \ \ \ )?\ \ \ ~ jjjjjjj  jZWWWWWoWWWt uuuuu'uu  up $zzzzz#_zz%$zz   rG