tj3 d dl mZmZmZ ddlmZ ddlmZ ddlm Z ddl
m Z m Z ddl
mZ e rd d lZdd
lmZ dd lmZ d Z G d
ded Z G d de Zd S ) )Any TypedDictoverload )
AudioInput)GenerationConfig)is_torch_available)ChatChatType )PipelineN)%MODEL_FOR_TEXT_TO_SPECTROGRAM_MAPPING)SpeechT5HifiGanzmicrosoft/speecht5_hifiganc ( e Zd ZU dZeed< eed< dS ) AudioOutputz
audio (`AudioInput`):
The generated audio waveform.
sampling_rate (`int`):
The sampling rate of the generated audio waveform.
audio
sampling_rateN)__name__
__module__ __qualname____doc__r __annotations__int b/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/pipelines/text_to_audio.pyr r ! s6 r r F)totalc @ e Zd ZdZdZdZdZdZdZ e d Z
ddd fd
Z d Z d
Z
ed ed ed
efd Zed ee d ed
ee fd Zed ed ed
efd Zed ee d ed
ee fd Z fdZ ddZd Z xZS )TextToAudioPipelinea
Text-to-audio generation pipeline using any `AutoModelForTextToWaveform` or `AutoModelForTextToSpectrogram`. This
pipeline generates an audio file from an input text and optional other conditional inputs.
Unless the model you're using explicitly sets these generation parameters in its configuration files
(`generation_config.json`), the following default values will be used:
- max_new_tokens: 256
Example:
```python
>>> from transformers import pipeline
>>> pipe = pipeline(model="suno/bark-small")
>>> output = pipe("Hey it's HuggingFace on the phone!")
>>> audio = output["audio"]
>>> sampling_rate = output["sampling_rate"]
```
Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial)
You can specify parameters passed to the model by using [`TextToAudioPipeline.__call__.forward_params`] or
[`TextToAudioPipeline.__call__.generate_kwargs`].
Example:
```python
>>> from transformers import pipeline
>>> music_generator = pipeline(task="text-to-audio", model="facebook/musicgen-small")
>>> # diversify the music generation by adding randomness with a high temperature and set a maximum music length
>>> generate_kwargs = {
... "do_sample": True,
... "temperature": 0.7,
... "max_new_tokens": 35,
... }
>>> outputs = music_generator("Techno music with high melodic riffs", generate_kwargs=generate_kwargs)
```
This pipeline can currently be loaded from [`pipeline`] using the following task identifiers: `"text-to-speech"` or
`"text-to-audio"`.
See the list of available models on [huggingface.co/models](https://huggingface.co/models?filter=text-to-speech).
TNF )max_new_tokens)vocoderr c t j |i | d | _ | j j t j v r?|6t j t
| j j n|| _ | j j j
dv rd | _ || _ | j | j j j | _ | j | j j }| j j dd }|C| d | D dD ]M}t+ ||d }||| _ t+ |dd t+ |j |d }||| _ N| j 4| j /t/ | j d r | j j j | _ d S d S d S d S )N)musicgenspeecht5generation_configc i | ]
\ }}||| S Nr ).0kvs r
z0TextToAudioPipeline.__init__.. s$ ^ ^ ^1PQP]q!P]P]P]r ) sample_rater codec_configfeature_extractor)super__init__r" model __class__r valuesr from_pretrainedDEFAULT_VOCODER_IDtodeviceconfig
model_type processorr __dict__getupdateto_dictitemsgetattrr. hasattrr/ ) selfr" r argskwargsr9
gen_configsampling_rate_namer3 s r r1 z TextToAudioPipeline.__init__m s $)&))) : #H#O#Q#Q Q Q ? /0BCCFFtzGXYYY
L : '+C C C !DN * < #!% !4!BD % Z&F ,001DdKKJ%
^ ^
0B0B0D0D0J0J0L0L ^ ^ ^___&F
;
;" '0BD I I
,)6D&& V^T::F$+F,?ASUY$Z$ZM$0-:* %$.*DQUQ_atIuIu*D!% !A!OD & %*D*D*D*Dr c L t |t r|g}| j j j dk rPd}t | j d rt | j j dd }|dddd}| | |}| j
| j
n| j }t |t r |j
|j fddd|}ne| j j j d k r"d
|D }| d d | j j j d k r d
|D } ||fi |ddi}|S )Nbarkr semantic_configmax_input_semantic_lengthFT)
max_lengthadd_special_tokensreturn_attention_maskreturn_token_type_ids)tokenize return_dictcsmc F g | ] }| d sd| n| S )[z[0]
startswithr) ts r
z2TextToAudioPipeline.preprocess.. s3 PPPa c):):A a PPPr rM diac F g | ] }| d sd| n| S )rT z[S1] rU rW s r rY z2TextToAudioPipeline.preprocess.. s3 RRR1<<+<+<C !RRRr return_tensorspt)
isinstancestrr2 r9 r: rB r& rA rJ r> r; tokenizerr
apply_chat_templatemessages
setdefault)rC textrE rL
new_kwargs preprocessoroutputs r
preprocessz TextToAudioPipeline.preprocess s dC 6D : '6 1 1 Jt-/@AA
o $T%;%KMhjm n n
(&+)-). J
f % % % F)- )Ct~~ dD ! ! G5\5
FF z +u44PP4PPP!!"6===z +u44RRTRRR!\$FF&FFFFFF
r c f | || j }|d }|d }| j r| || j }d|vr
| j |d< | | | ddi | j j j dv r d|vrd|d< | j j d i ||}nHt | r$t d | | j d i ||d
}| j | | }|S ) N)r8 forward_paramsgenerate_kwargsr& return_dict_in_generateT)rR output_audiozYou're using the `TextToAudioPipeline` with a forward-only model, but `generate_kwargs` is non empty. For forward-only TTA models, please use `forward_params` instead of `generate_kwargs`. For reference, the `generate_kwargs` used here are: r r )
_ensure_tensor_on_devicer8 r2 can_generater& r> r9 r: generatelen
ValueErrorkeysr" )rC model_inputsrE rj rk rg s r _forwardz TextToAudioPipeline._forward s ..vdk.JJ 01 !23 : " " $ $ E " ; ;OTXT_ ; ` `O #/997;7M 34
! !/ 2 2 2
! !#2(TZ(JJ<J>JJFF?##
dKZK_K_KaKad d
TZAA,A.AA!DF < #\\&))F
r text_inputsrj returnc d S r( r rC rv rj s r __call__z TextToAudioPipeline.__call__ s PSPSr c d S r( r ry s r rz z TextToAudioPipeline.__call__ s \_\_r c d S r( r ry s r rz z TextToAudioPipeline.__call__ s UXUXr c d S r( r ry s r rz z TextToAudioPipeline.__call__ s adadr c 8 t j |fi |S )aL
Generates speech/audio from the inputs. See the [`TextToAudioPipeline`] documentation for more information.
Args:
text_inputs (`str`, `list[str]`, `ChatType`, or `list[ChatType]`):
One or several texts to generate. If strings or a list of string are passed, this pipeline will
generate the corresponding text. Alternatively, a "chat", in the form of a list of dicts with "role"
and "content" keys, can be passed, or a list of such chats. When chats are passed, the model's chat
template will be used to format them before passing them to the model.
forward_params (`dict`, *optional*):
Parameters passed to the model generation/forward method. `forward_params` are always passed to the
underlying model.
generate_kwargs (`dict`, *optional*):
The dictionary of ad-hoc parametrization of `generate_config` to be used for the generation call. For a
complete overview of generate, check the [following
guide](https://huggingface.co/docs/transformers/en/main_classes/text_generation). `generate_kwargs` are
only passed to the underlying model if the latter is a generative model.
Return:
`AudioOutput` or a list of `AudioOutput`, which is a `TypedDict` with two keys:
- **audio** (`np.ndarray` of shape `(nb_channels, audio_length)`) -- The generated audio waveform.
- **sampling_rate** (`int`) -- The sampling rate of the generated audio waveform.
)r0 rz )rC rv rj r3 s r rz z TextToAudioPipeline.__call__ s$ 2 uww >>~>>>r c t | dd
| j |d< t | dd | j |d< | j |d< |r|ni |r|ni d}|i }i }|||fS )Nassistant_modelassistant_tokenizerr` )rj rk )rA r r` r )rC preprocess_paramsrj rk paramspostprocess_paramss r _sanitize_parametersz(TextToAudioPipeline._sanitize_parameters s 4*D 1 1 =151EO - . 4. 5 5 A+/>OK (595MO 1 2 1? FnnB2A Ir
$ " &*<<.4 sC ^^^RTRUU%u{U;;AACCKKMM^^^r r r r )r r )r^ dicttupler; decodelistrq r7 r r r r r r )rC r needs_decodings r postprocessz TextToAudioPipeline.postprocess% s eT " " % g!% k*
u
%
% !HE 1dn 8N))%00E eT " " P^^X]^^^E ZZ!^^EEqEEHHEH==CCEEMMOOE ,
r )NNN)r r r r _pipeline_calls_generate_load_processor_load_image_processor_load_feature_extractor_load_tokenizerr _default_generation_configr1 rh ru r r_ r r rz r r r r
__classcell__)r3 s @r r r - s 2 2h $ O ! #O "2!1" " " '+$ &P &P &P &P &P &P &PP& & &P( ( (T SCS3S;SSS XS
_DI__kIZ___ X_
XHXX XXX XX
dDNdcddS^N_ddd Xd? ? ? ? ?: = = = =.
r r )typingr r r audio_utilsr
generationr utilsr utils.chat_template_utilsr
r baser
r models.auto.modeling_autor !models.speecht5.modeling_speecht5r r6 r r r r r r s; , + + + + + + + + + $ $ $ $ $ $ ) ) ) ) ) ) & & & & & & 6 6 6 6 6 6 6 6 DLLLQQQQQQCCCCCC1 )5 O
O
O
O
O
( O
O
O
O
O
r