tjg3ddlZddlmZmZddlmZddlmZm Z m Z ddl m Z ddl mZmZmZerddlZejeZGdd ed ZeGd d e Zd gZdS)N) AudioInput make_list_of_audio_chat_template) BatchFeature)ProcessingKwargsProcessorMixinUnpack) TextInput)auto_docstringis_torch_availableloggingc,eZdZddidddddddd Zd S) GlmAsrProcessorKwargspaddingTi> max_length) sampling_ratereturn_attention_maskrptleft)return_tensors padding_side) text_kwargs audio_kwargs common_kwargsN)__name__ __module__ __qualname__ _defaultsj/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/models/glmasr/processing_glmasr.pyrr&sM t #%)#  #"    IIIr rF)totalc eZdZeZ dfd Ze ddeeezde dzd e dzd e ed e f fd Z dde dzdeeezdzd e effd Zd dZde fdZdeded efdZed eeffd Zed eefdZ d!deeeze zdeeezdzd e ed e fdZdddZdZded efdZxZS)"GlmAsrProcessorN<|pad|>&Please transcribe this audio into textc||_|||_||_||_t |||dS)a audio_token (`Optional[str]`, *optional*, defaults to `"<|pad|>`"): Special token used to represent audio inputs in the chat template. default_transcription_prompt (`str`, *optional*, defaults to `"Please transcribe this audio into text"`): Default prompt to use for transcription tasks when applying transcription requests. max_audio_len (`int`, *optional*, defaults to 655): Maximum length of audio sequences in seconds. Audio longer than this will be truncated. 655 gives approximately 8192 tokens, corresponding to the maximum sequence length of the text model. ) chat_templateN) audio_tokenconvert_tokens_to_idsaudio_token_iddefault_transcription_prompt max_audio_lensuper__init__)selffeature_extractor tokenizerr)r*r-r. __class__s r!r0zGlmAsrProcessor.__init__;sY$''==kJJ,H)* *I]SSSSSr Ftextaudio output_labelskwargsreturnc hd|vr(|ddkrt|jjd|rd|d<tjd ||d|}|rP|d}|d}d ||d k<d |||jjk<||d <t|d S)a0 output_labels (bool, *optional*, default=False): Whether to return labels for training. Returns: [`BatchFeature`]: A dictionary with tokenized text (`input_ids`, `attention_mask`) and audio features (`input_features`, `input_features_mask`). rrz% only supports `return_tensors='pt'`.Treturn_mm_token_type_idsr6r5mm_token_type_ids input_idsirlabels)data tensor_typer) ValueErrorr4rr/__call__popcloner3 pad_token_idr) r1r5r6r7r8 model_inputsr=r?r4s r!rCzGlmAsrProcessor.__call__Ss" v % %&1A*Bd*J*J 7^^^__ _  615F- .'uww'Ie$II&II  , , 0 01D E E !+.4466F-1F$) *<@F6T^88 9%+L "4@@@@r c tjd||d||R|Rt|t|kr4tdt|dt|ddSdSdS)Nr<zGot z text but z audios; they must match 1:1.r)r/validate_inputslenrB)r1r6r5r8r4s r!rIzGlmAsrProcessor.validate_inputsss Ae$AA&AAA   1c$ii3u::6M6MbCIIbbUbbbcc c   1 16M6Mr audio_lengths torch.Tensorc`d}dD]\}}}|d|zz|dz z dz |zdz}||z |zdz}|S)N))rrO)rOrrPrOr)r1rK merge_factorr kernel_sizestride num_tokenss r!_get_audio_token_lengthz'GlmAsrProcessor._get_audio_token_length~sc ,B ` ` (G[&*Q[8K!OLqPU[[^__MM#l2|CaG r c Jt|d|jjz}t|j|jjz}g}g}|D]}t|jd}t d||zdz |z} | |kr:t d||dz dd|jd|jd|} || t|| |z} t| D]:} | |z} t| dz|z| } ||| | ;|j |}|j|fi|}| d |d <tjd tj|d d |D}|||d <g}tt%|D].}|||}||/||fS)NrrrOzAudio duration (z.1fz s) exceeds zs; truncating to first zs.attention_maskinput_features_maskc6g|]}|Sr)sum).0ss r! z2GlmAsrProcessor._process_audio..s k k kQUUWW k k kr num_audio_tokens) audio_idx)intr2 chunk_lengthr.shapemaxloggerwarningappendminrange fetch_audiorDtorchstacksplitrZrUrJreplace_audio_token)r1r6r8 window_size max_windowsper_sample_windows flat_chunksaudio_el n_samplesn_wintime_capistartend audio_inputsrKaudio_replacementsidxreplacement_texts r!_process_audiozGlmAsrProcessor._process_audios&1D4J4WWXX $,0F0SSTT (*(*  8 8HHN1-..II 3a7KGHHE{""]y6/3J'J]]][_[m]]GKGY]]]$  % %e , , ,9ek&9::H5\\ 8 8K1q5K/::""8E#I#67777 8 &22599-t-kDDVDD .:.>.>?O.P.P *+  k kek,7L*M*Q*QRT*U*UWijj k k k  ,0+G+G +V+V '(U$$ 8 8C#77 PS7TT   % %&6 7 7 7 7///r rzr`c2|d|}|j|zS)Nr_)r*)r1rzr`r_s r!rnz#GlmAsrProcessor.replace_audio_tokens"'(:;IF"222r c2tjdgzS)NrX)r/model_input_names)r1r4s r!rz!GlmAsrProcessor.model_input_namessww(,A+BBBr cdgS)zNInput names returned always by subprocessors but not used in model's `forward`r_r)r1s r!unused_input_namesz"GlmAsrProcessor.unused_input_namess ###r promptc tt|}tr d|D}t|}|dkrt d| |jg|z}nt |tr|g|z}nt |ttfrt||kr#t dt|d|dg}|D]X}|| |jt |tr| |Jtdntd d t||D}|j |fd d d d |S) a  Prepare inputs for automatic speech recognition without manually writing the default transcription prompt. Args: audio (`str`, `list[str]`, `np.ndarray`, `torch.Tensor`, `list[np.ndarray]`, `list[torch.Tensor]`): Audio to transcribe. Strings are interpreted as local paths or URLs and will be loaded automatically by the chat template loader; NumPy arrays and PyTorch tensors are forwarded directly. prompt (`str` or `list[str]`, *optional*): Custom prompt(s) to include in the user turn. A list must be the same length as the batch. When `None`, each sample uses `"Transcribe the input speech."`. **kwargs: Additional keyword arguments forwarded to [`~GlmAsrProcessor.apply_chat_template`] (for example `text_kwargs`, `audio_kwargs`, ...). Returns: [`BatchFeature`]: Processor outputs ready to be passed to [`GlmAsrForConditionalGeneration.generate`]. cg|]V}t|tjr8|n|WSr) isinstancerkTensordetachcpunumpy)r[els r!r]z?GlmAsrProcessor.apply_transcription_request..sNsssac 2u|8T8T\299;;??,,22444Z\sssr rz)`audio` must contain at least one sample.Nz Received z prompt(s) for z$ audio sample(s); counts must match.z'Each prompt must be a string or `None`.z<`prompt` must be a string, a sequence of strings, or `None`.cdg|]-\}}dt|trd|dnd|dd|dgdg.S)userr6)typepath)rr6r5)rr5)rolecontent)rstr)r[ prompt_text audio_items r!r]z?GlmAsrProcessor.apply_transcription_request..sz    ( Z#&j#66D*===&- CC!'==       r T)tokenizeadd_generation_prompt return_dict) listrr rJrBr-rrtuplerg TypeErrorzipapply_chat_template) r1r6rr8 audio_items batch_sizepromptsitem conversationss r!apply_transcription_requestz+GlmAsrProcessor.apply_transcription_requests2/33STY3Z3Z.[.[    tssgrsssK%% ??HII I >89JFGG  $ $ \h+GG u . . \6{{j(( lF llJlllG O O<NN4#DEEEEc**ONN4((((#$MNNN  OZ[[ [   ,/w +D+D    (t'  "&       r ) strip_prefixcLjj|i|}|rfd|D}|S)aj Forward arguments to [`~PreTrainedTokenizer.decode`] and optionally remove the assistant framing the model was trained to produce. AF3 transcription requests respond with sentences such as `"The spoken content of the audio is "..."."`. Setting `strip_prefix=True` trims the fixed prefix for just the transcription text. c:g|]}|Sr)"_strip_assistant_prefix_and_quotes)r[r5r1s r!r]z*GlmAsrProcessor.decode..s'YYYt>>tDDYYYr )r3decode)r1rargsr8decodeds` r!rzGlmAsrProcessor.decode sF($.'888  ZYYYYQXYYYGr c|j|i|S)z)BC as previous examples used batch_decode)r)r1rr8s r! batch_decodezGlmAsrProcessor.batch_decodest{D+F+++r c|}dD]B}||r+|t|d}nC|dr|dd}t|dkr8|d|dkr&|ddvr|dd}|S) zi Remove the assistant prefix and surrounding quotes from a decoded transcription string. )z"The spoken content of the audio isz!The transcription of the audio isz!The content of the input audio isN.r^rPr>"'rO)strip startswithrJendswith)r1r5strippedprefixs r!rz2GlmAsrProcessor._strip_assistant_prefix_and_quotess ::<<   F ""6** #CKKMM288::    S ! ! -}**,,H x==A  (1+""="=(1+Q[B[B["~++--Hr )Nr%r&r')NF)NN)rKrLr9rL)N)rrrrvalid_processor_kwargsr0r r rrboolr rrCrrIrUr~dictrarrnpropertyrrrrrr __classcell__)r4s@r!r$r$7s2 %MTTTTTT0$(%* AA$y/)AD Ad{ A ./ A  AAAAA^AB$(37 d dD  d$y/)D0 d)* d d d d d d(0J(0(0(0(0T3333333C49CCCCCXC$DI$$$X$*.J J T#Y+J d3i$&J ./ J  J J J J X*/     ,,,ssr r$)rnp audio_utilsrrfeature_extraction_utilsrprocessing_utilsrrr tokenization_utils_baser utilsr r r rk get_loggerrrerr$__all__rr r!rs:,GGGGGGGG444444HHHHHHHHHH000000@@@@@@@@@@LLL  H % %,E"xxxxxnxxxv  r