tjddlZddlmZmZejeZerddlZ ddejdejdejdejd e d e d ejfd Z dd Z dS)N)is_torchaudio_availablelogging mean_volumelogitstargets logit_lengthstarget_lengthsblank_token_id reductionreturnc tstdd}||vr3td|ddd|Dd||j}t j| ||j ||j | |d }|d kr;| | z S|d kr| S|d kr)|| z S|d kr| S|S)a Compute standard RNN-T (RNN Transducer) loss (https://huggingface.co/papers/1211.3711). Thin wrapper around [`torchaudio.functional.rnnt_loss`]. torchaudio is queried with `reduction="none"` to get the per-sample negative log-likelihoods, and the requested reduction is applied here. The reduction names and formulas mirror NeMo's `RNNTLoss` (the reference implementation used to train/finetune Parakeet), so that loss magnitudes and gradient scaling match when finetuning other RNNT models like Parakeet: - `"mean_volume"`: sum of per-sample losses divided by the sum of target lengths (per-token average over the whole batch). This is what `nvidia/parakeet-rnnt-0.6b` is trained with (`rnnt_reduction: mean_volume`). - `"mean_batch"`: plain average of per-sample losses over the batch (NeMo's default). - `"mean"`: per-sample loss divided by its own target length, then averaged over the batch. - `"sum"`: sum of per-sample losses. - `"none"`: per-sample losses, unreduced. Args: logits: Joint token logits of shape `(batch, T, U+1, vocab_size)`. targets: Target labels of shape `(batch, U)`. logit_lengths: Encoder output lengths of shape `(batch,)`. target_lengths: Target lengths of shape `(batch,)`. blank_token_id: Blank token id. reduction: Loss reduction method. One of `"mean_volume"`, `"mean_batch"`, `"mean"`, `"sum"`, or `"none"`. Returns: Scalar loss tensor (or per-example losses if `reduction="none"`). zWComputing the RNN-T loss requires torchaudio. Install it with `pip install torchaudio`.)r mean_batchmeansumnonezInvalid reduction mode "z". Expected one of z, c34K|]}t|VdS)N)repr).0rs Y/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/loss/loss_rnnt.py zrnnt_loss..Ds*NqNq[\tTUwwNqNqNqNqNqNq.r)rrr r blankr rrrr)r ImportError ValueErrorjointodevice torchaudio functional rnnt_lossfloat contiguousintrr)rrr r r r valid_reductionslossess rr#r#sH # $ $ustttK((( ty t tTYYNqNq`pNqNqNqEqEq t t t   $&&v}55N  " , ,||~~((** 6=))--//#&&v}5599;;%))++ -FM!!zz||n224488:::: l " "{{}} f  --///55777 e  zz|| Mrc ,t||||||S)N)rrr r r r )r#)rlabelsr label_lengthsr r kwargss rParakeetForRNNTLossr-\s- #$%    r)r) torchutilsrr get_logger__name__loggerr!Tensorr&strr#r-rrr6s 44444444  H % %# ?? L? \?<?L ?  ?  ? \????P r