tjSUdZddlZddlmZddlmZddlZddlmZm Z dZ eGdd Z dd ej dzd ej ezezd ej d ededzf dZ dd ej dzd ej ezezd ej d ededzf dZddej dejdedzfdZddej dejdedzfdZ dd ej ezezdejdejd ededzdej dzf dZdS)a IMPORTANT NOTICE: Every class and function in this file is deprecated in favor of using the much more general `masking_utils.py` primitives. New code should not rely on it, it is only kept for backward compatibility for now, and will be removed in the future. N) dataclass)Union)is_torchdynamo_compiling is_tracingzThe attention mask API under `transformers.modeling_attn_mask_utils` (`AttentionMaskConverter`) is deprecated and will be removed in Transformers v5.10. Please use the new API in `transformers.masking_utils`.ceZdZUdZeed<eed<d dededzfdZ d!deded ed ej d e ej d fd ej dzf dZ d dej ded ej d edzd ej f dZe d"dejd ej d ej dededzf dZed dej d ej dedzfdZedejdefdZe d#dej dzdej dededzded ef dZdS)$AttentionMaskConvertera9 A utility attention mask class that allows one to: - Create a causal 4d mask - Create a causal 4d mask with slided window - Convert a 2d attention mask (batch_size, query_length) to a 4d attention mask (batch_size, 1, query_length, key_value_length) that can be multiplied with attention scores Examples: ```python >>> import torch >>> from transformers.modeling_attn_mask_utils import AttentionMaskConverter >>> converter = AttentionMaskConverter(True) >>> converter.to_4d(torch.tensor([[0, 0, 0, 1, 1]]), 5, key_value_length=5, dtype=torch.float32) tensor([[[[-3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38], [-3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38], [-3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38], [-3.4028e+38, -3.4028e+38, -3.4028e+38, 0.0000e+00, -3.4028e+38], [-3.4028e+38, -3.4028e+38, -3.4028e+38, 0.0000e+00, 0.0000e+00]]]]) ``` Parameters: is_causal (`bool`): Whether the attention mask should be a uni-directional (causal) or bi-directional mask. sliding_window (`int`, *optional*): Optionally, the sliding window masks can be created if `sliding_window` is defined to a positive integer. is_causalsliding_windowNctjtt||_||_|j#|jdkrt d|jddSdS)NrzaMake sure that when passing `sliding_window` that its value is a strictly positive integer, not ``)warningswarnDEPRECATION_MESSAGE FutureWarningr r ValueError)selfr r s c/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/modeling_attn_mask_utils.py__init__zAttentionMaskConverter.__init__Fs| )=999",   *t/Ba/G/GKtxuHKKK  + */G/Gcpu batch_size query_lengthkey_value_lengthdtypedevicestrreturnc|jstd|jd||f}||z }d}|ddks|j||||||j}|S)z Creates a causal 4D mask of (bsz, head_dim=1, query_length, key_value_length) shape and adds large negative bias to upper right hand triangular matrix (causal mask). z"Please use `to_causal_4d` only if z has `is_causal` set to True.Nrrpast_key_values_lengthr )r r __class__r _make_causal_mask) rrrrrr input_shaper"causal_4d_masks r to_causal_4dz#AttentionMaskConverter.to_causal_4dQs~ qo$.ooopp p"<0 !1L!@ r?Q  $"5"A!33'=#2 4Nrattention_mask_2dc|jd|f}d}|ddks|jB|jr;|td||z }||||j||j}n|jt d||||d|j}|?| | tj |j }|} | S) a Converts 2D attention mask to 4D attention mask by expanding mask to (bsz, head_dim=1, query_length, key_value_length) shape and by adding a large negative bias to not-attended positions. If attention_mask is causal, a causal mask will be added. rNr rzpThis attention mask converter is causal. Make sure to pass `key_value_length` to correctly create a causal mask.r!z?Sliding window is currently only implemented for causal masking)tgt_len)shaper r rr$rNotImplementedError _expand_maskto masked_fillbooltorchfinfomin) rr(rrrr%r&r"expanded_attn_maskexpanded_4d_masks rto_4dzAttentionMaskConverter.to_4drs5).q1<@  Oa  4#6#B#B' G&6 %D "!33(/'=#2 4NN ,%&ghh h"../@%Q\]_Q`.aadd  $    %!/!;!;>>  )y1}&:&:499R==!&L&LLaPPPwwu~~ !A % %9ek'3IQV_efffhlmsuvvvD  %->BH :eod%*&M&M&MX`aaaL()) $zz||   lEK,>,>,B C C CD$111$%,,S!WgH^>^___rrIr*ctjtt|\}}||n|}|ddddddf|d|||}tjd||z }| |tj tj |j S)zg Expands attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`. Nr?r;) rrrrr?rGr.r1tensorr/r0r2r3)rIrr*rHsrc_len expanded_mask inverted_masks rr-z#AttentionMaskConverter._expand_masks  )=999yy{{ W$0''gQQQdAAA-.55c1gwOORRSXYY  S666F (()9)9%*)E)Eu{SXGYGYG]^^^rrP min_dtypectjtt|jt jkrtd|t j ||kddS)a Attend to all tokens in masked rows from the expanded attention mask, for example the relevant first rows when using left padding. This is required by F.scaled_dot_product_attention memory-efficient attention path. Details: https://github.com/pytorch/pytorch/issues/110213 `expanded_mask` is [bsz, num_masks, tgt_seq_len, src_seq_len] or [bsz, tgt_seq_len, src_seq_len]. `attention_mask` is [bsz, src_seq_len]. The dimension num_masks of `expanded_mask` is most often 1, but it can also be the number of heads in the case of alibi attention bias. For example, if `expanded_mask` is (e.g. here left-padding case) ``` [[[[0, 0, 0], [0, 0, 0], [0, 0, 1]]], [[[1, 0, 0], [1, 1, 0], [1, 1, 1]]], [[[0, 0, 0], [0, 1, 0], [0, 1, 1]]]] ``` then the modified `expanded_mask` will be ``` [[[[1, 1, 1], <-- modified [1, 1, 1], <-- modified [0, 0, 1]]], [[[1, 0, 0], [1, 1, 0], [1, 1, 1]]], [[[1, 1, 1], <-- modified [0, 1, 0], [0, 1, 1]]]] ``` z\AttentionMaskConverter._unmask_unattended expects a float `expanded_mask`, got a BoolTensor.r T)r:keepdim) rrrrrr1r0rmulall)rPrRs r_unmask_unattendedz)AttentionMaskConverter._unmask_unattendedsoR  )=999  %* , ,n   %)MY,FBX\"]"]"]!]^^^rFattention_mask inputs_embeds is_trainingcrtjtt|jd|jd}}||z}t |}d} ||s|s|dks||kr |||krd} nJ|||krBt |jdkrdS|s&tj|dkr|dks||krd} | S)a9 Detects whether the optional user-specified attention_mask & the automatically created causal mask can be ignored in case PyTorch's SDPA is used, rather relying on SDPA's `is_causal` argument. In case no token is masked in the `attention_mask` argument, if `query_length == 1` or `key_value_length == query_length`, we rather rely on SDPA `is_causal` argument to use causal/non-causal masks, allowing to dispatch to the flash attention kernel (that can otherwise not be used if a custom `attn_mask` is passed). rrFNT) rrrrr+rlenr1rV) rXrYr"r rZ_rr is_tracing_ignore_causal_masks r_ignore_causal_mask_sdpaz/AttentionMaskConverter._ignore_causal_mask_sdpas"  )=999'-a0-2Ea2H<'*@@ // "  ! *$/ *!Q&&*:l*J*J#+/?./P/P%)"  #'7.'H'H>'((A--u  .UY~/B%C%C .1$$(8L(H(H)-&"!rN)rrN)NF)__name__ __module__ __qualname____doc__r0__annotations__intrr1rrrTensorr'r6 staticmethodSizer$r- FloatTensorfloatrWrarrr r #s<OOO  $ d    ".3   {  elE)*   L(, - -  <- - { - * -  - - - - ^ '(%) !`!`!`{!` !`!$ !` d !`!`!`\!`F _ _5< _ _cDj _ _ _\ _0_(0_0_0_0_\0_d &*! 8"8" t+8"|8"!$8"d 8"  8"  8"8"8"\8"8"8"rr rXr%rYr"r ctd|}|d|z}|=t|jdkr%|||d||j}n|t|jdkr|dd |d |f}t |j|kr(t d t |jd |d d |z }||tj tj |jj }n0| |d|d||j|j}|S)a Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)` Args: attention_mask (`torch.Tensor` or `None`): A 2D attention mask of shape `(batch_size, key_value_length)` input_shape (`tuple(int)` or `list(int)` or `torch.Size`): The input shape should be a tuple that defines `(batch_size, query_length)`. inputs_embeds (`torch.Tensor`): The embedded inputs as a torch Tensor. past_key_values_length (`int`): The length of the key value cache. sliding_window (`int`, *optional*): If the model uses windowed attention, a sliding window should be passed. Tr r r N)rrr\rrz#Incorrect 4D attention_mask shape: z ; expected: .rMr9)r r]r+r6rtuplerr/r.r1r0r2r3r'r) rXr%rYr"r attn_mask_converterrexpected_shaperQs r!_prepare_4d_causal_attention_maskrwDsx.14P^___"2)??!c.*>&?&?1&D&D,22 KO>NVcVi3    #N,@(A(AQ(F(F%a.![^=MN % & &. 8 8peNrss !!!!!! DDDDDDDDw  ]"]"]"]"]"]"]" ]"J "& 11L4'1e#d*1<1 1 $J 1111t"& 77L4'7e#d*7<7 7 $J 7777t X XU\ X%+ XPSVZPZ X X X X \\el\5;\Y\_cYc\\\\:#$!% e#d* ; L  $J   \D r