tj( UdZddlZddlZgZeeed<dejdefdZ dejde dejfd Z d e d e dzde fd Z ddejdejdejdejdzddf dZdS)zCDefines utilities for interacting with scaled_dot_product_attentionN__all__tensorsreturnc4td|DS)z0Returns True if any of the tensors requires gradc3$K|] }|jV dS)N) requires_grad).0ts W/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/torch/nn/attention/_utils.py z'_input_requires_grad..s$001q000000)any)rs r _input_requires_gradr s 00000 0 00r inpt_tensorog_sizecP|d|kr |dd|fS|S)z'Handles the unpad of the last dimension.N)size)rrs r _postprocess_flash_outputrs4w&&3=)) r head_dim_sizescalec8||Sdtj|z S)z For FlashAttention we pad the head dimension to be a multiple of 8 so we need to scale the output by the original head size and not the padded. Ng?)mathsqrt)rrs r _calculate_scalers$   =)) ))r Fquerykeyvalue attn_maskc X|sH|j|jks|j|jkr(td|jd|jd|jd|j|jks|j|jkr(td|jd|jd|jd|dks0|dks|dkrOtd |d |d |ddS) NzLExpected query, key, and value to have the same dtype, but got query.dtype: z , key.dtype: z, and value.dtype: z instead.zSExpected query, key, and value to have the same device type, but got query.device: z, key.device: z, and value.device: zUExpected query, key, and value to all be at least 2 dimensional, but got query.dim: z , key.dim: z and value.dim: )dtype ValueErrordevicedim)rrrr dropout_p is_causalr allow_lowp_kvs r _validate_sdpa_inputr*"sw  ;#) # #u{ek'A'A;(- ;;BE);;$)K;;;   |sz!!U\U\%A%A 9%*\ 9 9AD 9 9!& 9 9 9    yy{{Q#''))a--599;;?? Yyy{{ Y Y'*wwyy Y YBG))++ Y Y Y   ,;?r )NrFNF)__doc__rtorchrliststr__annotations__Tensorboolrintrfloatrr*r r r5s#III c15<1D1111 5<#%,*C* *****&*   <   < |d"         r