tj dZddlZddlZddlmZddlmZddlZddlmZddl m Z ddl m Z dd lmZmZdd lmZdd lmZdd lmZdd lmZddlmZmZmZmZmZddlm Z m!Z!ddl"m#Z#m$Z$ddl%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+ddl,m-Z-ej.e/Z0Gddej1Z2edeGddeZ3edeGddeZ4edeGdd eZ5ed!eGd"d#eZ6d$Z7d%Z8Gd&d'ej1Z9Gd(d)ej1Z:Gd*d+ej1Z;Gd,d-ej1Z<Gd.d/e'Z=Gd0d1e)Z>Gd2d3e(Z?Gd4d5eZ@eGd6d7e*ZAGd8d9eAZBeGd:d;eAZCed<Gd=d>eAZDed?Gd@dAeAZEedBGdCdDeeAZFgdEZGdS)FzPyTorch Swin Transformer model.N)Callable) dataclass)nn)initialization)ACT2FN) BackboneMixinfilter_output_hidden_states)GradientCheckpointingLayer)BackboneOutput)ALL_ATTENTION_FUNCTIONS)Unpack) ModelOutputTransformersKwargsauto_docstringlogging torch_int)can_return_tuplemerge_with_config_defaults)OutputRecordercapture_outputs)PreTrainedModel ViTAttentionViTLayerViTMLPViTPreTrainedModeleager_attention_forward) SwinConfigc^eZdZdZd deddffd ZdejdejfdZde fd Z xZ S) SwinDropPathzStochastic depth (DropPath) per sample, for residual blocks. Identity when ``drop_prob`` is 0 or outside training. See `Deep Networks with Stochastic Depth `_. drop_probreturnNcVt||_dSN)super__init__r$)selfr$ __class__s c/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/models/swin/modular_swin.pyr)zSwinDropPath.__init__7s$ " hidden_statesc|jdks|js|Sd|jz }|jdfd|jdz zz}t j||j|j}t j||z}| ||zS)Nr#rrrdtypedevice) r$trainingshapendimtorchrandr2r3floordiv)r*r. keep_probr5 random_tensors r,forwardzSwinDropPath.forward;s >S   & $Q')DM4F4J,KK 5 0CML`aaa  MI$=>>   ++m;;r-cd|jS)Nzp=)r$)r*s r, extra_reprzSwinDropPath.extra_reprDs$DN$$$r-)r#) __name__ __module__ __qualname____doc__floatr)r7Tensorr=strr? __classcell__r+s@r,r"r"0s ##%#$######