tj&ddlZddlZddlmZddlmZddlZddlmZddlm Z ddl m Z ddl mZmZdd lmZdd lmZdd lmZmZdd lmZdd lmZmZmZmZddlmZm Z ddl!m"Z"m#Z#ddl$m%Z%Gddej&Z'edeGddeZ(edeGddeZ)edeGddeZ*edeGdd eZ+Gd!d"ej&Z,Gd#d$ej&Z-Gd%d&ej&Z.Gd'd(ej&Z/ dMd*ej&d+ej0d,ej0d-ej0d.ej0dzd/e1dzd0e1d1eefd2Z2Gd3d4ej&Z3Gd5d6ej&Z4d7Z5d8Z6Gd9d:eZ7Gd;deZ9Gd?d@e9Z:eGdAdBe9Z;edCGdDdEe9Z<edFGdGdHe9Z=edIGdJdKee9Z>gdLZ?dS)NN)Callable) dataclass)nn)initialization)ACT2FN) BackboneMixinfilter_output_hidden_states)GradientCheckpointingLayer)BackboneOutput)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack) ModelOutputTransformersKwargsauto_docstring torch_int)can_return_tuplemerge_with_config_defaults)OutputRecordercapture_outputs) SwinConfigc^eZdZdZd deddffd ZdejdejfdZde fd Z xZ S) SwinDropPathzStochastic depth (DropPath) per sample, for residual blocks. Identity when ``drop_prob`` is 0 or outside training. See `Deep Networks with Stochastic Depth `_. drop_probreturnNcVt||_dSN)super__init__r)selfr __class__s d/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/transformers/models/swin/modeling_swin.pyr"zSwinDropPath.__init__1s$ " hidden_statesc|jdks|js|Sd|jz }|jdfd|jdz zz}t j||j|j}t j||z}| ||zS)Nrrrrdtypedevice) rtrainingshapendimtorchrandr+r,floordiv)r#r' keep_probr. random_tensors r%forwardzSwinDropPath.forward5s >S   & $Q')DM4F4J,KK 5 0CML`aaa  MI$=>>   ++m;;r&cd|jS)Nzp=)r)r#s r% extra_reprzSwinDropPath.extra_repr>s$DN$$$r&)r) __name__ __module__ __qualname____doc__floatr"r0Tensorr6strr8 __classcell__r$s@r%rr*s ##%#$######