tjTddlZddlmcmZddlmZddlmZm Z gdZ Gdde Z Gdd e Z Gd d ee Z Gd d e ZGddee ZGdde ZGddee ZdS)N)Tensor) _LazyNormBase _NormBase)InstanceNorm1dInstanceNorm2dInstanceNorm3dLazyInstanceNorm1dLazyInstanceNorm2dLazyInstanceNorm3dceZdZ ddddeded ed ed ed ed dffdZdZdZdZ dZ dfd Z de d e fdZ xZS) _InstanceNormh㈵>皙?FNT)bias num_featuresepsmomentumaffinetrack_running_statsrreturncV||d} tj|||||fi| d|idS)N)devicedtyper)super__init__) selfrrrrrrrrfactory_kwargs __class__s [/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/torch/nn/modules/instancenorm.pyrz_InstanceNorm.__init__si%+U;;                 ctNNotImplementedErrorrinputs r _check_input_dimz_InstanceNorm._check_input_dim-!!r!ctr#r$rs r _get_no_batch_dimz_InstanceNorm._get_no_batch_dim0r)r!cx||ddS)Nr)_apply_instance_norm unsqueezesqueezer&s r _handle_no_batch_inputz$_InstanceNorm._handle_no_batch_input3s0((););<<DDQGGGr!c tj||j|j|j|j|jp|j |j|jnd|j S)Ng) F instance_norm running_mean running_varweightrtrainingrrrr&s r r.z"_InstanceNorm._apply_instance_norm6sT      K I M 9!99!]6DMMC H   r!c |dd}||jsg} dD] } || z} | |vr| | !t| dkrk|ddd| D|jj| D]} || t |||||||dS)Nversion)r5r6raUnexpected running stats buffer(s) {names} for {klass} with track_running_stats=False. If state_dict is a checkpoint saved before 0.4.0, this may be expected because {klass} does not track running stats by default since 0.4.0. Please remove these keys from state_dict. If the running stats are actually needed, instead set track_running_stats=True in {klass} to enable them. See the documentation of {klass} for details.z and c3"K|] }d|dV dS)"N).0ks r z6_InstanceNorm._load_from_state_dict.._s**P*P8q888*P*P*P*P*P*Pr!)namesklass) getrappendlenformatjoinr__name__popr_load_from_state_dict) r state_dictprefixlocal_metadatastrict missing_keysunexpected_keys error_msgsr:running_stats_keysnamekeyrs r rJz#_InstanceNorm._load_from_state_dictBs<!$$Y55 ?4#;?!# 7 3 3tm*$$&--c222%&&**!!@AG%ll*P*P=O*P*P*PPP"n5AGAA   .((CNN3'''' %%            r!r'c ||||z }|||jkrR|jr1t d|d|jd||dtjd|dd||kr| |S| |S) Nzexpected input's size at dim=z to match num_features (z ), but got: .zinput's size at dim=z does not match num_features. You can silence this warning by not passing in num_features, which is not used because affine=False) stacklevel) r(dimr,sizerr ValueErrorwarningswarnr1r.)rr' feature_dims r forwardz_InstanceNorm.forwardps0 e$$$iikkD$:$:$<$<< ::k " "d&7 7 7{  SKSS*SS8= ;8O8OSSS  =;=== !  99;;$0022 2 2..u55 5((///r!)rrFFNNrN)rH __module__ __qualname__intfloatboolrr(r,r1r.rJrr_ __classcell__)rs@r rrs3$)          "          .""""""HHH    ,  , , , , , , \0V000000000r!rc&eZdZdZdefdZddZdS)ra Applies Instance Normalization. This operation applies Instance Normalization over a 2D (unbatched) or 3D (batched) input as described in the paper `Instance Normalization: The Missing Ingredient for Fast Stylization `__. .. math:: y = \frac{x - \mathrm{E}[x]}{ \sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta The mean and standard-deviation are calculated per-dimension separately for each object in a mini-batch. :math:`\gamma` and :math:`\beta` are learnable parameter vectors of size `C` (where `C` is the number of features or channels of the input) if :attr:`affine` is ``True``. The variance is calculated via the biased estimator, equivalent to `torch.var(input, correction=0)`. By default, this layer uses instance statistics computed from input data in both training and evaluation modes. If :attr:`track_running_stats` is set to ``True``, during training this layer keeps running estimates of its computed mean and variance, which are then used for normalization during evaluation. The running estimates are kept with a default :attr:`momentum` of 0.1. .. note:: This :attr:`momentum` argument is different from one used in optimizer classes and the conventional notion of momentum. Mathematically, the update rule for running statistics here is :math:`\hat{x}_\text{new} = (1 - \text{momentum}) \times \hat{x} + \text{momentum} \times x_t`, where :math:`\hat{x}` is the estimated statistic and :math:`x_t` is the new observed value. .. note:: :class:`InstanceNorm1d` and :class:`LayerNorm` are very similar, but have some subtle differences. :class:`InstanceNorm1d` is applied on each channel of channeled data like multidimensional time series, but :class:`LayerNorm` is usually applied on entire sample and often in NLP tasks. Additionally, :class:`LayerNorm` applies elementwise affine transform, while :class:`InstanceNorm1d` usually doesn't apply affine transform. Args: num_features: number of features or channels :math:`C` of the input eps: a value added to the denominator for numerical stability. Default: 1e-5 momentum: the value used for the running_mean and running_var computation. Default: 0.1 affine: a boolean value that when set to ``True``, this module has learnable affine parameters, initialized the same way as done for batch normalization. Default: ``False`` track_running_stats: a boolean value that when set to ``True``, this module tracks the running mean and variance, and when set to ``False``, this module does not track such statistics and always uses batch statistics in both training and eval modes. Default: ``False`` bias: If set to ``False``, the layer will not learn an additive bias (only relevant if :attr:`affine` is ``True``). Default: ``True`` Shape: - Input: :math:`(N, C, L)` or :math:`(C, L)` - Output: :math:`(N, C, L)` or :math:`(C, L)` (same shape as input) Examples:: >>> # Without Learnable Parameters >>> m = nn.InstanceNorm1d(100) >>> # With Learnable Parameters >>> m = nn.InstanceNorm1d(100, affine=True) >>> input = torch.randn(20, 100, 40) >>> output = m(input) rcdSNrWr=r+s r r,z InstanceNorm1d._get_no_batch_dimqr!Nc||dvr%td|ddSN)rWzexpected 2D or 3D input (got D input)rYr[r&s r r(zInstanceNorm1d._check_input_dim? 99;;f $ $RUYY[[RRRSS S % $r!r`rHrarb__doc__rcr,r(r=r!r rrsVDDL3TTTTTTr!rc*eZdZdZeZdefdZddZdS)r aA :class:`torch.nn.InstanceNorm1d` module with lazy initialization of the ``num_features`` argument. The ``num_features`` argument of the :class:`InstanceNorm1d` is inferred from the ``input.size(1)``. The attributes that will be lazily initialized are `weight`, `bias`, `running_mean` and `running_var`. Check the :class:`torch.nn.modules.lazy.LazyModuleMixin` for further documentation on lazy modules and their limitations. Args: num_features: :math:`C` from an expected input of size :math:`(N, C, L)` or :math:`(C, L)` eps: a value added to the denominator for numerical stability. Default: 1e-5 momentum: the value used for the running_mean and running_var computation. Default: 0.1 affine: a boolean value that when set to ``True``, this module has learnable affine parameters, initialized the same way as done for batch normalization. Default: ``False`` track_running_stats: a boolean value that when set to ``True``, this module tracks the running mean and variance, and when set to ``False``, this module does not track such statistics and always uses batch statistics in both training and eval modes. Default: ``False`` bias: If set to ``False``, the layer will not learn an additive bias (only relevant if :attr:`affine` is ``True``). Default: ``True`` Shape: - Input: :math:`(N, C, L)` or :math:`(C, L)` - Output: :math:`(N, C, L)` or :math:`(C, L)` (same shape as input) rcdSrir=r+s r r,z$LazyInstanceNorm1d._get_no_batch_dimrjr!Nc||dvr%td|ddSrlror&s r r(z#LazyInstanceNorm1d._check_input_dimrpr!r`) rHrarbrrr cls_to_becomercr,r(r=r!r r r sZ8#M3TTTTTTr!r c&eZdZdZdefdZddZdS)ra Applies Instance Normalization. This operation applies Instance Normalization over a 4D input (a mini-batch of 2D inputs with additional channel dimension) as described in the paper `Instance Normalization: The Missing Ingredient for Fast Stylization `__. .. math:: y = \frac{x - \mathrm{E}[x]}{ \sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta The mean and standard-deviation are calculated per-dimension separately for each object in a mini-batch. :math:`\gamma` and :math:`\beta` are learnable parameter vectors of size `C` (where `C` is the input size) if :attr:`affine` is ``True``. The standard-deviation is calculated via the biased estimator, equivalent to `torch.var(input, correction=0)`. By default, this layer uses instance statistics computed from input data in both training and evaluation modes. If :attr:`track_running_stats` is set to ``True``, during training this layer keeps running estimates of its computed mean and variance, which are then used for normalization during evaluation. The running estimates are kept with a default :attr:`momentum` of 0.1. .. note:: This :attr:`momentum` argument is different from one used in optimizer classes and the conventional notion of momentum. Mathematically, the update rule for running statistics here is :math:`\hat{x}_\text{new} = (1 - \text{momentum}) \times \hat{x} + \text{momentum} \times x_t`, where :math:`\hat{x}` is the estimated statistic and :math:`x_t` is the new observed value. .. note:: :class:`InstanceNorm2d` and :class:`LayerNorm` are very similar, but have some subtle differences. :class:`InstanceNorm2d` is applied on each channel of channeled data like RGB images, but :class:`LayerNorm` is usually applied on entire sample and often in NLP tasks. Additionally, :class:`LayerNorm` applies elementwise affine transform, while :class:`InstanceNorm2d` usually doesn't apply affine transform. Args: num_features: :math:`C` from an expected input of size :math:`(N, C, H, W)` or :math:`(C, H, W)` eps: a value added to the denominator for numerical stability. Default: 1e-5 momentum: the value used for the running_mean and running_var computation. Default: 0.1 affine: a boolean value that when set to ``True``, this module has learnable affine parameters, initialized the same way as done for batch normalization. Default: ``False`` track_running_stats: a boolean value that when set to ``True``, this module tracks the running mean and variance, and when set to ``False``, this module does not track such statistics and always uses batch statistics in both training and eval modes. Default: ``False`` bias: If set to ``False``, the layer will not learn an additive bias (only relevant if :attr:`affine` is ``True``). Default: ``True`` Shape: - Input: :math:`(N, C, H, W)` or :math:`(C, H, W)` - Output: :math:`(N, C, H, W)` or :math:`(C, H, W)` (same shape as input) Examples:: >>> # Without Learnable Parameters >>> m = nn.InstanceNorm2d(100) >>> # With Learnable Parameters >>> m = nn.InstanceNorm2d(100, affine=True) >>> input = torch.randn(20, 100, 35, 45) >>> output = m(input) rcdSNrmr=r+s r r,z InstanceNorm2d._get_no_batch_dimGrjr!Nc||dvr%td|ddSN)rmzexpected 3D or 4D input (got rnror&s r r(zInstanceNorm2d._check_input_dimJrpr!r`rqr=r!r rrsVFFP3TTTTTTr!rc*eZdZdZeZdefdZddZdS)r a3A :class:`torch.nn.InstanceNorm2d` module with lazy initialization of the ``num_features`` argument. The ``num_features`` argument of the :class:`InstanceNorm2d` is inferred from the ``input.size(1)``. The attributes that will be lazily initialized are `weight`, `bias`, `running_mean` and `running_var`. Check the :class:`torch.nn.modules.lazy.LazyModuleMixin` for further documentation on lazy modules and their limitations. Args: num_features: :math:`C` from an expected input of size :math:`(N, C, H, W)` or :math:`(C, H, W)` eps: a value added to the denominator for numerical stability. Default: 1e-5 momentum: the value used for the running_mean and running_var computation. Default: 0.1 affine: a boolean value that when set to ``True``, this module has learnable affine parameters, initialized the same way as done for batch normalization. Default: ``False`` track_running_stats: a boolean value that when set to ``True``, this module tracks the running mean and variance, and when set to ``False``, this module does not track such statistics and always uses batch statistics in both training and eval modes. Default: ``False`` bias: If set to ``False``, the layer will not learn an additive bias (only relevant if :attr:`affine` is ``True``). Default: ``True`` Shape: - Input: :math:`(N, C, H, W)` or :math:`(C, H, W)` - Output: :math:`(N, C, H, W)` or :math:`(C, H, W)` (same shape as input) rcdSryr=r+s r r,z$LazyInstanceNorm2d._get_no_batch_dimorjr!Nc||dvr%td|ddSr{ror&s r r(z#LazyInstanceNorm2d._check_input_dimrrpr!r`) rHrarbrrrrvrcr,r(r=r!r r r OZ:#M3TTTTTTr!r c&eZdZdZdefdZddZdS)r a Applies Instance Normalization. This operation applies Instance Normalization over a 5D input (a mini-batch of 3D inputs with additional channel dimension) as described in the paper `Instance Normalization: The Missing Ingredient for Fast Stylization `__. .. math:: y = \frac{x - \mathrm{E}[x]}{ \sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta The mean and standard-deviation are calculated per-dimension separately for each object in a mini-batch. :math:`\gamma` and :math:`\beta` are learnable parameter vectors of size C (where C is the input size) if :attr:`affine` is ``True``. The standard-deviation is calculated via the biased estimator, equivalent to `torch.var(input, correction=0)`. By default, this layer uses instance statistics computed from input data in both training and evaluation modes. If :attr:`track_running_stats` is set to ``True``, during training this layer keeps running estimates of its computed mean and variance, which are then used for normalization during evaluation. The running estimates are kept with a default :attr:`momentum` of 0.1. .. note:: This :attr:`momentum` argument is different from one used in optimizer classes and the conventional notion of momentum. Mathematically, the update rule for running statistics here is :math:`\hat{x}_\text{new} = (1 - \text{momentum}) \times \hat{x} + \text{momentum} \times x_t`, where :math:`\hat{x}` is the estimated statistic and :math:`x_t` is the new observed value. .. note:: :class:`InstanceNorm3d` and :class:`LayerNorm` are very similar, but have some subtle differences. :class:`InstanceNorm3d` is applied on each channel of channeled data like 3D models with RGB color, but :class:`LayerNorm` is usually applied on entire sample and often in NLP tasks. Additionally, :class:`LayerNorm` applies elementwise affine transform, while :class:`InstanceNorm3d` usually doesn't apply affine transform. Args: num_features: :math:`C` from an expected input of size :math:`(N, C, D, H, W)` or :math:`(C, D, H, W)` eps: a value added to the denominator for numerical stability. Default: 1e-5 momentum: the value used for the running_mean and running_var computation. Default: 0.1 affine: a boolean value that when set to ``True``, this module has learnable affine parameters, initialized the same way as done for batch normalization. Default: ``False`` track_running_stats: a boolean value that when set to ``True``, this module tracks the running mean and variance, and when set to ``False``, this module does not track such statistics and always uses batch statistics in both training and eval modes. Default: ``False`` bias: If set to ``False``, the layer will not learn an additive bias (only relevant if :attr:`affine` is ``True``). Default: ``True`` Shape: - Input: :math:`(N, C, D, H, W)` or :math:`(C, D, H, W)` - Output: :math:`(N, C, D, H, W)` or :math:`(C, D, H, W)` (same shape as input) Examples:: >>> # Without Learnable Parameters >>> m = nn.InstanceNorm3d(100) >>> # With Learnable Parameters >>> m = nn.InstanceNorm3d(100, affine=True) >>> input = torch.randn(20, 100, 35, 45, 10) >>> output = m(input) rcdSNr|r=r+s r r,z InstanceNorm3d._get_no_batch_dimrjr!Nc||dvr%td|ddSN)r|zexpected 4D or 5D input (got rnror&s r r(zInstanceNorm3d._check_input_dimrpr!r`rqr=r!r r r wsVEEN3TTTTTTr!r c*eZdZdZeZdefdZddZdS)r aEA :class:`torch.nn.InstanceNorm3d` module with lazy initialization of the ``num_features`` argument. The ``num_features`` argument of the :class:`InstanceNorm3d` is inferred from the ``input.size(1)``. The attributes that will be lazily initialized are `weight`, `bias`, `running_mean` and `running_var`. Check the :class:`torch.nn.modules.lazy.LazyModuleMixin` for further documentation on lazy modules and their limitations. Args: num_features: :math:`C` from an expected input of size :math:`(N, C, D, H, W)` or :math:`(C, D, H, W)` eps: a value added to the denominator for numerical stability. Default: 1e-5 momentum: the value used for the running_mean and running_var computation. Default: 0.1 affine: a boolean value that when set to ``True``, this module has learnable affine parameters, initialized the same way as done for batch normalization. Default: ``False`` track_running_stats: a boolean value that when set to ``True``, this module tracks the running mean and variance, and when set to ``False``, this module does not track such statistics and always uses batch statistics in both training and eval modes. Default: ``False`` bias: If set to ``False``, the layer will not learn an additive bias (only relevant if :attr:`affine` is ``True``). Default: ``True`` Shape: - Input: :math:`(N, C, D, H, W)` or :math:`(C, D, H, W)` - Output: :math:`(N, C, D, H, W)` or :math:`(C, D, H, W)` (same shape as input) rcdSrr=r+s r r,z$LazyInstanceNorm3d._get_no_batch_dimrjr!Nc||dvr%td|ddSrror&s r r(z#LazyInstanceNorm3d._check_input_dimrpr!r`) rHrarbrrr rvrcr,r(r=r!r r r rr!r )r\torch.nn.functionalnn functionalr3torchr batchnormrr__all__rrr rr r r r=r!r rs////////   p0p0p0p0p0Ip0p0p0fLTLTLTLTLT]LTLTLT^$T$T$T$T$T $T$T$TNNTNTNTNTNT]NTNTNTb%T%T%T%T%T %T%T%TPMTMTMTMTMT]MTMTMT`%T%T%T%T%T %T%T%T%T%Tr!