a  d@sVddlZddlZddlZddlmZddZdd d Zdd d ZdddZddZ dS)N)_calculate_fan_in_and_fan_outcCsdd}||d|ks(||d|kr6tjddd||||}||||}|d|dd|d|||td|||j||d|S) NcSsdt|tddS)N?@)matherfsqrt)xr `/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/timm/layers/weight_init.pynorm_cdf sz _trunc_normal_..norm_cdfzjmean is more than 2 std from [a, b] in nn.init.trunc_normal_. The distribution of values may be incorrect.) stacklevelr)minmax) warningswarnuniform_Zerfinv_mul_rradd_Zclamp_)tensormeanstdabr lur r r _trunc_normal_s  rrrcCs<t t|||||WdS1s.0YdS)aFills the input Tensor with values drawn from a truncated normal distribution. The values are effectively drawn from the normal distribution :math:`\mathcal{N}(\text{mean}, \text{std}^2)` with values outside :math:`[a, b]` redrawn until they are within the bounds. The method used for generating the random values works best when :math:`a \leq \text{mean} \leq b`. NOTE: this impl is similar to the PyTorch trunc_normal_, the bounds [a, b] are applied while sampling the normal with mean/std applied, therefore a, b args should be adjusted to match the range of mean, std args. Args: tensor: an n-dimensional `torch.Tensor` mean: the mean of the normal distribution std: the standard deviation of the normal distribution a: the minimum cutoff value b: the maximum cutoff value Examples: >>> w = torch.empty(3, 5) >>> nn.init.trunc_normal_(w) N)torchno_gradrrrrrrr r r trunc_normal_+s r#cCsLt0t|dd|||||Wdn1s>0Y|S)aFills the input Tensor with values drawn from a truncated normal distribution. The values are effectively drawn from the normal distribution :math:`\mathcal{N}(\text{mean}, \text{std}^2)` with values outside :math:`[a, b]` redrawn until they are within the bounds. The method used for generating the random values works best when :math:`a \leq \text{mean} \leq b`. NOTE: this 'tf' variant behaves closer to Tensorflow / JAX impl where the bounds [a, b] are applied when sampling the normal distribution with mean=0, std=1.0 and the result is subsquently scaled and shifted by the mean and std args. Args: tensor: an n-dimensional `torch.Tensor` mean: the mean of the normal distribution std: the standard deviation of the normal distribution a: the minimum cutoff value b: the maximum cutoff value Examples: >>> w = torch.empty(3, 5) >>> nn.init.trunc_normal_(w) rrN)r r!rrrr"r r r trunc_normal_tf_Fs .r$fan_innormalc Cst|\}}|dkr|}n"|dkr(|}n|dkr<||d}||}|dkrdt|t|ddn|dkrt"|jt|dWdn1s0Yn\|d krtd |}t|| |Wdn1s0Yntd |dS) Nr%fan_outZfan_avgr truncated_normalg۶%?)rr&uniformzinvalid distribution ) rr$rrr r!Znormal_r ValueError) rZscalemode distributionr%r'ZdenomZvarianceboundr r r variance_scaling_cs$   2 .r/cCst|ddddS)Nr%r()r,r-)r/)rr r r lecun_normal_|sr0)rrrr)rrrr)rr%r&) r rrZ torch.nn.initrrr#r$r/r0r r r r s #