a  dBI@s4ddlmZddlmZmZmZmZmZmZddl Z ddl m Z ddl m Z ddl mZddlmZddlmZmZmZdd lmZmZesd gngZesed g7Zerdd lmZdd lmZddlmZn Gddde ZdZdZdd gZesdd gZGdddeZ e e e e e dddZ!GdddeZ"dS))deepcopy)AnyListOptionalSequenceTupleUnionN)Tensor)Module)adaptive_avg_pool2d)Metric)_MATPLOTLIB_AVAILABLE_TORCH_FIDELITY_AVAILABLE_TORCH_GREATER_EQUAL_1_9)_AX_TYPE_PLOT_OUT_TYPE!FrechetInceptionDistance.__init__FrechetInceptionDistance.plot)FeatureExtractorInceptionV3)vassert))interpolate_bilinear_2d_like_tensorflow1xc@s eZdZdS)_FeatureExtractorInceptionV3N)__name__ __module__ __qualname__rr_/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/torchmetrics/image/fid.pyr%srFrechetInceptionDistancecsreZdZdZdeeeeeddfdd Zeddfdd Z e e e d fd d d Z e e d d dZ ZS)NoTrainInceptionV3z*Module that nevers leaves evaluation mode.N)name features_listfeature_extractor_weights_pathreturncst||||dSN)super__init__eval)selfrr r! __class__rrr%4szNoTrainInceptionV3.__init__)moder"cs tdS)z.Force network to always be in evaluation mode.F)r$train)r'r*r(rrr+>szNoTrainInceptionV3.train.)xr"cstt|o|jtjkdi|j}t|dr@||j n |tj }t ||j |j fdd}|dd}| |}||}||}||}d|vrt|ddd d d<|dt|d krtfd d |jDS||}||}||}d |vr\t|ddd d d <|d t|d kr\tfdd |jDS||}||}||}||}||}||}||}| |}d|vrt|ddd d d<|dt|d krtfdd |jDS|!|}|"|}|#|}|$|}t%|d}d|vrx|d<|dt|d krxtfdd |jDSd|vr|&|j'j(j)}|d<|dt|d krtfdd |jDS||j'j*+d }n |'|}|d<tfdd |jDS)aForward method of inception net. Copy of the forward method from this file: https://github.com/toshas/torch-fidelity/blob/master/torch_fidelity/feature_extractor_inceptionv3.py with a single line change regarding the casting of `x` in the beginning. Corresponding license file (Apache License, Version 2.0): https://github.com/toshas/torch-fidelity/blob/master/LICENSE.md z6Expecting image as torch.Tensor with dtype=torch.uint8_dtypeF)sizeZ align_cornersZ64)r0)Z output_sizerc3s|]}|VqdSr#r.0afeaturesrr bz=NoTrainInceptionV3._torch_fidelity_forward..Z192c3s|]}|VqdSr#rr2r5rrr7lr8Z768c3s|]}|VqdSr#rr2r5rrr7{r8r0Z2048c3s|]}|VqdSr#rr2r5rrr7r8Zlogits_unbiasedc3s|]}|VqdSr#rr2r5rrr7r8Zlogitsc3s|]}|VqdSr#rr2r5rrr7r8),rtorchZ is_tensordtypeuint8r copyhasattrtor-floatrZINPUT_IMAGE_SIZEZ Conv2d_1a_3x3Z Conv2d_2a_3x3Z Conv2d_2b_3x3Z MaxPool_1r squeezeremovelentupleZ Conv2d_3b_1x1Z Conv2d_4a_3x3Z MaxPool_2ZMixed_5bZMixed_5cZMixed_5dZMixed_6aZMixed_6bZMixed_6cZMixed_6dZMixed_6eZMixed_7aZMixed_7bZMixed_7cZAvgPoolflattenmmZfcZweightTZbias unsqueeze)r'r,Zremaining_featuresrr5r_torch_fidelity_forwardBsx  "                                 z*NoTrainInceptionV3._torch_fidelity_forwardcCs ||}|d|jddS)z8Forward pass of neural network with reshaping of output.rr1)rHZreshapeshape)r'r,outrrrforwards zNoTrainInceptionV3.forward)N)rrr__doc__strrrr%boolr+r rrHrK __classcell__rrr(rr1s Vr)mu1sigma1mu2sigma2r"cCsR||jdd}||}tj||jjdd}||d|S)arCompute adjusted version of `Fid Score`_. The Frechet Inception Distance between two multivariate Gaussians X_x ~ N(mu_1, sigm_1) and X_y ~ N(mu_2, sigm_2) is d^2 = ||mu_1 - mu_2||^2 + Tr(sigm_1 + sigm_2 - 2*sqrt(sigm_1*sigm_2)). Args: mu1: mean of activations calculated on predicted (x) samples sigma1: covariance matrix over activations calculated on predicted (x) samples mu2: mean of activations calculated on target (y) samples sigma2: covariance matrix over activations calculated on target (y) samples Returns: Scalar value of the distance between sets. r1dim)Zsquaresumtracer9ZlinalgZeigvalssqrtreal)rPrQrRrSr4bcrrr _compute_fidsr]cseZdZUdZdZeed<dZeed<dZeed<dZ e ed<e ed<e ed <e ed <e ed <e ed <e ed <e ed<d$e ee feeeddfdd Ze eddddZe dddZddfdd Ze eejfddfdd Zd%ee e ee feeed!d"d#ZZS)&ruKCalculate Fréchet inception distance (FID_) which is used to access the quality of generated images. .. math:: FID = \|\mu - \mu_w\|^2 + tr(\Sigma + \Sigma_w - 2(\Sigma \Sigma_w)^{\frac{1}{2}}) where :math:`\mathcal{N}(\mu, \Sigma)` is the multivariate normal distribution estimated from Inception v3 (`fid ref1`_) features calculated on real life images and :math:`\mathcal{N}(\mu_w, \Sigma_w)` is the multivariate normal distribution estimated from Inception v3 features calculated on generated (fake) images. The metric was originally proposed in `fid ref1`_. Using the default feature extraction (Inception v3 using the original weights from `fid ref2`_), the input is expected to be mini-batches of 3-channel RGB images of shape ``(3xHxW)``. If argument ``normalize`` is ``True`` images are expected to be dtype ``float`` and have values in the ``[0,1]`` range, else if ``normalize`` is set to ``False`` images are expected to have dtype ``uint8`` and take values in the ``[0, 255]`` range. All images will be resized to 299 x 299 which is the size of the original training data. The boolian flag ``real`` determines if the images should update the statistics of the real distribution or the fake distribution. This metric is known to be unstable in its calculatations, and we recommend for the best results using this metric that you calculate using `torch.float64` (default is `torch.float32`) which can be set using the `.set_dtype` method of the metric. .. note:: using this metrics requires you to have torch 1.9 or higher installed .. note:: using this metric with the default feature extractor requires that ``torch-fidelity`` is installed. Either install as ``pip install torchmetrics[image]`` or ``pip install torch-fidelity`` As input to ``forward`` and ``update`` the metric accepts the following input - ``imgs`` (:class:`~torch.Tensor`): tensor with images feed to the feature extractor with - ``real`` (:class:`~bool`): bool indicating if ``imgs`` belong to the real or the fake distribution As output of `forward` and `compute` the metric returns the following output - ``fid`` (:class:`~torch.Tensor`): float scalar tensor with mean FID value over samples Args: feature: Either an integer or ``nn.Module``: - an integer will indicate the inceptionv3 feature layer to choose. Can be one of the following: 64, 192, 768, 2048 - an ``nn.Module`` for using a custom feature extractor. Expects that its forward method returns an ``(N,d)`` matrix where ``N`` is the batch size and ``d`` is the feature size. reset_real_features: Whether to also reset the real features. Since in many cases the real dataset does not change, the features can be cached them to avoid recomputing them which is costly. Set this to ``False`` if your dataset does not change. kwargs: Additional keyword arguments, see :ref:`Metric kwargs` for more info. Raises: ValueError: If torch version is lower than 1.9 ModuleNotFoundError: If ``feature`` is set to an ``int`` (default settings) and ``torch-fidelity`` is not installed ValueError: If ``feature`` is set to an ``int`` not in [64, 192, 768, 2048] TypeError: If ``feature`` is not an ``str``, ``int`` or ``torch.nn.Module`` ValueError: If ``reset_real_features`` is not an ``bool`` Example: >>> import torch >>> _ = torch.manual_seed(123) >>> from torchmetrics.image.fid import FrechetInceptionDistance >>> fid = FrechetInceptionDistance(feature=64) >>> # generate two slightly overlapping image intensity distributions >>> imgs_dist1 = torch.randint(0, 200, (100, 3, 299, 299), dtype=torch.uint8) >>> imgs_dist2 = torch.randint(100, 255, (100, 3, 299, 299), dtype=torch.uint8) >>> fid.update(imgs_dist1, real=True) >>> fid.update(imgs_dist2, real=False) >>> fid.compute() tensor(12.7202) Fhigher_is_betteris_differentiablefull_state_updategplot_lower_boundreal_features_sumreal_features_cov_sumreal_features_num_samplesfake_features_sumfake_features_cov_sumfake_features_num_samples inceptionTN)featurereset_real_features normalizekwargsr"c  stjfi|tstdt|trp|}ts8tdd}||vrZtd|d|dtdt |gd|_ n>t|t r||_ t j d d d t jd }| |jd }ntdt|tstd||_t|tstd||_||f}|jdt |dd|jdt |dd|jdt d dd|jdt |dd|jdt |dd|jdt d dddS)NzQFrechetInceptionDistance metric requires that PyTorch is version 1.9.0 or higher.zFrechetInceptionDistance metric requires that `Torch-fidelity` is installed. Either install as `pip install torchmetrics[image]` or `pip install torch-fidelity`.)@iriz3Integer input to argument `feature` must be one of z , but got .zinception-v3-compat)rr r)r0+rs)r:r1z'Got unknown input to argument `feature`z4Argument `reset_real_features` expected to be a boolz*Argument `normalize` expected to be a boolrbrW)Zdist_reduce_fxrcrdrerfrg)r$r%r ValueError isinstanceintrModuleNotFoundErrorrrMrhr r9randintr;rI TypeErrorrNrkrlZ add_statezerosdoubleZtensorlong) r'rjrkrlrmZ num_featuresZvalid_int_inputZ dummy_imageZ mx_nb_feetsr(rrr%sB    r)imgsrZr"cCs|jr|dn|}||}|j|_|}|dkrF|d}|r|j|j dd7_|j | |7_ |j |jd7_ nB|j|j dd7_|j| |7_|j|jd7_dS)z)Update the state with extracted features.rqr0rrTN)rlbyterhr: orig_dtyper{rUrGrbrWrctrErdrIrerfrg)r'r}rZr6rrrupdateEs   zFrechetInceptionDistance.update)r"cCs|jdks|jdkrtd|j|jd}|j|jd}|j|j||}||jd}|j |j||}||jd}t | d|| d| |j S)zWCalculate FID score based on accumulated extracted features from the two distributions.rVzVMore than one sample is required for both the real and fake distributed to compute FIDrr0)rdrg RuntimeErrorrbrGrercrrErfr]r@r>r)r'Z mean_realZ mean_fakeZ cov_real_numZcov_realZ cov_fake_numZcov_fakerrrcomputeWsz FrechetInceptionDistance.computecsP|jsBt|j}t|j}t|j}t||_||_||_n tdS)zReset metric states.N)rkrrbrcrdr$reset)r'rbrcrdr(rrrds    zFrechetInceptionDistance.resetr )dst_typer"cs$t|}t|jtr ||j_|S)zTransfer all metric state to specific dtype. Special version of standard `type` method. Arguments: dst_type: the desired type as ``torch.dtype`` or string )r$ set_dtyperurhrr-)r'rrJr(rrrqs  z"FrechetInceptionDistance.set_dtype)valaxr"cCs |||S)ajPlot a single or multiple values from the metric. Args: val: Either a single result from calling `metric.forward` or `metric.compute` or a list of these results. If no value is provided, will automatically call `metric.compute` and plot that result. ax: An matplotlib axis object. If provided will add plot to that axis Returns: Figure and Axes object Raises: ModuleNotFoundError: If `matplotlib` is not installed .. plot:: :scale: 75 >>> # Example plotting a single value >>> import torch >>> from torchmetrics.image.fid import FrechetInceptionDistance >>> imgs_dist1 = torch.randint(0, 200, (100, 3, 299, 299), dtype=torch.uint8) >>> imgs_dist2 = torch.randint(100, 255, (100, 3, 299, 299), dtype=torch.uint8) >>> metric = FrechetInceptionDistance(feature=64) >>> metric.update(imgs_dist1, real=True) >>> metric.update(imgs_dist2, real=False) >>> fig_, ax_ = metric.plot() .. plot:: :scale: 75 >>> # Example plotting multiple values >>> import torch >>> from torchmetrics.image.fid import FrechetInceptionDistance >>> imgs_dist1 = lambda: torch.randint(0, 200, (100, 3, 299, 299), dtype=torch.uint8) >>> imgs_dist2 = lambda: torch.randint(100, 255, (100, 3, 299, 299), dtype=torch.uint8) >>> metric = FrechetInceptionDistance(feature=64) >>> values = [ ] >>> for _ in range(3): ... metric.update(imgs_dist1(), real=True) ... metric.update(imgs_dist2(), real=False) ... values.append(metric.compute()) ... metric.reset() >>> fig_, ax_ = metric.plot(values) )Z_plot)r'rrrrrplot}s0r)riTF)NN)rrrrLr^rN__annotations__r_r`rar?r r rrvrr%rrrrMr9r:rrrrrrrOrrr(rrs> M     3   )#r<rtypingrrrrrrr9r Ztorch.nnr Ztorch.nn.functionalr Ztorchmetrics.metricr Ztorchmetrics.utilities.importsr rrZtorchmetrics.utilities.plotrrZ__doctest_skip__Z,torch_fidelity.feature_extractor_inceptionv3rrZtorch_fidelity.helpersrZ,torch_fidelity.interpolate_compat_tensorflowrrr]rrrrrs0         m