a  d? @sddlZddlZddlmZddlmZmZmZmZm Z ddl Z ddl m Z m Z ddl mZddlmZmZddd d Zesd gZn dd lmZeee jjjd ddZGddde j jZGddde j jZGddde j jZ d?e ee dddZ!d@e ee"dfe dddZ#dAe e$e d d!d"Z%dBe e"e d$d%d&Z&Gd'd(d(e jZ'Gd)d*d*e jZ(Gd+d,d,e jZ)Gd-d.d.e)Z*e eed/d0d1Z+e e e jeee e e"e ffd2d3d4Z,dCe e e e"fed6e d7d8d9Z-dDe e ed<ed6ee d=d>d Z.dS)EN) namedtuple)List NamedTupleOptionalTupleUnion)Tensornn)Literal)_TORCHVISION_AVAILABLE_TORCHVISION_GREATER_EQUAL_0_13ZSqueezeNet1_1_WeightsZAlexNet_WeightsZ VGG16_Weights) squeezenet1_1alexnetvgg16)learned_perceptual_image_patch_similarity)models)net pretrainedreturncCsRtr<|r(tt|ttt|jdj}qNtt|ddj}ntt||dj}|S)zGet torchvision network. Args: net: Name of network pretrained: If pretrained weights should be used )weightsN)r)r getattrtv _weight_mapZ IMAGENET1K_V1features)rrpretrained_featuresrl/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/torchmetrics/functional/image/lpips.py_get_net0s  rcs<eZdZdZd eeddfdd Zeedd d ZZ S) SqueezeNetzSqueezeNet implementation.FTN requires_gradrrc sttd|}d|_g}tdtddtddtddtddtddtdd g}|D]6}tj}|D]}|t |||qn| |q\t ||_ |s| D] } d | _qdS) Nr  F)super__init__rN_slicesrangetorchr Sequential add_modulestrappend ModuleListslices parametersr ) selfr rrr3Zfeature_rangesZ feature_rangeseqiparam __class__rrr*Es  :    zSqueezeNet.__init__xrcCs8tdgd}g}|jD]}||}||q||S)Process input.squeeze_output)relu1relu2relu3relu4relu5Zrelu6Zrelu7)rr3r1)r5r<r>ZrelusZslice_rrrforwardWs   zSqueezeNet.forward)FT __name__ __module__ __qualname____doc__boolr*rrrD __classcell__rrr9rrBsrcs<eZdZdZd eeddfdd Zeedd d ZZ S) AlexnetzAlexnet implementation.FTNrcs2ttd|}tj|_tj|_tj|_tj|_ tj|_ d|_ t dD]}|j t|||q^t ddD]}|j t|||qt ddD]}|j t|||qt ddD]}|j t|||qt ddD]}|j t|||q|s.|D] }d|_q dS)Nrr#r"r$r%r'Fr)r*rr-r r.slice1slice2slice3slice4slice5r+r,r/r0r4r )r5r rZalexnet_pretrained_featuresr<r8r9rrr*es*         zAlexnet.__init__r;c Csd||}|}||}|}||}|}||}|}||}|}tdgd}||||||S)r=alexnet_outputs)r?r@rArBrCrNrOrPrQrRr) r5r<hZh_relu1Zh_relu2Zh_relu3Zh_relu4Zh_relu5rSrrrrD}s     zAlexnet.forward)FTrErrr9rrLbsrLcs<eZdZdZd eeddfdd Zeedd d ZZ S) Vgg16zVgg16 implementation.FTNrcs2ttd|}tj|_tj|_tj|_tj|_ tj|_ d|_ t dD]}|j t|||q^t ddD]}|j t|||qt ddD]}|j t|||qt ddD]}|j t|||qt ddD]}|j t|||q|s.|D] }d|_q dS) Nrr# FrM)r5r rZvgg_pretrained_featuresr<r8r9rrr*s*         zVgg16.__init__r;c Csd||}|}||}|}||}|}||}|}||}|}tdgd}||||||S)r= vgg_outputs)Zrelu1_2Zrelu2_2Zrelu3_3Zrelu4_3Zrelu5_3rT) r5r<rUZ h_relu1_2Z h_relu2_2Z h_relu3_3Z h_relu4_3Z h_relu5_3r\rrrrDs     z Vgg16.forward)FTrErrr9rrVsrVT)in_tenskeepdimrcCs|jddg|dS)z1Spatial averaging over heigh and width of images.r"r^)mean)r]r^rrrspatial_averagesrb@rd.)r]out_hwrcCstj|ddd|S)z+Upsample input with bilinear interpolation.bilinearF)sizemode align_corners)r ZUpsample)r]rerrrupsamsrj绽|=)in_featepsrcCs&ttj|dddd}|||S)zNormalize tensors.r"T)Zdimr^)r-sqrtsum)rlrmZ norm_factorrrrnormalize_tensorsrqrd)r<rgrcCsN|jd|kr4|jd|kr4tjjj|||fddStjjj|||fdddS)zlhttps://github.com/toshas/torch-fidelity/blob/master/torch_fidelity/sample_similarity_lpips.py#L127C22-L132.area)rhrfF)rhri)shaper-r Z functionalZ interpolate)r<rgrrr resize_tensorsrvcs6eZdZdZddfdd ZeedddZZS) ScalingLayerzScaling layer.N)rcsbt|jdtgddddddfdd|jdtgddddddfdddS)Nshift)gQgI +gMbȿF) persistentscale)gZd;O?gy&1?g?)r)r*Zregister_bufferr-r)r5r9rrr*s *zScalingLayer.__init__)inprcCs||j|jSr=)rxrz)r5r{rrrrDszScalingLayer.forward)rFrGrHrIr*rrDrKrrr9rrwsrwcs>eZdZdZd eeeddfdd Zeedd d ZZ S) NetLinLayerz,A single linear layer which does a 1x1 conv.rnFN)chn_inchn_out use_dropoutrc sHt|rtgng}|tj||dddddg7}tj||_dS)NrnrF)ZstridepaddingZbias)r)r*r ZDropoutZConv2dr.model)r5r~rrZlayersr9rrr*s  zNetLinLayer.__init__r;cCs ||Sr|)r)r5r<rrrrDszNetLinLayer.forward)rnF) rFrGrHrIintrJr*rrDrKrrr9rr}s r}c sneZdZd eedeeeeeeeeedd fdd Zd e e eee e e e e e ffd d d Z ZS)_LPIPSTalexFNrvggsqueeze) rrspatial pnet_rand pnet_tuner model_path eval_moderesizerc st||_||_||_||_| |_t|_|jdvrJt } gd|_ n2|jdkrdt } gd|_ n|jdkr|t } gd|_ t |j |_| |j |jd|_t|j d|d |_t|j d |d |_t|j d |d |_t|j d |d |_t|j d |d |_|j|j|j|j|jg|_|jdkrdt|j d|d |_t|j d|d |_|j|j|jg7_t|j|_|r|durtjtjt |jdd|d}|j!t"j#|dddd|r|$dS)aInitializes a perceptual loss torch.nn.Module. Args: pretrained: This flag controls the linear layers should be pretrained version or random net: Indicate backbone to use, choose between ['alex','vgg','squeeze'] spatial: If input should be spatial averaged pnet_rand: If backbone should be random or use imagenet pre-trained weights pnet_tune: If backprop should be enabled use_dropout: If dropout layers should be added model_path: Model path to load pretained models from eval_mode: If network should be in evaluation mode resize: If input should be resized to this size )rr)rdrr)rdrrr)rdrrrrrr)rr r)rrnr"r_rWr#Nz..z lpips_models/z.pthcpu)Z map_locationF)strict)%r)r*Z pnet_typerrrrrw scaling_layerrVZchnsrLrlenLrr}Zlin0Zlin1Zlin2Zlin3Zlin4linsZlin5Zlin6r r2ospathabspathjoininspectgetfileZload_state_dictr-loadeval) r5rrrrrrrrrnet_typer9rrr*sJ          z_LPIPS.__init__)in0in1 retperlayer normalizerc CsR|rd|d}d|d}||||}}|jdurXt||jd}t||jd}|j||j|}}iii} } } t|jD]>} t|| t|| | | <| | <| | | | d| | <qg} t|jD]\} |jr| t |j | | | t |j dddq| t|j | | | ddqt| }|rN|| fS|S)Nr"rn)rg)reTr`)rrrvrrDr,rrqrr1rjrtuplerurbrp)r5rrrrZ in0_inputZ in1_inputZouts0Zouts1Zfeats0Zfeats1ZdiffskkresvalrrrrD5s*   "0"z_LPIPS.forward) TrFFFTNTN)FF)rFrGrHrJr rr0rr*rrrrrDrKrrr9rrs4Hrcs(eZdZdZeddfdd ZZS) _NoTrainLpipsz8Wrapper to make sure LPIPS never leaves evaluation mode.)rhrcs tdS)z.Force network to always be in evaluation mode.F)r)train)r5rhr9rrr[sz_NoTrainLpips.train)rFrGrHrIrJrrKrrr9rrXsr)imgrrcCsD|r|dko&|dkn |dk}|jdkoB|jddkoB|S)z1Check that input is a valid image to the network.g?grrrWrnr_)maxminndimru)rrZ value_checkrrr _valid_img`s(r)img1img2rrrc Cst||rt||shtd|jd|jd||gd||gd|rXddgnddgd ||||d }||jdfS) NzeExpected both input arguments to be normalized tensors with shape [N, 3, H, W]. Got input with shape z and z and values in range z+ when all values are expected to be in the rrnrrz range.)r)r ValueErrorrurrr)rrrrlossrrr _lpips_updatefs rra)rpra) sum_scorestotal reductionrcCs|dkr||S|S)Nrar)rrrrrr_lpips_computersrrFr)rrrrrrcCs,t|d}t||||\}}t|||S)aThe Learned Perceptual Image Patch Similarity (`LPIPS_`) calculates the perceptual similarity between two images. LPIPS essentially computes the similarity between the activations of two image patches for some pre-defined network. This measure has been shown to match human perception well. A low LPIPS score means that image patches are perceptual similar. Both input image patches are expected to have shape ``(N, 3, H, W)``. The minimum size of `H, W` depends on the chosen backbone (see `net_type` arg). Args: img1: first set of images img2: second set of images net_type: str indicating backbone network type to use. Choose between `'alex'`, `'vgg'` or `'squeeze'` reduction: str indicating how to reduce over the batch dimension. Choose between `'sum'` or `'mean'`. normalize: by default this is ``False`` meaning that the input is expected to be in the [-1,1] range. If set to ``True`` will instead expect input to be in the ``[0,1]`` range. Example: >>> import torch >>> _ = torch.manual_seed(123) >>> from torchmetrics.functional.image.lpips import learned_perceptual_image_patch_similarity >>> img1 = (torch.rand(10, 3, 100, 100) * 2) - 1 >>> img2 = (torch.rand(10, 3, 100, 100) * 2) - 1 >>> learned_perceptual_image_patch_similarity(img1, img2, net_type='squeeze') tensor(0.1008, grad_fn=) )r)rrrrp)rrrrrrrrrrrrvs" )T)rc)rk)rd)ra)rraF)/rr collectionsrtypingrrrrrr-rr Ztyping_extensionsr Ztorchmetrics.utilities.importsr r rZ__doctest_skip__Z torchvisionrrr0rJmodules containerr.rModulerrLrVrbrrjfloatrqrvrwr}rrrrrrrrrrsP    ++ k( "