a d^]@sUddlmZmZmZmZmZddlZddlmZddl mm Z ddl m Z ddlmZmZddlmZddlmZdZeed<d d gZeeed <gd Zeeed <gdZeeed<eeedZeeeefed<dddDZeeefed<d1eeeejfdddZ eeejdddZ!Gdddej"Z#Gdd d ej"Z$Gd!d"d"ej"Z%e&ejd#d$d%Z'Gd&d'd'ej"Z(Gd(d)d)ej"Z)Gd*d+d+ej"Z*eeed,d-d.Z+Gd/d0d0ej"Z,dS)2)CallableDictListTupleUnionN)pi)GaussianBlur2dSpatialGradient)cart2pol)create_meshgridg;f?sqrt2gpu?g "? COEFFS_N1_K1)"g#[?x@ٔ+?K '"? COEFFS_N2_K8)rrrgˈfF? COEFFS_N3_K8)xyrhophithetaCOEFFScCsi|]}|d|dqS)zChttps://github.com/manyids2/mkd_pytorch/raw/master/mkd_pytorch/mkd-z-64.pth).0krr[/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/kornia/feature/mkd.py sr)cartpolarconcaturls ) patch_sizereturncCs^t||dd}|ddddddf}|ddddddf}t||\}}||||d}|S)z1Get cartesian and polar parametrizations of grid.TheightwidthZnormalized_coordinatesrN)xyrhophi)r r )r!kgridr'r(r)r*Z grid_dictrrr get_grid_dicts r,)d1d2r"cCs`tj||dgtjd}t|D]:}t|D],}|||||df<|||||df<q,q |S)z&Get order for doing kronecker product.)Zdtyperr&)torchzerosint64range)r-r. kron_orderijrrrget_kron_order#s   r7csHeZdZdZddfdd ZejejdddZedd d Z Z S) MKDGradientsaModule, which computes gradients of given patches, stacked as [magnitudes, orientations]. Given gradients $g_x$, $g_y$ with respect to $x$, $y$ respectively, - $\mathbox{mags} = $\sqrt{g_x^2 + g_y^2 + eps}$ - $\mathbox{oris} = $\mbox{tan}^{-1}(\nicefrac{g_y}{g_x})$. Args: patch_size: Input patch size in pixels. Returns: gradients of given patches. Shape: - Input: (B, 1, patch_size, patch_size) - Output: (B, 2, patch_size, patch_size) Example: >>> patches = torch.rand(23, 1, 32, 32) >>> gradient = MKDGradients() >>> g = gradient(patches) # 23x2x32x32 Nr"cs$td|_tdddd|_dS)N:0yE>diffr&F)modeorder normalized)super__init__epsr gradself __class__rrr@Ds zMKDGradients.__init__r'r"cCst|tjstdt|t|jdks>> oris = torch.rand(23, 1, 32, 32) >>> vm = VonMisesKernel(patch_size=32, ... coeffs=[0.14343168, ... 0.268285, ... 0.21979234]) >>> emb = vm(oris) # 23x7x32x32 N)r!coeffsr"cst||_t|}|d|t|d}||_d|d|_t dd||g}t |d}| ddd}t d|dg}t ||d|d<t |dd||dd<| ddd}|d||d||d|dS)Nr_r&r/emb0frangeweights)r?r@r!r0Ztensorregister_bufferrPndonesZarangeZreshaper1sqrt)rDr!r_Zb_coeffsrerarbrcrErrr@qs"      zVonMisesKernel.__init__rGcCst|tjstdt|t|jdkr:|jddkrJtd|jtj tj|j }| | | dddd}|j ||}t|}t|}tj|||gdd}|j|}|S)NrHrIr&rJrrK)rLr0rMrNrOrPrQrRjitannotateratorepeatsizerbcossinrSrc)rDr'rarbemb1emb2Z embeddingrrrrUs    zVonMisesKernel.forwardr9cCsT|jjddt|jddt|jddt|jddt|jdS)N( patch_size=, zn=zd=zcoeffs=))rFrWr\r!rerfr_rCrrrrXs8     zVonMisesKernel.__repr__)rWrYrZr[intrlisttupler@r0rMrUr\rXr]rrrErr^Zsr^csbeZdZdZdeeddfdd Zejejdd d Z ejejd d d Z e dddZ Z S)EmbedGradientsa<Module that computes gradient embedding, weighted by sqrt of magnitudes of given patches. Args: patch_size: Input patch size in pixels. relative: absolute or relative gradients. Returns: Gradient embedding. Shape: - Input: (B, 2, patch_size, patch_size) - Output: (B, 7, patch_size, patch_size) Examples: >>> grads = torch.rand(23, 2, 32, 32) >>> emb_grads = EmbedGradients(patch_size=32, ... relative=False) >>> emb = emb_grads(grads) # 23x7x32x32 r FN)r!relativer"cst||_||_d|_t|tdd|_t||dd}t |dddddddf|dddddddf\}}| d|dS) Nr:rr!r_Tr#rr&r*) r?r@r!rzrAr^rkernelr r rd)rDr!rzr+_r*rErrr@s >zEmbedGradients.__init__)magsr"cCst||j}|S)z@Embed square roots of magnitudes with eps for numerical reasons.)r0rhrA)rDr~rrremb_magsszEmbedGradients.emb_mags)gradsr"cCst|tjstdt|t|jdks.cSs&i|]\}}||ddqS)r) unsqueezefloatrrrrrrr{r&N) rr rwkeysrQitemsr^rZsqueezer7rf index_select) kernel_typerZcoeffs_Zparams_rr!Z grids_normedZvm_aZvm_bZemb_aZemb_br4Zspatial_kernelrrrspatial_kernel_embeddings$  0rcseZdZdZdeeeeeddfdd Zee j d d d Z e e j e j fd ddZ e j e j dddZed ddZZS)ExplicitSpacialEncodingaModule that computes explicit cartesian or polar embedding. Args: kernel_type: Parametrization of kernel ``'polar'`` or ``'cart'``. fmap_size: Input feature map size in pixels. in_dims: Dimensionality of input feature map. do_gmask: Apply gaussian mask. do_l2: Apply l2-normalization. Returns: Explicit cartesian or polar embedding. Shape: - Input: (B, in_dims, fmap_size, fmap_size) - Output: (B, out_dims, fmap_size, fmap_size) Example: >>> emb_ori = torch.rand(23, 7, 32, 32) >>> ese = ExplicitSpacialEncoding(kernel_type='polar', ... fmap_size=32, ... in_dims=7, ... do_gmask=True, ... do_l2=True) >>> desc = ese(emb_ori) # 23x175x32x32 rr TN)r fmap_sizein_dimsdo_gmaskdo_l2r"c st|dvr t|d||_||_||_||_||_t||_ d|_ t |j|j }|jrz|j dd|_ ||j }| d|d|jd|_|j|j|_|j|_|\}}| d|| d|dS) N)rrz" is not valid, use polar or cart).r)sigmaembrrqidx1)r?r@NotImplementedErrorrrrrrr,gridgmaskr get_gmaskrdrrQd_embout_dimsodims init_kron) rDrrrrrrrqrrErrr@(s*      z ExplicitSpacialEncoding.__init__)rr"cCs6|jd|jd}td|d|d}|S)zCompute Gaussian mask.r)r`r/)rmaxr0exp)rDrZnorm_rhorrrrrPsz!ExplicitSpacialEncoding.get_gmaskr9cCsNt|j|j}tjtj|j}t|d|dddf}||dddffS)z3Initialize helper variables to calculate kronecker.r&Nr) r7rrr0rirjrMrr)rDZkronZ_embrqrrrrVsz!ExplicitSpacialEncoding.init_kronrGcCst|tjstdt|t|jdk|jd|jkBsTtd|jd|jtj tj|j }t |d|}||j }|jdd}|jrtj|dd}|S)NrHrIr&z!Invalid input shape, we expect Bxz xHxW. Got: )r/rK)rLr0rMrNrOrPrQrrRrirjrrrqsumrF normalize)rDr'rrpoutputrrrrU]s   zExplicitSpacialEncoding.forwardcCsx|jjddt|jddt|jddt|jddt|jddt|jddt|jd S) Nrr kernel_type=rtz fmap_size=in_dims=z out_dims=z do_gmask=zdo_l2=ru) rFrWr\rrrrrrrCrrrrXjsP     z ExplicitSpacialEncoding.__repr__)rr rTT)rWrYrZr[r\rvrr@rr0rMrrrrUrXr]rrrErr s$( rc seZdZdZdeeeeeeejffdfe e e e ddfdd Z eeeeejffdd d d Z dd d dZ dd ddZdd ddZdd ddZejejdddZed ddZZS) WhiteningaGModule, performs supervised or unsupervised whitening. This is based on the paper "Understanding and Improving Kernel Local Descriptors". See :cite:`mukundan2019understanding` for more details. Args: xform: Variant of whitening to use. None, 'lw', 'pca', 'pcaws', 'pcawt'. whitening_model: Dictionary with keys 'mean', 'eigvecs', 'eigvals' holding torch.Tensors. in_dims: Dimensionality of input descriptors. output_dims: (int) Dimensionality reduction. keval: Shrinkage parameter. t: Attenuation parameter. Returns: l2-normalized, whitened descriptors. Shape: - Input: (B, in_dims, fmap_size, fmap_size) - Output: (B, out_dims, fmap_size, fmap_size) Examples: >>> descs = torch.rand(23, 238) >>> whitening_model = {'pca': {'mean': torch.zeros(238), ... 'eigvecs': torch.eye(238), ... 'eigvals': torch.ones(238)}} >>> whitening = Whitening(xform='pcawt', ... whitening_model=whitening_model, ... in_dims=238, ... output_dims=128, ... keval=40, ... t=0.7) >>> wdescs = whitening(descs) # 23x128 (ffffff?N)xformwhitening_modelr output_dimskevaltr"cst||_||_||_||_d|_t||}||_t j t |dd|_ t j t |ddd|fdd|_t j t |d|dd|_|dur||dS)NrT)Z requires_grad)r?r@rrrrpvalminrnn Parameterr0r1meanZeyeevecsrgevalsload_whitening_parameters)rDrrrrrrrErrr@s  &zWhitening.__init__)rr"cCs|jdkrdnd}||}|d|j_|dddd|jf|j_|dd|j|j_|j|j|j|j d}||jdS)NlwpcarZeigvecsZeigvals)rrZpcawspcawt) rrdatarrr _modify_pca _modify_lw _modify_pcaws _modify_pcawt)rDralgoZwh_modelZ modificationsrrrrs z#Whitening.load_whitening_parametersr9cCs d|_dS)zModify powerlaw parameter.g?N)rrCrrrrszWhitening._modify_pcacCsdS)zNo modification required.NrrCrrrrszWhitening._modify_lwcCs>|j|j}d||j|}|jtt|d|j_dS)zShrinkage for eigenvalues.r&N)rrrr0diagpowr)rDalpharrrrrs zWhitening._modify_pcawscCs,d|j}|jtt|j||j_dS)zAttenuation for eigenvalues.rN)rrr0rrrr)rDmrrrrs zWhitening._modify_pcawtrGcCs|t|tjstdt|t|jdks>> patches = torch.rand(23, 1, 32, 32) >>> mkd = MKDDescriptor(patch_size=32, ... kernel_type='concat', ... whitening='pcawt', ... training_set='liberty', ... output_dims=128) >>> desc = mkd(patches) # 23x128 r rrlibertyrNr!r whitening training_setrr"csDt||_||_||_||_d|d|_td|j|jfd|_t |_ d}d}|jdkrh||gn|jg|_ d|_ |d |d i}i|_ |j D]H} t||| d } t| || jjd } t| | |j | <|j | j 7_ qt||j |_|jdur8d d} tjjt|j| d} | |}t|||j |jd|_|j|_ |dS)Nffffff?@r replicaterrrrTFr!rzrrrcSs|SrVrZstoragelocrrrFrz(MKDDescriptor.__init__..Z map_locationrr)r?r@r!rrrrr smoothingr8 gradientsparametrizationsrfeatsryrr|rfr Sequentialrrr0hubload_state_dict_from_urlrrwhitening_layereval)rDr!rrrrZpolar_sZcart_sZrelative_orientationsparametrizationZgradient_embeddingZspatial_encoding storage_fcnwhitening_modelsrrErrr@s>      zMKDDescriptor.__init__)patchesr"cCst|tjstdt|t|jdks.r)r0rrr)rrrrrrrrload_whitening_modelsrcsFeZdZdZdeeeeeddfd d Zejejd d d Z Z S)SimpleKDz+Example to write custom Kernel Descriptors.r rrrrNrc st|dk}d|d}||_td||fd}t} t||d} t||| jjd} t |t ||| j |d} t || | | | |_dS) Nrrrrrrrr)r?r@r!rr8ryrr|rfrrrrrr) rDr!rrrrrzrrrZoriZeseZwhrErrr@s   zSimpleKD.__init__rGcCs ||SrV)rrrrrrUszSimpleKD.forward)r rrrr) rWrYrZr[rvr\r@r0rMrUr]rrrErrsr)r )-typingrrrrrr0Ztorch.nnrZtorch.nn.functionalZ functionalrZkornia.constantsrZkornia.filtersrr Zkornia.geometry.conversionsr Z kornia.utilsr r r__annotations__r rrrr\rrvrMr,r7Moduler8r^rydictrrrrrrrrrrs6        -SAv}