a dAN@s@ddlZddlmZmZddlZddlmmZddl m Z m Z m Z ddl mZddlmZddlmZmZejejddd Zejejdd d Zejejdd d ZejejejdddZd@ejeejeejejdddZejeeejfejdddZdAejeejdddZejejdddZdBejeejdd d!ZdCejed"d#d$Z ejejejd%d&d'Z!ejejejd%d(d)Z"dDejejeejd+d,d-Z#dEejejee$ejd/d0d1Z%dFejejee$ejd/d2d3Z&dGejejeejd4d5d6Z'ejd7d8d9Z(ejd:d;d<Z)ejejejd=d>d?Z*dS)HN)OptionalUnion)angle_to_rotation_matrixconvert_points_from_homogeneousrad2deg)transform_points)pyrdown)KORNIA_CHECK_LAFKORNIA_CHECK_SHAPE)LAFreturncCstt|d}|dddddf|dddddf|dddddf|dddddf|}|S)aRReturn a scale of the LAFs. Args: LAF: tensor [BxNx2x3] or [BxNx2x2]. Returns: tensor BxNx1x1. Shape: - Input: :math: `(B, N, 2, 3)` - Output: :math: `(B, N, 1, 1)` Example: >>> input = torch.ones(1, 5, 2, 3) # BxNx2x3 >>> output = get_laf_scale(input) # BxNx1x1 g|=.r)r abssqrt)r epsoutr[/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/kornia/feature/laf.py get_laf_scale s\rcCst||d}|S)aKReturn a center (keypoint) of the LAFs. Args: LAF: tensor [BxNx2x3]. Returns: tensor BxNx2. Shape: - Input: :math: `(B, N, 2, 3)` - Output: :math: `(B, N, 2)` Example: >>> input = torch.ones(1, 5, 2, 3) # BxNx2x3 >>> output = get_laf_center(input) # BxNx2 .r)r )r rrrrget_laf_center$srcCs*t|t|d|d}t|dS)asReturn orientation of the LAFs, in degrees. Args: LAF: (torch.Tensor): tensor [BxNx2x3]. Returns: torch.Tensor: tensor BxNx1 . Shape: - Input: :math: `(B, N, 2, 3)` - Output: :math: `(B, N, 1)` Example: >>> input = torch.ones(1, 5, 2, 3) # BxNx2x3 >>> output = get_laf_orientation(input) # BxNx1 ).rr ).rr)r torchatan2r unsqueeze)r Z angle_radrrrget_laf_orientation:sr)r angles_degreesr cCst||jdd\}}t|||dd}tjtt|||ddddddddf||||ddddddddfgdd||dd}|S)a Change the orientation of the LAFs. Args: LAF: tensor [BxNx2x3]. angles: tensor BxNx1, in degrees. Returns: tensor [BxNx2x3]. Shape: - Input: :math: `(B, N, 2, 3)`, `(B, N, 1)` - Output: :math: `(B, N, 2, 3)` Nrdim)r shaperviewrcatbmm make_upright)r rBNZrotmatZlaf_outrrrset_laf_orientationPs\r()xyscaleorir c Cst|gd|j}|j}|jdd\}}|durJtj||dd||d}|durftj||d||d}t|gdt|gdtjt| d| dgdd }t ||}|S) a@Return orientation of the LAFs, in radians. Useful to create kornia LAFs from OpenCV keypoints. Args: xy: tensor [BxNx2]. scale: tensor [BxNx1x1]. If not provided, scale = 1 is assumed ori: tensor [BxNx1]. If not provided orientation = 0 is assumed Returns: tensor BxNx2x3. )r&r'2Nrr )devicedtype)r&r'1r/)r&r'r/rr) r r-r.r!roneszerosr#rsqueezer scale_laf) r)r*r+r-r.r&r'Z unscaled_laflafrrrlaf_from_center_scale_origs " r5)r4 scale_coefr cCst|tur,t|tjur,tdt|t||ddddddddf}tj|||ddddddddfgddS)a[Multiplies region part of LAF ([:, :, :2, :2]) by a scale_coefficient. So the center, shape and orientation of the local feature stays the same, but the region area changes. Args: laf: tensor [BxNx2x3] or [BxNx2x2]. scale_coef: broadcastable tensor or float. Returns: tensor BxNx2x3. Shape: - Input: :math:`(B, N, 2, 3)` - Input: :math:`(B, N,)` or () - Output: :math:`(B, N, 1, 1)` Example: >>> input = torch.ones(1, 5, 2, 3) # BxNx2x3 >>> scale = 0.5 >>> output = scale_laf(input, scale) # BxNx2x3 z1scale_coef should be float or torch.Tensor Got {}Nrrr)typefloatrTensor TypeErrorformatr r#)r4r6Zcenterless_lafrrrr3s  r3& .>)r4rr cCs$t|t|}|}t|dddddfd|dddddfd|}tj||t|gdd}tj|dddddf|dddddf|dddddf|dddddf||||gdd}tjtj||gdd|dddddfgdd}t||S)aRectify the affine matrix, so that it becomes upright. Args: laf: tensor of LAFs. eps : for safe division. Returns: tensor of same shape. Shape: - Input: :math:`(B, N, 2, 3)` - Output: :math:`(B, N, 2, 3)` Example: >>> input = torch.ones(1, 5, 2, 3) # BxNx2x3 >>> output = make_upright(input) # BxNx2x3 .rr rrrN)r rrrr# contiguous zeros_liker3)r4rZdetr*Zb2a2Zlaf1_ellZlaf2_ellZlaf_unit_scalerrrr%s> ^ 2r%)ellsr c Cst|}|dkr&td||\}}}|dkrNtd||dddf}t|}|dddf}|dddf||d}tj ||||gdd  ||dd } tj | |dd df ||dd gdd } | S) acConvert ellipse regions to LAF format. Ellipse (a, b, c) and upright covariance matrix [a11 a12; 0 a22] are connected by inverse matrix square root: A = invsqrt([a b; b c]). See also https://github.com/vlfeat/vlfeat/blob/master/toolbox/sift/vl_frame2oell.m Args: ells: tensor of ellipses in Oxford format [x y a b c]. Returns: tensor of ellipses in LAF format. Shape: - Input: :math:`(B, N, 5)` - Output: :math:`(B, N, 2, 3)` Example: >>> input = torch.ones(1, 10, 5) # BxNx5 >>> output = ellipse_to_laf(input) # BxNx2x3 rz/ellipse shape should be must be [BxNx5]. Got {}.rr<rrNr ) lensizer:r;rrrr>clampstackr"Zinverser#) r?Zn_dimsr&r'r Za11Za12Za22Za21Arrrrellipse_to_lafs   &*rG2)r n_ptsr c CsJt||\}}}}tjttddtj|ddt tddtj|ddt |ddgdd}tjt gd dd|gddd |||d}||j|j}t gd ddd ||dd}tj| ddd||j|jgdd}t||dddddd}t| |||dS)aWConvert LAFs to boundary points of the regions + center. Used for local features visualization, see visualize_laf function. Args: LAF: n_pts: number of points to output. Returns: tensor of boundary points. Shape: - Input: :math:`(B, N, 2, 3)` - Output: :math:`(B, N, n_pts, 2)` rrr rr)rJ?r)r rCrr#sinZlinspacemathpircosr0Ztensorr"expandtor-r.r$permuter) r rIr&r'_ptsZauxZHLAFZpts_hrrrlaf_to_boundary_pointss"" 8&, rU)r img_idxcCsJt|t|||dd}|ddd}|d|dfS)aReturn numpy array for drawing LAFs (local features). Args: LAF: n_pts: number of boundary points to output. Returns: tensor of boundary points. Shape: - Input: :math:`(B, N, 2, 3)` - Output: :math:`(B, N, n_pts, 2)` Examples: x, y = get_laf_pts_to_draw(LAF, img_idx) plt.figure() plt.imshow(kornia.utils.tensor_to_image(img[img_idx])) plt.plot(x, y, 'r') plt.show() r rr.r.r )r rUdetachrRcpunumpy)r rVrTZpts_nprrrget_laf_pts_to_drawsr\)r imagesr c Cstt||\}}}}t|}t|}t||}tdddd|j|j|}||d<||d<| ||S)aDe-normalize LAFs from scale to image scale. B,N,H,W = images.size() MIN_SIZE = min(H,W) [a11 a21 x] [a21 a22 y] becomes [a11*MIN_SIZE a21*MIN_SIZE x*W] [a21*MIN_SIZE a22*MIN_SIZE y*H] Args: LAF: images: images, LAFs are detected in. Returns: the denormalized lafs. Shape: - Input: :math:`(B, N, 2, 3)` - Output: :math:`(B, N, 2, 3)` r rrrrrrrrr r r rCr8minrr0rQr.r-Z expand_as r r]rShwZwfZhfZmin_sizeZcoefrrrdenormalize_laf4s $rec Cs|t||\}}}}t|}t|}t||}tdddd|j|j|}d||d<d||d<| ||S)aNormalize LAFs to [0,1] scale from pixel scale. See below: B,N,H,W = images.size() MIN_SIZE = min(H,W) [a11 a21 x] [a21 a22 y] becomes: [a11/MIN_SIZE a21/MIN_SIZE x/W] [a21/MIN_SIZE a22/MIN_SIZE y/H] Args: LAF: (torch.Tensor). images: (torch.Tensor) images, LAFs are detected in Returns: LAF: (torch.Tensor). Shape: - Input: :math:`(B, N, 2, 3)` - Output: :math:`(B, N, 2, 3)` r rrrKr^r_r`rbrrr normalize_lafUs $  rf )imgr PSr c Cst||\}}}}|\}}}}t||} tj| ||dd|||||gdd} d| ddddft|d | ddddf<d| dddd ft|d | dddd f<| S) zHelper function for affine grid generation. Args: img: image tensor of shape :math:`(B, CH, H, W)`. LAF: laf with shape :math:`(B, N, 2, 3)`. PS: patch size to be extracted. Returns: grid rrF) align_corners@.NrrKr )r rCreFZ affine_gridr"cloner8) rhr rir&r'rSchrcrdZ LAF_renormgridrrr'generate_patch_grid_from_normalized_LAFus  *44rpT)rhr4ri normalize_lafs_before_extractionr c Cst||rt||}n|}|\}}}}|\} } }}g} t| D]f} t|| | d|| | d||j} | tj || | d | d|||| dddqHt j | dd | | |||S)aExtract patches defined by LAFs from image tensor. No smoothing applied, huge aliasing (better use extract_patches_from_pyramid). Args: img: images, LAFs are detected in. laf: PS: patch size. normalize_lafs_before_extraction: if True, lafs are normalized to image size. Returns: patches with shape :math:`(B, N, CH, PS,PS)`. r rborderFZ padding_moderjr)r rfrCrangerprQr-appendrl grid_samplerPrr#r")rhr4rirqnlafrSrnrcrdr&r'rirorrrextract_patches_simples$  , ryc Cst||rt||}n|}|\}}}}|\}}} } dtt||t|} | } |} d}t ||||| |j  |j }t| d| d|kr| \}}} } t|D]}| ||k}|dkrq|dkd}t| ||d|||d|ddddf|}tj| ||d|d|| | |dd d }|||dddd|qt| } |d7}q|S) ajExtract patches defined by LAFs from image tensor. Patches are extracted from appropriate pyramid level. Args: laf: images: images, LAFs are detected in. PS: patch size. normalize_lafs_before_extraction: if True, lafs are normalized to image size. Returns: patches with shape :math:`(B, N, CH, PS,PS)`. rkrrrrr NrrFrs)r rfrCrrer8log2Zrelulongrr1rQr.r-rartr2sumr"rprlrvrPZmasked_scatter_r)rhr4rirqrwr&r'rSrnrcrdr*Zpyr_idxZcur_imgZ cur_pyr_levelrrxZ scale_maskroZpatchesrrrextract_patches_from_pyramids: " 4  r})r4r]rrr cCsnt||\}}}}t|d}|d|k|d||k|d|k|d||k}|jddd}|S)a`Check if the LAF is touching or partly outside the image boundary. Returns the mask of LAFs, which are fully inside the image, i.e. valid. Args: laf: :math:`(B, N, 2, 3)`. images: images, lafs are detected in :math:`(B, CH, H, W)`. border: additional border. Returns: mask with shape :math:`(B, N)`. rWrXrrr)r rCrUra)r4r]rrrSrcrdrTZgood_lafs_maskrrrlaf_is_inside_images  6r)r4cCs<t|tj|d|d|d|d|dgdd}|S)zConvert local affine frame(LAF) to alternative representation: coordinates of LAF center, LAF-x unit vector, LAF-y unit vector. Args: laf: :math:`(B, N, 2, 3)`. Returns: threepts :math:`(B, N, 2, 3)`. rrWrXrr)r rrE)r4Z three_ptsrrrlaf_to_three_pointss 0r)threeptscCs4tj|d|d|d|d|dgdd}|S)zConvert three points to local affine frame. Order is (0,0), (0, 1), (1, 0). Args: threepts: :math:`(B, N, 2, 3)`. Returns: laf :math:`(B, N, 2, 3)`. rWrrXrr)rrE)rr4rrrlaf_from_three_points s &r)trans_01lafs_1r c Cst|t|std|j|jks.td|jd|jdksJtd|jddkrf|jddksntd|\}}}}t|}| dd dd  ||dd }t ||}| ||dd  dd dd }t |S) aFunction that applies perspective transformations to a set of local affine frames (LAFs). Args: trans_01: tensor for perspective transformations of shape :math:`(B, 3, 3)`. lafs_1: tensor of lafs of shape :math:`(B, N, 2, 3)`. Returns: tensor of N-dimensional points of shape :math:`(B, N, 2, 3)`. Examples: >>> rng = torch.manual_seed(0) >>> lafs_1 = torch.rand(2, 4, 2, 3) # BxNx2x3 >>> lafs_1 tensor([[[[0.4963, 0.7682, 0.0885], [0.1320, 0.3074, 0.6341]], [[0.4901, 0.8964, 0.4556], [0.6323, 0.3489, 0.4017]], [[0.0223, 0.1689, 0.2939], [0.5185, 0.6977, 0.8000]], [[0.1610, 0.2823, 0.6816], [0.9152, 0.3971, 0.8742]]], [[[0.4194, 0.5529, 0.9527], [0.0362, 0.1852, 0.3734]], [[0.3051, 0.9320, 0.1759], [0.2698, 0.1507, 0.0317]], [[0.2081, 0.9298, 0.7231], [0.7423, 0.5263, 0.2437]], [[0.5846, 0.0332, 0.1387], [0.2422, 0.8155, 0.7932]]]]) >>> trans_01 = torch.eye(3).repeat(2, 1, 1) # Bx3x3 >>> trans_01.shape torch.Size([2, 3, 3]) >>> lafs_0 = perspective_transform_lafs(trans_01, lafs_1) # BxNx2x3 z Input type is not a torch.Tensorz!Tensor must be in the same devicerz2Input batch size must be the same for both tensorsrrz#Transformation should be homographyr r)r rZ is_tensorr:r-r! ValueErrorrCrrRZreshaperr"r) rrbsnrSZ threepts_1Zpoints_1Zpoints_0Z threepts_0rrrperspective_transform_lafss+   r)NN)r<)rH)r)rg)rgT)rgT)r)+rMtypingrrrZtorch.nn.functionalnnZ functionalrlZkornia.geometry.conversionsrrrZkornia.geometry.linalgrZkornia.geometry.transformrZkornia.testingr r r9rrrr(r5r8r3r%rGintrUr\rerfrpboolryr}rrrrrrrrsL    $1#!   '  1