a dm@sUddlmZmZmZddlZddlmZddlmZddl m Z ddl m Z m Z ddlmZddlmZdd lmZiZeeefed <d ed <d ed<dddddgdddddgddddddddd d!dd"d#d$ dddd%d&gdd'd(ZGd)d*d*ejZdS)+)CallableDictOptionalN)resize)build_backbone)FinePreprocessLocalFeatureTransformer)CoarseMatching) FineMatching)PositionEncodingSineurlsz;http://cmp.felk.cvut.cz/~mishkdmy/models/loftr_outdoor.ckptoutdoorz:http://cmp.felk.cvut.cz/~mishkdmy/models/loftr_indoor.ckptZindoorZ ResNetFPN)T)r)Z initial_dimZ block_dimsrr)selfcrossrrrrrrZlinearF)d_modeld_ffnnhead layer_names attention temp_bug_fixg?rZ dual_softmaxg?g?g?) ZthrZ border_rmZ match_typeZdsmax_temperatureZ skh_itersZskh_init_bin_scoreZ skh_prefilterZtrain_coarse_percentZtrain_pad_num_gt_minrr)rrrrr)Z backbone_type resolutionZfine_window_sizeZfine_concat_coarse_featZ resnetfpncoarse match_coarsefinecsXeZdZdZdefeeedfdd Ze eee j fdddZ fd d Z ZS) LoFTRaModule, which finds correspondences between two images. This is based on the original code from paper "LoFTR: Detector-Free Local Feature Matching with Transformers". See :cite:`LoFTR2021` for more details. If the distance matrix dm is not provided, :py:func:`torch.cdist` is used. Args: config: Dict with initiliazation parameters. Do not pass it, unless you know what you are doing`. pretrained: Download and set pretrained weights to the model. Options: 'outdoor', 'indoor'. 'outdoor' is trained on the MegaDepth dataset and 'indoor' on the ScanNet. Returns: Dictionary with image correspondences and confidence scores. Example: >>> img1 = torch.rand(1, 1, 320, 200) >>> img2 = torch.rand(1, 1, 128, 128) >>> input = {"image0": img1, "image1": img2} >>> loftr = LoFTR('outdoor') >>> out = loftr(input) r) pretrainedconfigcst||_t||_t|dd|ddd|_t|d|_t |d|_ t ||_ t|d|_ t|_||_|dur|tvrtdtdd }tjjt||d }||d |dS) Nr rr)rr!r"z$pretrained should be None or one of cSs|S)N)Zstoragelocr&r&c/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/kornia/feature/loftr/loftr.py]z LoFTR.__init__..)Z map_location state_dict)super__init__r%rbackboner pos_encodingr loftr_coarser coarse_matchingrfine_preprocess loftr_finer fine_matchingr$r keys ValueErrortorchZhubZload_state_dict_from_urlload_state_dicteval)rr$r%Z storage_fcnZpretrained_dict __class__r&r(r-Ks&    zLoFTR.__init__)datareturncCsP||dd|djdd|djddd|d|dkr|tj|d|dgdd \}}||d ||d \}}\}}n&||d||d\}}\}}||jdd|jdd|jdd|jddd ||ddd d }|j\}} } } | |d| }||ddd d }|j\} } }}| | d|}d}}d|vrt |d|ddd d}d|vrt |d|ddd d}| ||||\}}|j |||||d||||||\}}|ddkr |||\}}||||ddddd}i}|D]\}}||||<q4|S)a Args: data: dictionary containing the input data in the following format: Keyword Args: image0: left image with shape :math:`(N, 1, H1, W1)`. image1: right image with shape :math:`(N, 1, H2, W2)`. mask0 (optional): left image mask. '0' indicates a padded position :math:`(N, H1, W1)`. mask1 (optional): right image mask. '0' indicates a padded position :math:`(N, H2, W2)`. Returns: - ``keypoints0``, matching keypoints from image0 :math:`(NC, 2)`. - ``keypoints1``, matching keypoints from image1 :math:`(NC, 2)`. - ``confidence``, confidence score [0, 1] :math:`(NC)`. - ``batch_indexes``, batch indexes for the keypoints and lafs :math:`(NC)`. Zimage0rrNZimage1)bshw0_ihw1_ir?r@)Zdimr>)hw0_chw1_cZhw0_fZhw1_frrZmask0rAZnearest) interpolationZmask1rB)mask_c0mask_c1Z keypoints0Z keypoints1 confidenceZ batch_indexes)Zmkpts0_fZmkpts1_fZmconfZb_ids)updatesizeshaper.r7catsplitr/ZpermuteZreshaperflattenr0r1r2r3r4items)rr<Zfeats_cZfeats_fZfeat_c0Zfeat_c1Zfeat_f0Zfeat_f1nhwcZn1h1Zw1Zc1rFrGZfeat_f0_unfoldZfeat_f1_unfoldZ rename_keysoutkvr&r&r(forwardbsP0$(&      z LoFTR.forwardcsNt|D]&}|dr ||||ddd<q tj|g|Ri|S)Nzmatcher.r)listr5 startswithpopreplacer,r8)rr+argskwargsrVr:r&r(r8s zLoFTR.load_state_dict)__name__ __module__ __qualname____doc__ default_cfgrstrrr-dictr7ZTensorrXr8 __classcell__r&r&r:r(r#2sPr#)typingrrrr7Ztorch.nnnnZkornia.geometryrr.rZ loftr_modulerr Zutils.coarse_matchingr Zutils.fine_matchingr Zutils.position_encodingr r re__annotations__rdModuler#r&r&r&r(sH