a d6@sdZddlmZmZmZmZddlZddlmZddl mm Z ddl m Z m Z GdddejZGdddejZGd d d ejZGd d d ejZGd ddejZddZGdddejZGdddejZGdddejZGdddejZdS)z%Implements several backbone networks.)DictListTupleUnionN) pixel_shufflesoftmaxcsDeZdZdZd eeeeedfdd Zejejd d d ZZ S) HourglassBackboneaHourglass network, taken from https://github.com/zhou13/lcnn. Args: input_channel: number of input channels. depth: number of residual blocks per hourglass module. num_stacks: number of hourglass modules stacked together. num_blocks: number of layers in each residual block. num_classes: number of heads for the output of a hourglass module. ) input_channeldepth num_stacks num_blocks num_classesc s4tt|_tfi|j|||||d|_dS)Nheadrrrrinput_channels)super__init__ MultitaskHeadrhgnet)selfr rrrr __class__g/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/kornia/feature/sold2/backbones.pyrs zHourglassBackbone.__init__) input_imagesreturncCs ||SN)r)rrrrrforward'szHourglassBackbone.forward)r r r r r __name__ __module__ __qualname____doc__intrtorchTensorr" __classcell__rrrrr s   rcs6eZdZedfdd ZejejdddZZS)r)rc stt|d}dgdgdgg}g}t|gD]:}|ttj||dddtjddtj||ddq4t ||_ dS) Nr r r ) kernel_sizepaddingTZinplacer-) rrr(sumappendnn SequentialConv2dReLU ModuleListheads)rrmZ head_sizer8Zoutput_channelsrrrr,s   zMultitaskHead.__init__xr cstjfdd|jDddS)Ncsg|] }|qSrr).0rr;rr =z)MultitaskHead.forward..r Zdim)r)catr8rr;rr=rr"<szMultitaskHead.forward) r$r%r&r(rr)r*r"r+rrrrr+srcsReZdZd eeeeeeeffejjdfdd Z ej ej dddZ Z S) Bottleneck2Dr N)inplanesplanesstride downsamplecstt||_tj||dd|_t||_tj||d|dd|_t||_ tj||ddd|_ tj dd|_ ||_ ||_dS)Nr r0r,r-rFr.r Tr/)rrr3 BatchNorm2dbn1r5conv1bn2conv2bn3conv3r6relurGrF)rrDrErFrGrrrrAs    zBottleneck2D.__init__r:cCs~|}||}||}||}||}||}||}||}||}||}|jdurr||}||7}|Sr!)rJrPrKrLrMrNrOrG)rr;Zresidualoutrrrr"Ps           zBottleneck2D.forward)r N) r$r%r&r(rrr)r3Modulerr*r"r+rrrrrC@s rCcseZdZdejjeeeedfdd ZejjeeejjdddZejjeeeejj dd d Z eej ej d d d Z ej ej dddZ ZS) Hourglassr )blockrrEr expansioncs2t||_||_||_||||||_dSr!)rrrrTrU_make_hour_glassr)rrTrrErrUrrrrhs  zHourglass.__init__)rTrrEr cCs4g}td|D]}||||j|qtj|S)Nr)ranger2rUr3r4)rrTrrElayers_rrr_make_residualoszHourglass._make_residual)rTrrErr c Csng}t|D]V}g}tdD]}|||||q|dkrR||||||t|q t|S)Nr,r)rWr2rZr3r7) rrTrrErhgliresrYrrrrVus  zHourglass._make_hour_glass)nr;r c Cs|j|dd|}tj|ddd}|j|dd|}|dkrV||d|}n|j|dd|}|j|dd|}tj||jddd}||}|S)Nr rr rFr,)size)rFZ max_pool2d_hour_glass_forwardZ interpolateshape) rr^r;Zup1Zlow1Zlow2Zlow3Zup2rQrrrrbszHourglass._hour_glass_forwardr:cCs||j|Sr!)rbrrBrrrr"szHourglass.forward)r )r$r%r&r)r3rRr(rrZr7rVr*rbr"r+rrrrrSgs   rSc seZdZdZdejjejjeeeeeedfdd Zdejjeee ee eeffejjddd Z eeejjd d d Z ej ej d ddZZS) HourglassNetz,Hourglass model from Newell et al ECCV 2016.r )rTrrrrrrrUc std|_d|_||_||_tj||jdddd|_t |j|_ tj dd|_ | ||jd |_| ||jd |_| ||jd |_tjddd |_|j|j} ggggggf\} } } } }}t|D]}| t|||j|| | ||j|| || | | || ||d kr|tj| | d d |tj|| d d qt| |_t| |_t| |_t| |_t||_t||_dS) N@r r,rHTr/r r_r0)rrrDZ num_featsrrUr3r5rKrIrJr6rPrZlayer1layer2layer3Z MaxPool2dmaxpoolrWr2rS_make_fcr7rr]fcscorefc_score_)rrTrrrrrrrUchr[r]rmrnrorpr\rrrrs8         zHourglassNet.__init__r )rTrEblocksrFr cCsd}|dks|j||jkrr?zhg..rrrrrr)rdrCget)kwargsmodelrrrrs rcs>eZdZdZd eedfdd Zejejddd ZZ S) SuperpointDecoderzJunction decoder based on the SuperPoint architecture. Args: input_feat_dim: channel size of the input features. Returns: the junction heatmap, with shape (B, H, W). rf)input_feat_dim grid_sizecsTttjjdd|_tjj|ddddd|_tjjddddd d|_||_ dS) NTr/r,r r rHAr) rrr)r3r6rPr5convPaconvPbr~)rr}r~rrrrs  zSuperpointDecoder.__init__input_featuresr cCs^|||}||}t|dd}t|ddddddddf|jdddf}|S)Nr r@r)rPrrrrr~)rrfeatsemiZ junc_probZ junc_predrrrr" s   4zSuperpointDecoder.forward)rfr|r#rrrrr{sr{csTeZdZdZd eeedfdd Zeeeddd Zej ej d d d Z Z S)PixelShuffleDecodera"Pixel shuffle decoder used to predict the line heatmap. Args: input_feat_dim: channel size of the input features. num_upsample: how many upsamples are performed. output_channel: number of output channels. Returns: the (B, 1, H, W) line heatmap. rfr )r} num_upsampleoutput_channelc st|||_td|_g}|ttj ||jdddddt |jdtj dd|jddD]6}|ttj ||ddddt |tj ddqr|ttj |jd|ddddt ||_ dS) Nr rr,r rHTr/r)rrget_channel_confZ channel_confr3Z PixelShuffle pixshuffler2r4r5rIr6r7conv_block_lst)rr}rrrZchannelrrrr!s.       zPixelShuffleDecoder.__init__)rr cCs|dkrgdSgdS)z2Get num of channels based on number of upsampling.r )rre)rrerr r)rrrrrrDsz$PixelShuffleDecoder.get_channel_confrcCs`|}|jddD]}||}||}q|jd|}t|dddddddddf}|S)Nrr r@)rrr)rrrQrTheatmaprrrr"Js $zPixelShuffleDecoder.forward)rfr r ) r$r%r&r'r(rrrr)r*r"r+rrrrrs #rcs<eZdZdZd edfdd ZejejdddZZ S) SuperpointDescriptorzDescriptor decoder based on the SuperPoint arcihtecture. Args: input_feat_dim: channel size of the input features. Returns: the semi-dense descriptors with shape (B, 128, H/4, W/4). rf)r}csNttjjdd|_tjj|ddddd|_tjjdddddd|_dS) NTr/rr,r rHrfr) rrr)r3r6rPr5rr)rr}rrrras zSuperpointDescriptor.__init__rcCs|||}||}|Sr!)rPrr)rrrrrrrr"gs zSuperpointDescriptor.forward)rfr#rrrrrXsrcs.eZdZdZedfdd ZddZZS)SOLD2NetuFull network for SOLD². Args: model_cfg: the configuration as a Dict. Returns: a Dict with the following values: junctions: heatmap of junctions. heatmap: line heatmap. descriptors: semi-dense descriptors. ) model_cfgcsbt||_tfi|jd|_d}t||jd|_t|dd|_d|jvr^t ||_ dS)NZ backbone_cfgrr~r )ruse_descriptor) rrcfgr backbone_netr{junction_decoderrheatmap_decoderrdescriptor_decoder)rrZ feat_channelrrrr}s  zSOLD2Net.__init__cCsD||}||}||}||d}d|jvr@|||d<|S)N) junctionsrrZ descriptors)rrrrr)rrfeaturesrZheatmapsoutputsrrrr"s     zSOLD2Net.forward)r$r%r&r'rrr"r+rrrrrqs r)r'typingrrrrr)Ztorch.nnr3Ztorch.nn.functionalZ functionalrarrrRrrrCrSrdrr{rrrrrrrs '+[B