a  þdRã@szddlZddlmZddlmZddlmZddlmZGdd„dej ƒZ Gd d „d ej ƒZ e  ¡Gd d „d ej ƒƒZ dS) éN)Únn)Ú functional)Ú ARCH_REGISTRYé)Ú flow_warpcs(eZdZdZ‡fdd„Zdd„Z‡ZS)Ú BasicModulezŽBasic module of SPyNet. Note that unlike the architecture in spynet_arch.py, the basic module here contains batch normalization. csÆtt|ƒ ¡t tjdddddddt d¡tjdd tjdd dddddt d ¡tjdd tjd ddddddt d¡tjdd tjdd dddddt d ¡tjdd tjd d dddd ¡ |_dS)Néé éréF)Ú in_channelsÚ out_channelsÚ kernel_sizeÚstrideÚpaddingZbiasT©Zinplaceé@éé)r r rrr) ÚsuperrÚ__init__rZ SequentialÚConv2dZ BatchNorm2dÚReLUÚ basic_module)Úself©Ú __class__©ú_/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/basicsr/archs/tof_arch.pyrs÷zBasicModule.__init__cCs | |¡S)a Args: tensor_input (Tensor): Input tensor with shape (b, 8, h, w). 8 channels contain: [reference image (3), neighbor image (3), initial flow (2)]. Returns: Tensor: Estimated flow with shape (b, 2, h, w) )r)rZ tensor_inputrrrÚforwards zBasicModule.forward©Ú__name__Ú __module__Ú __qualname__Ú__doc__rrÚ __classcell__rrrrr s rcs*eZdZdZd‡fdd„ Zdd„Z‡ZS)Ú SPyNetTOFa/SPyNet architecture for TOF. Note that this implementation is specifically for TOFlow. Please use spynet_arch.py for general use. They differ in the following aspects: 1. The basic modules here contain BatchNorm. 2. Normalization and denormalization are not done here, as they are done in TOFlow. Paper: Optical Flow Estimation using a Spatial Pyramid Network Code reference: https://github.com/Coldog2333/pytoflow Args: load_path (str): Path for pretrained SPyNet. Default: None. NcsLtt|ƒ ¡t dd„tdƒDƒ¡|_|rH| tj |dd„dd¡dS)NcSsg|] }tƒ‘qSr)r)Ú.0Ú_rrrÚ >óz&SPyNetTOF.__init__..écSs|S©Nr)ZstorageÚlocrrrÚ@r*z$SPyNetTOF.__init__..)Z map_locationÚparams) rr&rrZ ModuleListÚrangerZload_state_dictÚtorchÚload)rZ load_pathrrrr;szSPyNetTOF.__init__c Csä| ¡\}}}}|g}|g}tdƒD]@}| dtj|ddddd¡| dtj|ddddd¡q$|d |d|d|d¡}tdƒD]T}tj|ddd d d } | |j|t  ||t |||   dddd ¡ƒ| gd ¡ƒ}qŠ|S) zì Args: ref (Tensor): Reference image with shape of (b, 3, h, w). supp: The supporting image to be warped: (b, 3, h, w). Returns: Tensor: Estimated optical flow: (b, 2, h, w). r rrF)ÚinputrrZcount_include_padrr+ZbilinearT)r3Z scale_factorÚmodeZ align_cornersg@r) Úsizer0ÚinsertÚFZ avg_pool2dZ new_zerosZ interpolaterr1ÚcatrÚpermute) rÚrefZsuppÚ num_batchesr(ÚhÚwÚflowÚiZflow_uprrrrBs     *ÿzSPyNetTOF.forward)Nr rrrrr&*sr&cs:eZdZdZd ‡fdd„ Zdd„Zdd„Zd d „Z‡ZS) ÚTOFlowa"PyTorch implementation of TOFlow. In TOFlow, the LR frames are pre-upsampled and have the same size with the GT frames. Paper: Xue et al., Video Enhancement with Task-Oriented Flow, IJCV 2018 Code reference: 1. https://github.com/anchen1011/toflow 2. https://github.com/Coldog2333/pytoflow Args: adapt_official_weights (bool): Whether to adapt the weights translated from the official implementation. Set to false if you want to train from scratch. Default: False Fc sÈtt|ƒ ¡||_|rdnd|_| dt gd¢¡ dddd¡¡| dt gd¢¡ dddd¡¡t ƒ|_ t   dd d dd ¡|_ t   d d d dd ¡|_t   d d d¡|_t   d dd¡|_t jd d |_dS)Nrr Úmean)g ×£p= ß?gÉv¾Ÿ/Ý?g–C‹lçûÙ?rÚstd)gZd;ßOÍ?gyé&1¬Ì?gÍÌÌÌÌÌÌ?éré r+Tr)rr@rÚadapt_official_weightsÚref_idxZregister_bufferr1ZTensorÚviewr&ÚspynetrrÚconv_1Úconv_2Úconv_3Úconv_4rÚrelu)rrErrrros""zTOFlow.__init__cCs||j|jSr,)rArB©rÚimgrrrÚ normalizeƒszTOFlow.normalizecCs||j|jSr,)rBrArNrrrÚ denormalize†szTOFlow.denormalizec Csb|jr,|dd…gd¢dd…dd…dd…f}| ¡\}}}}}| | dd||¡¡}| ||d||¡}|dd…|jdd…dd…dd…f}g}tdƒD]d} | |jkr°| |¡q–|dd…| dd…dd…dd…f} | || ¡} | t| |   dddd¡ƒ¡q–t j |dd } |  |d||¡} |  |  | ¡¡} |  | | ¡¡} |  | | ¡¡} | | ¡|} | | ¡S) zˆ Args: lrs: Input lr frames: (b, 7, 3, h, w). Returns: Tensor: SR frame: (b, 3, h, w). N)r rrrr+éééÿÿÿÿr r rrr)Zdim)rEr5rPrGrFr0ÚappendrHrr9r1ÚstackrMrIrJrKrLrQ) rZlrsr;Znum_lrsr(r<r=Zlr_refZ lr_alignedr?Zlr_suppr>Úhrrrrr‰s( &$   " zTOFlow.forward)F) r!r"r#r$rrPrQrr%rrrrr@]s r@)r1rZtorch.nnrr7Zbasicsr.utils.registryrZ arch_utilrÚModulerr&Úregisterr@rrrrÚs    !3