a  dD!@sdZddlZddlmZmZmZmZmZddlZddlm Z ddl m m Z ddl mZmZddlmZddlmZeeZGd d d e jZGd d d eZdeeeeedddZdS)a? Image to Patch Embedding using Conv2d A convolution based approach to patchifying a 2D image w/ embedding projection. Based on code in: * https://github.com/google-research/vision_transformer * https://github.com/google-research/big_vision/tree/main/big_vision Hacked together by / Copyright 2020 Ross Wightman N)CallableListOptionalTupleUnion)nn)Formatnchw_to) to_2tuple)_assertc sTeZdZUdZeed<deeeeeeee ee e d fd d Z d d Z Z S) PatchEmbed! 2D Image to Patch Embedding output_fmtNTimg_size patch_sizein_chans embed_dim norm_layerflattenrbiasc stt||_|dur\t||_tddt|j|jD|_|jd|jd|_nd|_d|_d|_|durd|_ t ||_ n||_ t j |_ t j|||||d|_|r||nt |_dS)NcSsg|]\}}||qSr).0sprr`/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/timm/layers/patch_embed.py -z'PatchEmbed.__init__..rrF)Z kernel_sizeZstrider)super__init__r rrtuplezip grid_sizeZ num_patchesrr rNCHWrZConv2dprojZIdentitynorm selfrrrrrrrr __class__rr r$s    zPatchEmbed.__init__cCs|j\}}}}|jdurht||jdkd|d|jddt||jdkd|d|jdd||}|jr|ddd}n|jtjkrt ||j}| |}|S)NrInput image height (z) doesn't match model ().rInput image width () shaperr r)r transposerr r(r r*)r,xBCHWrrr forward?s ((    zPatchEmbed.forward)rrrrNTNT)__name__ __module__ __qualname____doc__r __annotations__rintrboolstrr$r: __classcell__rrr-r r s* !r c sheZdZUdZeed<deeeeeeee ee e d fd d Z e e jeefd d dZZS)PatchEmbedWithSizerrrrrrNTrc s tj||||||||ddS)Nr)r#r$r+r-rr r$Ss zPatchEmbedWithSize.__init__)returncCs|j\}}}}|jdurpt||jddkd|d|jddt||jddkd|d|jdd||}|jdd}|jr|ddd}n|jtj krt ||j}| |}||fS) Nrr/z#) must be divisible by patch size (r0rr1r2) r3rr rr)rr4rr r(r r*)r,r5r6r7r8r9r'rrr r:is ,,    zPatchEmbedWithSize.forward)rrrrNTNT)r;r<r=r>r r?rr@rrArBr$rtorchTensorrr:rCrrr-r rDNs* rDbicubicTF)new_size interpolation antialiasverbosec s ddlzddl}|j}Wn0tyJttdr:tj}n dsFJdYn0t|jdksbJdtdksvJd |jd d}t|tkr|S|rt d |jd d dfddfdd}||} t j | jfdd} ||| dddd} | |S)a/Resample the weights of the patch embedding kernel to target resolution. We resample the patch embedding kernel by approximately inverting the effect of patch resizing. Code based on: https://github.com/google-research/big_vision/blob/b00544b81f8694488d5f36295aeb7972f3755ffe/big_vision/models/proj/flexi/vit.py With this resizing, we can for example load a B/8 filter into a B/16 model and, on 2x larger input image, the result will match. Args: patch_embed: original parameter to be resized. new_size (tuple(int, int): target shape (height, width)-only. interpolation (str): interpolation for resize antialias (bool): use anti-aliasing filter in resize verbose (bool): log operation Returns: Resized patch embedding kernel. rNvmapFzLfunctorch or a version of torch with vmap is required for FlexiViT resizing.zFour dimensions expectedr2zNew shape should only be hwrFzResize patch embedding z to z, w/ z interpolation.cs,t|d}tj||dd}|S)N)NN.)sizemoderL)rr.)rGrHFZ interpolatenumpy)Zx_np _new_sizeZx_tfZ x_upsampled)rLrKrr resizesz$resample_patch_embed..resizecsTg}t|D]4}|}d|||<|||dq|jS)Ng?)rangeprodzerosZ unravel_indexappendreshapestackT)Z _old_sizerTmatiZ basis_vec)nprUrr get_resize_mats  z,resample_patch_embed..get_resize_matcs|d}|S)NrV)r[)kernelZresampled_kernel)rJresize_mat_pinvrr resample_kernelsz-resample_patch_embed..resample_kernelr)rS functorchrN ImportErrorhasattrrGlenr3r%_loggerinforHZlinalgZpinvr]) Z patch_embedrJrKrLrMrerNZold_sizeraZ resize_matrdZv_resample_kernelr)rLrKrJr`rUrcr resample_patch_embedys,     rk)rITF)r>loggingtypingrrrrrrGrZtorch.nn.functionalZ functionalrRformatr r Zhelpersr Z trace_utilsr getLoggerr;riModuler rDr@rBrArkrrrr s(     5.