a  dd@sdZddlZddlmZmZmZddlZddlmm Z ddlmZddl m Z m Z mZmZmZmZddlmZddlmZdd lmZmZmZdd lmZmZmZmZmZm Z dd l!m"Z#gZ$dqd dZ%drddZ&dsddZ'dtddZ(e)ee%ddddde%ddddde&dddddddd e&dddd!dddd e&dd!dd"dddd fdd#d$d$d%d&d'ee%dd(dd)e%dddd)e&dddd"ddd*e&dd!dd+ddd*e&dd"dd,ddd*fdd#d$d$d%d-d'ee%dd(dd)e%dddd)e&dd"dd,ddd*e&dd.dd/ddd*e&dd0dd1ddd*fdd#d$d$d%d2d'ee%dd(dd)e%dddd)e&dd"dd,ddd*e&dd.dd/ddd*e&dd0dd1ddd*fdd#d$d$d3e)d4d5d2d6e(de(d7e(d8e(de(d9e(d:e(dd; Z*eGdd?d?ej+Z-Gd@dAdAej+Z.eGdBdCdCej+Z/edDe,edEe/dudGdHZ0dvdIdJZ1dwdKdLZ2ee2dMdNe2dMdNe2dMdNe2dMdOdPe2dMdOdPe2dMdOdPe2dMdOdPe2dMdOdPe2dMdOdPe2dMdOdPe2dMdOdPe2dMdOdPe2dMdOdPe2dMdQdRddSe2dMdQdRddSe2dMdQdRddSdTZ3edxedUdVdWZ4edyedUdXdYZ5edzedUdZd[Z6ed{edUd\d]Z7ed|edUd^d_Z8ed}edUd`daZ9ed~edUdbdcZ:ededUdddeZ;ededUdfdgZdjdkdldmdndodpdS)a  MobileViT Paper: V1: `MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer` - https://arxiv.org/abs/2110.02178 V2: `Separable Self-attention for Mobile Vision Transformers` - https://arxiv.org/abs/2206.02680 MobileVitBlock and checkpoints adapted from https://github.com/apple/ml-cvnets (original copyright below) License: https://github.com/apple/ml-cvnets/blob/main/LICENSE (Apple open source) Rest of code, ByobNet, and Transformer block hacked together by / Copyright 2022, Ross Wightman N)CallableTupleOptional)nn) to_2tuplemake_divisible GroupNorm1ConvMlpDropPath is_exportable)build_model_with_cfg)register_notrace_module)register_modelgenerate_default_cfgsregister_model_deprecations)register_block ByoBlockCfg ByoModelCfgByobNetLayerFn num_groups)Block@c Cstd|||d|tddddS)NZbottler T)Z bottle_inZ linear_out)typedcsgsbr block_kwargs)rdictrrrrr#^/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/timm/models/mobilevit.py_inverted_residual_block!s  r%c Cs,t||||dtdd|dt|||ddfS)Nr" mobilevitr )transformer_dimtransformer_depth patch_size)rrrrr r%rr!rrrr(r)r*rr#r#r$_mobilevit_block(sr-@?c Cs.t||||dtdd|d|dt||ddfS)Nr" mobilevit2r )r)r*)rrrrrrr r+)rrrr)r*rZtransformer_brr#r#r$_mobilevitv2_block6s r2?c sd}dkr"tfdd|D}ttd|ddddtd |dd ddtd|d d d d td|d d d d td|d d d d ftd ddddd}|S)N)@ir3csg|]}t|qSr#)int).0r multiplierr#r$ Fz$_mobilevitv2_cfg..r rr/r"r.)rrrr)r& 3x3silu)blocksstem_chs stem_type stem_pool downsample act_layer)tuplerr%r2r8)r;Zchscfgr#r:r$_mobilevitv2_cfgCs" rKr"r>0r4r,P`r@rArBi@)rCrDrErFrGrH num_featuresr?)rrr)rrrr(r)r*xr7r5iseg?)Zrd_ratio)rCrDrErFrGZ attn_layerZ attn_kwargsrQg?g?g?g?) mobilevit_xxs mobilevit_xs mobilevit_sZ semobilevit_smobilevitv2_050mobilevitv2_075mobilevitv2_125mobilevitv2_100mobilevitv2_150mobilevitv2_175mobilevitv2_200cseZdZdZdddddddddd d d d d d dejfeeeeeeeee eefeeeeeeeee ee e d fdd Z ejejdddZZS)MobileVitBlockzS MobileViT block Paper: https://arxiv.org/abs/2110.02178?context=cs.LG Nr>r r3r r r/r.r&F)in_chsout_chs kernel_sizestride bottle_ratio group_sizedilation mlp_ratior(r)r* num_heads attn_dropdrop no_fusiondrop_path_ratelayerstransformer_norm_layerc stt|ptt||}|p(|}p8t||j||||||dd|_tj |ddd|_ tj fddt | D|_ |_j|ddd|_|rd|_nj||||dd|_t| |_|jd|jd|_dS) Nrrhrigroupsrlr Frhbiasc s(g|] }tdjd qS)T)rmrnZqkv_biasro proj_drop drop_pathrH norm_layer)TransformerBlockactr9_rorprrrsrmrnr(rtr#r$r<s z+MobileVitBlock.__init__..rhri)superrb__init__rrr conv_norm_actconv_kxkrConv2dconv_1x1 Sequentialrange transformernorm conv_proj conv_fusionrr* patch_area)selfrfrgrhrirjrkrlrmr(r)r*rnrorprqrrrsrtkwargsrv __class__rr$rs(       zMobileVitBlock.__init__xreturncCs|}||}||}|j\}}|j\}}}}t|||t|||} } | || |} } | | } d}| |ks| |krtj|| | fddd}d}|||| || | dd}|||| |j  dd||j | d}| |}| |}| ||j | d}| dd||| | ||}| dd||| || |}|rntj|||fddd}||}|jdur|tj||fdd }|S) NFbilinearsizemodeZ align_cornersTr r.r>dim)rrr*shapemathceilF interpolatereshapeZ transposerrr contiguousviewrrtorchcat)rrZshortcutpatch_hpatch_wBCHWnew_hnew_w num_patch_h num_patch_w num_patchesrr#r#r$forwards2   & *      zMobileVitBlock.forward)__name__ __module__ __qualname____doc__rZ LayerNormr8rfloatrboolrrrrTensorr __classcell__r#r#rr$rbsN `_ Args: embed_dim (int): :math:`C_{in}` from an expected input of size :math:`(B, C_{in}, P, N)` mlp_ratio (float): Inner dimension ratio of the FFN relative to embed_dim drop (float): Dropout rate. Default: 0.0 attn_drop (float): Dropout rate for attention in multi-head attention. Default: 0.0 drop_path (float): Stochastic depth rate Default: 0.0 norm_layer (Callable): Normalization layer. Default: layer_norm_2d Shape: - Input: :math:`(B, C_{in}, P, N)` where :math:`B` is batch size, :math:`C_{in}` is input embedding dim, :math:`P` is number of pixels in a patch, and :math:`N` is number of patches, - Output: same shape as the input r/reN)rrmrprorzrcsrt|ptj}|pt}|||_t|||d|_t||_ |||_ t |t ||||d|_ t||_dS)N)rrory)Z in_featuresZhidden_featuresrHrp)rrrZSiLUrnorm1rattnr drop_path1norm2r r8mlp drop_path2)rrrmrprorzrHr{rr#r$rs      zLinearTransformerBlock.__init__rcCsj|dur$|||||}n(|}||}|||}|||}|||||}|S)N)rrrrrr)rrrresr#r#r$rs  zLinearTransformerBlock.forward)r/rerereNN)N) rrrrr8rrrrrrrr#r#rr$rs rcseZdZdZddddddddd d d d defeeeeeeeeeefeeeeeeeee e d fd d Z e j e j dddZZS)MobileVitV2Blockz8 This class defines the `MobileViTv2 block <>`_ Nr>r3r rcr/r.rdre)rfrgrhrjrkrlrmr(r)r*rorprrrsrtc stt|ptt||}|p(|}p8t||j|||d||dd|_tj |ddd|_ tj fddt | D|_ |_j|dddd|_t| |_|jd|jd|_t|_dS) Nr rruFrwc s$g|]}tjdqS))rmrorprzrHr{)rr}r~rorprrrsrmr(rtr#r$r<s z-MobileVitV2Block.__init__..)rhriZ apply_act)rrrrrrrrrrrrrrrrrr*rr coreml_exportable)rrfrgrhrjrkrlrmr(r)r*rorprrrsrtrrvrrr$rs$      zMobileVitV2Block.__init__rc Cs~|j\}}}}|j\}}t|||t|||}} ||| |} } | | } ||ksh| |kr~tj||| fddd}||}||}|jd}|jrtj |||f||fd}n$| ||| || | ddddd d }| ||d | }| |}| |}|jr4| ||||| | }tj||d }n<| ||||| |  ddd d dd}| ||| || |}||}|S) NrTrr rrr>r.r&r)Zupscale_factor)rr*rrrrrrrZunfoldrZpermuterrZ pixel_shuffler) rrrrrrrrrrrrrr#r#r$rs. &   $  $ zMobileVitV2Block.forward)rrrrrr8rrrrrrrrrrr#r#rr$rsB 2rr'r1FcKs0tt||f|st|nt|tddd|SNT)Zflatten_sequential)Z model_cfgZ feature_cfgr r model_cfgsr!variantZ cfg_variant pretrainedrr#r#r$_create_mobilevit#srcKs0tt||f|st|nt|tddd|Srrrr#r#r$_create_mobilevit2+src Ks |ddddddddd d d |S) Ni)r>r6r6)rdrdg?Zbicubic)rerere)r3r3r3z stem.convzhead.fcF) urlZ num_classes input_size pool_sizecrop_pct interpolationmeanZstdZ first_conv classifierZfixed_input_sizer#)rrr#r#r$_cfg3srztimm/) hf_hub_idg"~j?)rr)r>r7r7) r)rrrr)zmobilevit_xxs.cvnets_in1kzmobilevit_xs.cvnets_in1kzmobilevit_s.cvnets_in1kzmobilevitv2_050.cvnets_in1kzmobilevitv2_075.cvnets_in1kzmobilevitv2_100.cvnets_in1kzmobilevitv2_125.cvnets_in1kzmobilevitv2_150.cvnets_in1kzmobilevitv2_175.cvnets_in1kzmobilevitv2_200.cvnets_in1k$mobilevitv2_150.cvnets_in22k_ft_in1k$mobilevitv2_175.cvnets_in22k_ft_in1k$mobilevitv2_200.cvnets_in22k_ft_in1k(mobilevitv2_150.cvnets_in22k_ft_in1k_384(mobilevitv2_175.cvnets_in22k_ft_in1k_384(mobilevitv2_200.cvnets_in22k_ft_in1k_384)rcKstdd|i|S)NrXr)rXrrrr#r#r$rXosrXcKstdd|i|S)NrYr)rYrrr#r#r$rYtsrYcKstdd|i|S)NrZr)rZrrr#r#r$rZysrZcKstdd|i|S)Nr[r)r[rrr#r#r$r[~sr[cKstdd|i|S)Nr\r)r\rrr#r#r$r\sr\cKstdd|i|S)Nr^r)r^rrr#r#r$r^sr^cKstdd|i|S)Nr]r)r]rrr#r#r$r]sr]cKstdd|i|S)Nr_r)r_rrr#r#r$r_sr_cKstdd|i|S)Nr`r)r`rrr#r#r$r`sr`cKstdd|i|S)Nrar)rarrr#r#r$rasrarrrrrr)Zmobilevitv2_150_in22ft1kZmobilevitv2_175_in22ft1kZmobilevitv2_200_in22ft1kZmobilevitv2_150_384_in22ft1kZmobilevitv2_175_384_in22ft1kZmobilevitv2_200_384_in22ft1k)r)r&r)r.r/r0)r3)NF)NF)rA)F)F)F)F)F)F)F)F)F)F)?rrtypingrrrrZtorch.nn.functionalrZ functionalrZ timm.layersrrrr r r Z_builderr Z _features_fxr _registryrrrZbyobnetrrrrrrZvision_transformerrr|__all__r%r-r2rKr!rModulerbrrrrrrZ default_cfgsrXrYrZr[r\r^r]r_r`rarr#r#r#r$sd               Lkw:]     1