a  d)J@sdZddlmZmZddlmZddlmZmZddl m Z m Z m Z m Z gZee e ddd ddd d e ddd ddd d e d ddddd de dddddd d fdddddede e ddd ddd d e d dgdd ddd de d dgddddd de dddddddfdddddd ded! e e ddd ddd d e d d"d"d ddd de d d#dddd de d ddddd dfdddddded$e e ddd dd%d d e ddd dd%d d e d dddd%d de ddddd%d d fdddddd&ded%d'd( e e dddddd)d e ddd*ddd)d e dd+d ddd)d e ddd ddd)d fdd,dd-ed.dd/d0e e ddd ddd d e ddd ddd d e d ddddd de dddddd d fdddd-ed.dd/d0e e ddd ddd d e d dgdd ddd de d dgddddd de dddddddfdddddd d-ed.dd/d! e e ddd ddd d e d d"d"d ddd d-ed.dd"d1d2 e d d#dddd de d ddddd dfddddd-ed.dd/d3e e ddd dd%d d e ddd dd%d d e d dddd%d de ddddd%d d fddddd&d-ed.dd%d4d5e e ddd ddd d e ddd ddd d e d ddddd de dddddd d fdddd6ed7d8d0e e ddd ddd d e d d"d"d ddd de d d#dddd de d ddddd dfddddd6ed7d8d3e e ddd ddd d e ddd ddd d e d ddddd de dddddd d fdddd6ed9d8d0e e ddd:dd%dd e dd#d;dd%dd e d dddd%ddfd?ddddd ed d@edddAd-edBddCdDdE e e ddd ddd d e d d"d ddd d6edFd8dGe d d#dddd d-eddHdGe d ddddd dedGfdddddIe e ddd ddd d e d d"d ddd d-eddHdGe d d#dddd d-eddHdGe d ddddd dedGfdddddIdJZddLdMZddNdOZeedPdQddRdSdTedUdQddRdSdVdWeddRdSdXedYdQddRdSdTedRdSdRdZed[dQdRdSdRd\ed]dQdRdSdRdVd^ed_dQdRdSdRdVd^ed`dQdRdSdRdVd^edadQdbdRdSdVdcedddQdbdRdSdeedfdQddRdSdVdWedgdQdhdhdidjdkdjdVdl edmdQddRdSdTedndQddRdSdTdoZede dpdqdrZede dpdsdtZede dpdudvZede dpdwdxZede dpdydzZede dpd{d|Zede dpd}d~Zede dpddZede dpddZede dpddZede dpddZede dpddZede dpddZ ede dpddZ!ede dpddZ"d9S)a Bring-Your-Own-Attention Network A flexible network w/ dataclass based config for stacking NN blocks including self-attention (or similar) layers. Currently used to implement experimental variants of: * Bottleneck Transformers * Lambda ResNets * HaloNets Consider all of the models definitions here as experimental WIP and likely to change. Hacked together by / copyright Ross Wightman, 2021. IMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STD)build_model_with_cfg)register_modelgenerate_default_cfgs) ByoBlockCfg ByoModelCfgByobNetinterleave_blocksbottleg?)typedcsgsbri)r self_attni)typesrrrrrri@ZtieredZmaxpoolTZ bottleneck)blocksstem_chs stem_type stem_poolfixed_input_sizeself_attn_layerself_attn_kwargs)reveryrrrrrigZd;O?)rrrrrZsiluise) rrrr act_layer num_features attn_layerrr)rrrrr$rrrZeca)dim_head) rrrrrr$r&rrg? Z7x7Zhalo) block_size halo_size)rrrrrr)r.r/Z num_heads) rr!rrrrrrr)rrrrr$rr)r.r/r*)rrrrr$r&rrlambda )rN0` i  )Zrd_ratio)Z bottle_inZ linear_outgQ?)r.r/Zqk_ratio) rrrZ downsampler%r$r&Z attn_kwargsZ block_kwargsrr )rrrrrrrr)r/)rrrrr$) botnet26t sebotnet33ts botnet50tseca_botnext26ts halonet_h1 halonet26t sehalonet33ts halonet50tseca_halonext26tslambda_resnet26tlambda_resnet50tslambda_resnet26rpt_256 haloregnetz_blamhalobotnet50tshalo2botnet50tsFcKs0tt||f|st|nt|tddd|S)NT)Zflatten_sequential)Z model_cfgZ feature_cfg)rr model_cfgsdict)variantZ cfg_variant pretrainedkwargsrN\/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/timm/models/byoanet.py_create_byoanetsrPc Ks"|dddddttddddd |S) Nir rRr8r8gffffff?Zbicubiczstem.conv1.convzhead.fcF) urlZ num_classes input_size pool_sizecrop_pct interpolationmeanstd first_conv classifierrmin_input_sizer)rTrMrNrNrO_cfgsr^zshttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/botnet26t_c1_256-167a0e9f.pthztimm/)r rr)r-r-)rT hf_hub_idrrUrVzxhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/sebotnet33ts_a1h2_256-957e3c3e.pthgGz?)rTr_rrUrVrW)rrUrVzxhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/eca_botnext26ts_c_256-95a898f6.pth)rUrVr]zuhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/halonet26t_a1h_256-3083328c.pth)rTr_rUrVr]zthttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/sehalonet33ts_256-87e053f9.pth)rTr_rUrVr]rWzwhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/halonet50ts_a1h2_256-f3a3daee.pthzyhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/eca_halonext26ts_c_256-06906299.pthzyhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/lambda_resnet26t_c_256-e5a5c857.pth)r r+r+)rTr_r]rUrVrWz|https://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/lambda_resnet50ts_a1h_256-b87370f7.pth)rTr_r]rUrVz{https://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/lambda_resnet26rpt_c_256-ab00292d.pthzxhttps://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/haloregnetz_c_raa_256-c8ad7616.pth)?r`r`z stem.convrQrS) rTr_rYrZr[rUrVr]rWz}https://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/lamhalobotnet50ts_a1h2_256-fe3d9445.pthz{https://github.com/rwightman/pytorch-image-models/releases/download/v0.1-attn-weights/halo2botnet50ts_a1h2_256-fd9c11a3.pth)zbotnet26t_256.c1_in1kzsebotnet33ts_256.a1h_in1kzbotnet50ts_256.untrainedzeca_botnext26ts_256.c1_in1kzhalonet_h1.untrainedzhalonet26t.a1h_in1kzsehalonet33ts.ra2_in1kzhalonet50ts.a1h_in1kzeca_halonext26ts.c1_in1kzlambda_resnet26t.c1_in1kzlambda_resnet50ts.a1h_in1kzlambda_resnet26rpt_256.c1_in1kzhaloregnetz_b.ra3_in1kzlamhalobotnet50ts_256.a1h_in1kzhalo2botnet50ts_256.a1h_in1k)returncKs|ddtdd|i|S)z4 Bottleneck Transformer w/ ResNet26-T backbone. img_sizer botnet26t_256r:rL)rcr: setdefaultrPrLrMrNrNrOrc\s rccKstdd|i|S)zY Bottleneck Transformer w/ a ResNet33-t backbone, SE attn for non Halo blocks, SiLU, sebotnet33ts_256r;rL)rgr;rPrfrNrNrOrgdsrgcKs|ddtdd|i|S)z> Bottleneck Transformer w/ ResNet50-T backbone, silu act. rbrbotnet50ts_256r<rL)rir<rdrfrNrNrOriks ricKs|ddtdd|i|S)z> Bottleneck Transformer w/ ResNet26-T backbone, silu act. rbreca_botnext26ts_256r=rL)rjr=rdrfrNrNrOrjss rjcKstdd|i|S)za HaloNet-H1. Halo attention in all stages as per the paper. NOTE: This runs very slowly! r>rL)r>rhrfrNrNrOr>{sr>cKstdd|i|S)zJ HaloNet w/ a ResNet26-t backbone. Halo attention in final two stages r?rL)r?rhrfrNrNrOr?sr?cKstdd|i|S)zc HaloNet w/ a ResNet33-t backbone, SE attn for non Halo blocks, SiLU, 1-2 Halo in stage 2,3,4. r@rL)r@rhrfrNrNrOr@sr@cKstdd|i|S)zT HaloNet w/ a ResNet50-t backbone, silu act. Halo attention in final two stages rArL)rArhrfrNrNrOrAsrAcKstdd|i|S)zT HaloNet w/ a ResNet26-t backbone, silu act. Halo attention in final two stages rBrL)rBrhrfrNrNrOrBsrBcKstdd|i|S)zG Lambda-ResNet-26-T. Lambda layers w/ conv pos in last two stages. rCrL)rCrhrfrNrNrOrCsrCcKstdd|i|S)zR Lambda-ResNet-50-TS. SiLU act. Lambda layers w/ conv pos in last two stages. rDrL)rDrhrfrNrNrOrDsrDcKs|ddtdd|i|S)zN Lambda-ResNet-26-R-T. Lambda layers w/ rel pos embed in last two stages. rbrrErL)rErdrfrNrNrOrEs rEcKstdd|i|S)z Halo + RegNetZ rFrL)rFrhrfrNrNrOrFsrFcKstdd|i|S)z3 Combo Attention (Lambda + Halo + Bot) Network lamhalobotnet50ts_256rGrL)rkrGrhrfrNrNrOrksrkcKstdd|i|S)z1 Combo Attention (Halo + Halo + Bot) Network halo2botnet50ts_256rHrL)rlrHrhrfrNrNrOrlsrl)NF)r")F)F)F)F)F)F)F)F)F)F)F)F)F)F)F)#__doc__Z timm.datarrZ_builderr _registryrrZbyobnetr r r r __all__rJrIrPr^Z default_cfgsrcrgrirjr>r?r@rArBrCrDrErFrkrlrNrNrNrOs                      q   A