a  dI"@s2dZddlZddlmZddlmZmZddlmZm Z m Z ddl m Z ddl mZmZdd lmZGd d d ejZGd d d ejZd$ddZd%ddZeeddeddeeddedddZed&edddZed'edddZed(edddZed)edd d!Zed*edd"d#ZdS)+a Selective Kernel Networks (ResNet base) Paper: Selective Kernel Networks (https://arxiv.org/abs/1903.06586) This was inspired by reading 'Compounding the Performance Improvements...' (https://arxiv.org/abs/2001.06268) and a streamlined impl at https://github.com/clovaai/assembled-cnn but I ended up building something closer to the original paper with some modifications of my own to better balance param count vs accuracy. Hacked together by / Copyright 2020 Ross Wightman N)nnIMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STD)SelectiveKernel ConvNormAct create_attn)build_model_with_cfg)register_modelgenerate_default_cfgs)ResNetcsReZdZdZddddddddejejddddffdd ZddZdd Z Z S) SelectiveKernelBasicr N@cstt||pi}t| | d}|dks2Jd|dksBJd||}||j}| pZ| } t||f|| ||d|||_t||fd| dd ||_t | ||_ | d d |_ ||_ ||_ dS) N act_layer norm_layerr z)BasicBlock only supports cardinality of 1rz0BasicBlock doest not support changing base width)stridedilationaa_layer drop_layerF) kernel_sizer apply_actTZinplace)superr__init__dict expansionrconv1rconv2rseact downsample drop_path)selfinplanesplanesrr# cardinality base_width sk_kwargs reduce_firstrfirst_dilationrr attn_layerr drop_blockr$ conv_kwargs first_planes outplanes __class__Z/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/timm/models/sknet.pyrs:    zSelectiveKernelBasic.__init__cCs*t|jjdddur&tj|jjjdSNweight)getattrr bnrinitzeros_r7r%r4r4r5zero_init_last@sz#SelectiveKernelBasic.zero_init_lastcCsj|}||}||}|jdur,||}|jdur@||}|jdurT||}||7}||}|SN)rr r!r$r#r"r%xZshortcutr4r4r5forwardDs         zSelectiveKernelBasic.forward __name__ __module__ __qualname__rrZReLUZ BatchNorm2drr=rA __classcell__r4r4r2r5rs$'rcsReZdZdZddddddddejejddddffdd ZddZd d Z Z S) SelectiveKernelBottleneckr Nrcstt||pi}t| | d}tt||d|}||}||j}| pT| } t||fddi||_ t ||f|| |||d|||_ t||fddd||_ t | ||_| dd |_||_||_dS) Nrrrr )rrgroupsrrF)rrTr)rrGrrintmathfloorrrrrr conv3rr!r"r#r$)r%r&r'rr#r(r)r*r+rr,rrr-rr.r$r/widthr0r1r2r4r5rVs.    z"SelectiveKernelBottleneck.__init__cCs*t|jjdddur&tj|jjjdSr6)r8rMr9rr:r;r7r<r4r4r5r=|sz(SelectiveKernelBottleneck.zero_init_lastcCst|}||}||}||}|jdur6||}|jdurJ||}|jdur^||}||7}||}|Sr>)rr rMr!r$r#r"r?r4r4r5rAs          z!SelectiveKernelBottleneck.forwardrBr4r4r2r5rGSs$&rGFcKstt||fi|Sr>)r r )variant pretrainedkwargsr4r4r5_create_skresnetsrRc Ks|dddddttddd |S) Ni)rrT)rUg?ZbicubicrZfc) urlZ num_classesZ input_sizeZ pool_sizeZcrop_pct interpolationmeanZstd first_conv classifierr)rVrQr4r4r5_cfgsr[ztimm/)Z hf_hub_idzconv1.0)rY)zskresnet18.ra_in1kzskresnet34.ra_in1kzskresnet50.untrainedzskresnet50d.untrainedzskresnext50_32x4d.ra_in1k)returncKsBtdddd}tftgdt|ddd|}td |fi|S) zConstructs a Selective Kernel ResNet-18 model. Different from configs in Select Kernel paper or "Compounding the Performance Improvements..." this variation splits the input channels to the selective convolutions to keep param count down. ?TZrd_ratioZ rd_divisor split_input)rararar*Fblocklayers block_argsr= skresnet18rrrRrPrQr*Z model_argsr4r4r5rgsrgcKsBtdddd}tftgdt|ddd|}td |fi|S) zConstructs a Selective Kernel ResNet-34 model. Different from configs in Select Kernel paper or "Compounding the Performance Improvements..." this variation splits the input channels to the selective convolutions to keep param count down. r]r^Tr_rrHrrbFrc skresnet34rhrir4r4r5rlsrlcKs>tdd}tftgdt|ddd|}td|fi|S)zConstructs a Select Kernel ResNet-50 model. Different from configs in Select Kernel paper or "Compounding the Performance Improvements..." this variation splits the input channels to the selective convolutions to keep param count down. Tr`rjrbFrc skresnet50rrGrRrir4r4r5rns rnc KsDtdd}tftgddddt|ddd|}td |fi|S) zConstructs a Select Kernel ResNet-50-D model. Different from configs in Select Kernel paper or "Compounding the Performance Improvements..." this variation splits the input channels to the selective convolutions to keep param count down. Trmrj deeprbF)rdreZ stem_widthZ stem_typeZavg_downrfr= skresnet50drorir4r4r5rrs  rrc KsFtdddd}tftgdddt|ddd|}td |fi|S) zConstructs a Select Kernel ResNeXt50-32x4d model. This should be equivalent to the SKNet-50 model in the Select Kernel Paper g?rpFr_rjrHrb)rdrer(r)rfr=skresnext50_32x4drorir4r4r5rss  rs)F)rS)F)F)F)F)F)__doc__rKZtorchrZ timm.datarrZ timm.layersrrrZ_builderr _registryr r Zresnetr ModulerrGrRr[Z default_cfgsrgrlrnrrrsr4r4r4r5s<    ==