a  dE_@sdZddlZddlmZmZmZddlZddlZddlmZddl m Z m Z m Z ddl mZddlmZmZmZdd lmZdd lmZmZmZmZd d lmZeeZd ZdZ gdZ!dZ"dZ#dgZ$dZ%dZ&ee'dddZ(d2ee'efe)dddZ*Gdddej+Z,Gdddej-Z.Gdddej+Z/Gd d!d!ej+Z0Gd"d#d#ej+Z1Gd$d%d%ej+Z2Gd&d'd'ej+Z3Gd(d)d)ej+Z4Gd*d+d+eZ5ed,e%Gd-d.d.e5Z6ed/e%Gd0d1d1e5Z7dS)3z PyTorch EfficientNet model.N)OptionalTupleUnion)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)ACT2FN)BaseModelOutputWithNoAttention(BaseModelOutputWithPoolingAndNoAttention$ImageClassifierOutputWithNoAttention)PreTrainedModel)add_code_sample_docstringsadd_start_docstrings%add_start_docstrings_to_model_forwardlogging)EfficientNetConfigrzgoogle/efficientnet-b7)rirztabby, tabby cataN This model is a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. Parameters: config ([`EfficientNetConfig`]): Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [`~PreTrainedModel.from_pretrained`] method to load the model weights. aB Args: pixel_values (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)`): Pixel values. Pixel values can be obtained using [`AutoImageProcessor`]. See [`AutoImageProcessor.__call__`] for details. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. )config num_channelscCsJ|j}||j9}t|t||d||}|d|krB||7}t|S)z< Round number of filters based on depth multiplier. g?)Z depth_divisorZwidth_coefficientmaxint)rrZdivisorZnew_dimr/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/transformers/models/efficientnet/modeling_efficientnet.py round_filtersVs   rT) kernel_sizeadjustcCsrt|tr||f}|dd|ddf}|rR|dd|d|dd|dfS|d|d|d|dfSdS)aJ Utility function to get the tuple padding value for the depthwise convolution. Args: kernel_size (`int` or `tuple`): Kernel size of the convolution layers. adjust (`bool`, *optional*, defaults to `True`): Adjusts padding value to apply to right and bottom sides of the input. rrrN) isinstancer)rrcorrectrrr correct_pades $r"cs:eZdZdZedfdd ZejejdddZZ S)EfficientNetEmbeddingszL A module that corresponds to the stem module of the original work. rcshtt|d|_tjdd|_tj|j|jddddd|_ tj |j|j |j d |_ t|j|_dS) N )rrrrpaddingr rvalidFrstrider'bias)epsmomentum)super__init__rout_dimr ZeroPad2dr'Conv2dr convolution BatchNorm2dbatch_norm_epsbatch_norm_momentum batchnormr hidden_act activationselfr __class__rrr/~s  zEfficientNetEmbeddings.__init__) pixel_valuesreturncCs,||}||}||}||}|SN)r'r3r7r9)r;r>featuresrrrforwards     zEfficientNetEmbeddings.forward) __name__ __module__ __qualname____doc__rr/torchTensorrB __classcell__rrr<rr#ys r#cseZdZdfdd ZZS) EfficientNetDepthwiseConv2drr rTzerosc s*||} tj|| |||||||d dS)N) in_channels out_channelsrr*r'dilationgroupsr+ padding_mode)r.r/) r;rLZdepth_multiplierrr*r'rNr+rPrMr<rrr/s z$EfficientNetDepthwiseConv2d.__init__)rr rrrTrK)rCrDrEr/rIrrr<rrJsrJcs@eZdZdZeeeedfdd Zejej dddZ Z S)EfficientNetExpansionLayerz_ This corresponds to the expansion phase of each block in the original implementation. rin_dimr0r*csBttj||dddd|_tj||jd|_t|j |_ dS)NrsameFrLrMrr'r+) num_featuresr,) r.r/rr2 expand_convr4r5 expand_bnr r8 expand_act)r;rrSr0r*r<rrr/s z#EfficientNetExpansionLayer.__init__ hidden_statesr?cCs"||}||}||}|Sr@)rWrXrYr;r[rrrrBs   z"EfficientNetExpansionLayer.forward) rCrDrErFrrr/rG FloatTensorrHrBrIrrr<rrQs rQcsBeZdZdZeeeeedfdd Zej ej dddZ Z S)EfficientNetDepthwiseLayerzk This corresponds to the depthwise convolution phase of each block in the original implementation. rrSr*radjust_paddingcsvt||_|jdkrdnd}t||d}tj|d|_t||||dd|_tj ||j |j d|_ t |j|_dS) Nrr(rT)rr&Fr)rVr,r-)r.r/r*r"rr1depthwise_conv_padrJdepthwise_convr4r5r6depthwise_normr r8 depthwise_act)r;rrSr*rr`Zconv_padr'r<rrr/s    z#EfficientNetDepthwiseLayer.__init__rZcCs6|jdkr||}||}||}||}|S)Nr)r*rbrcrdrer\rrrrBs      z"EfficientNetDepthwiseLayer.forward rCrDrErFrrboolr/rGr]rHrBrIrrr<rr^sr^csBeZdZdZd eeeedfdd Zej ej dddZ Z S) EfficientNetSqueezeExciteLayerzl This corresponds to the Squeeze and Excitement phase of each block in the original implementation. FrrS expand_dimexpandcst|r|n||_tdt||j|_tjdd|_ tj |j|jddd|_ tj |j|jddd|_ t |j|_t|_dS)Nr)Z output_sizerT)rLrMrr')r.r/dimrrZsqueeze_expansion_ratioZdim_serZAdaptiveAvgPool2dsqueezer2reducerkr r8 act_reduceZSigmoid act_expand)r;rrSrjrkr<rrr/s$  z'EfficientNetSqueezeExciteLayer.__init__rZcCsF|}||}||}||}||}||}t||}|Sr@)rmrnrorkrprGmul)r;r[inputsrrrrBs      z&EfficientNetSqueezeExciteLayer.forward)Frfrrr<rrhsrhcsHeZdZdZeeeeeedfdd Ze j e j e j dddZ Z S)EfficientNetFinalBlockLayerz[ This corresponds to the final phase of each block in the original implementation. rrSr0r* drop_rateid_skipcsXt|dko| |_tj||dddd|_tj||j|jd|_ tj |d|_ dS)NrrTFrUrap) r.r/ apply_dropoutrr2 project_convr4r5r6 project_bnDropoutdropout)r;rrSr0r*rurvr<rrr/s  z$EfficientNetFinalBlockLayer.__init__) embeddingsr[r?cCs0||}||}|jr,||}||}|Sr@)rzr{ryr})r;r~r[rrrrB*s    z#EfficientNetFinalBlockLayer.forwardrCrDrErFrrfloatrgr/rGr]rHrBrIrrr<rrss rsc sJeZdZdZeeeeeeeeed fdd Ze j e j dddZ Z S)EfficientNetBlocka This corresponds to the expansion and depthwise convolution phase of each block in the original implementation. Args: config ([`EfficientNetConfig`]): Model configuration class. in_dim (`int`): Number of input channels. out_dim (`int`): Number of output channels. stride (`int`): Stride size to be used in convolution layers. expand_ratio (`int`): Expand ratio to set the output dimensions for the expansion and squeeze-excite layers. kernel_size (`int`): Kernel size for the depthwise convolution layer. drop_rate (`float`): Dropout rate to be used in the final phase of each block. id_skip (`bool`): Whether to apply dropout and sum the final hidden states with the input embeddings during the final phase of each block. Set to `True` for the first block of each stage. adjust_padding (`bool`): Whether to apply padding to only right and bottom side of the input kernel before the depthwise convolution operation, set to `True` for inputs with odd input sizes. ) rrSr0r* expand_ratiorrurvr`c st||_|jdkrdnd|_||} |jrDt||| |d|_t||jrR| n|||| d|_t||| |jd|_ t ||jr| n|||||d|_ dS)NrTFrRr_rirt) r.r/rrkrQ expansionr^rcrhsqueeze_exciters projection) r;rrSr0r*rrrurvr`Z expand_in_dimr<rrr/Ps4    zEfficientNetBlock.__init__rZcCs<|}|jdkr||}||}||}|||}|S)Nr)rrrcrr)r;r[r~rrrrBys     zEfficientNetBlock.forwardrrrr<rr5s)rcsFeZdZdZedfdd Zd ejee ee e ddd Z Z S) EfficientNetEncoderz Forward propogates the embeddings through each EfficientNet block. Args: config ([`EfficientNetConfig`]): Model configuration class. r$c s~t|_|j_fddt|j}tfdd|jD}d}g}t|D]}t ||j|}t ||j |}|j |} |j |} |j |} t|j|D]} | dkrdnd} | dkrdn| } | dkr|n|}||jvrdnd}|j||}t|||| | | || |d }|||d7}qqVt|_tj|t |d dd dd _tj|j|j|jd _t|j_dS)Ncsttj|Sr@)rmathceildepth_coefficient)Zrepeats)r;rr round_repeatssz3EfficientNetEncoder.__init__..round_repeatsc3s|]}|VqdSr@r).0n)rrr z/EfficientNetEncoder.__init__..rTFr) rrSr0r*rrrurvr`irTrUra)r.r/rrlenrLsumZnum_block_repeatsrangerrMstridesZ kernel_sizesZ expand_ratiosZdepthwise_paddingZdrop_connect_raterappendrZ ModuleListblocksr2top_convr4 hidden_dimr5r6top_bnr r8top_activation)r;rZnum_base_blocksZ num_blocksZcurr_block_numrirSr0r*rrjrvr`rublockr<)rr;rr/sZ           zEfficientNetEncoder.__init__FT)r[output_hidden_states return_dictr?cCst|r |fnd}|jD]}||}|r||f7}q||}||}||}|shtdd||fDSt||dS)Ncss|]}|dur|VqdSr@r)rvrrrrrz.EfficientNetEncoder.forward..)last_hidden_stater[)rrrrtupler )r;r[rrZall_hidden_statesrrrrrBs     zEfficientNetEncoder.forward)FT) rCrDrErFrr/rGr]rrgr rBrIrrr<rrs:rc@s2eZdZdZeZdZdZdZddZ d dd Z d S) EfficientNetPreTrainedModelz An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models. efficientnetr>TcCsjt|tjtjfr@|jjjd|jjd|j durf|j j n&t|tj rf|j j |jj ddS)zInitialize the weightsg)meanZstdNg?) r rLinearr2ZweightdataZnormal_rZinitializer_ranger+Zzero_Z LayerNormZfill_)r;modulerrr _init_weightss   z)EfficientNetPreTrainedModel._init_weightsFcCst|tr||_dSr@)r rZgradient_checkpointing)r;rvaluerrr_set_gradient_checkpointings z7EfficientNetPreTrainedModel._set_gradient_checkpointingN)F) rCrDrErFr config_classZbase_model_prefixZmain_input_nameZsupports_gradient_checkpointingrrrrrrrs rzUThe bare EfficientNet model outputting raw features without any specific head on top.c sdeZdZedfdd Zeeeee e de dd e j eeeeeee fddd ZZS) EfficientNetModelr$cs~t|||_t||_t||_|jdkrDtj |j dd|_ n.|jdkrbtj |j dd|_ nt d|j|dS)NrT)Z ceil_moderz2config.pooling must be one of ['mean', 'max'] got )r.r/rr#r~rencoderZ pooling_typerZ AvgPool2drpoolerZ MaxPool2d ValueErrorZpooling post_initr:r<rrr/s     zEfficientNetModel.__init__Zvision) checkpoint output_typerZmodalityexpected_outputN)r>rrr?cCs|dur |n|jj}|dur |n|jj}|dur8td||}|j|||d}|d}||}||jdd}|s||f|ddSt |||j dS)Nz You have to specify pixel_valuesrrrrr)r pooler_outputr[) rruse_return_dictrr~rrZreshapeshaper r[)r;r>rrZembedding_outputZencoder_outputsr pooled_outputrrrrBs*  zEfficientNetModel.forward)NNN)rCrDrErr/rEFFICIENTNET_INPUTS_DOCSTRINGr_CHECKPOINT_FOR_DOCr _CONFIG_FOR_DOC_EXPECTED_OUTPUT_SHAPErGr]rrgrrrBrIrrr<rrs$  rz EfficientNet Model with an image classification head on top (a linear layer on top of the pooled features), e.g. for ImageNet. c sdeZdZfddZeeeeee e dde j e e je ee eeeefdddZZS) "EfficientNetForImageClassificationcsrt||j|_||_t||_tj|jd|_ |jdkrNt |j |jnt |_ tjdd|_|dS)Nrwrr)rl)r.r/ num_labelsrrrrr|Z dropout_rater}rrZIdentity classifierZSoftmaxclassifier_actrr:r<rrr/Gs  $z+EfficientNetForImageClassification.__init__)rrrrN)r>labelsrrr?c Cs|dur |n|jj}|j|||d}|r.|jn|d}||}||}||}d}|dur@|jjdur|jdkrd|j_n4|jdkr|j t j ks|j t j krd|j_nd|j_|jjdkrt } |jdkr| ||}n | ||}nN|jjdkr"t} | |d|j|d}n|jjdkr@t} | ||}|sp|f|dd} |durl|f| S| St|||jd S) a labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the image classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). NrrZ regressionZsingle_label_classificationZmulti_label_classificationr)losslogitsr[)rrrrr}rrZ problem_typerZdtyperGlongrrrmrviewrr r[) r;r>rrroutputsrrrZloss_fctoutputrrrrBTsB       "     z*EfficientNetForImageClassification.forward)NNNN)rCrDrEr/rrr_IMAGE_CLASS_CHECKPOINTr r_IMAGE_CLASS_EXPECTED_OUTPUTrGr]rZ LongTensorrgrrrBrIrrr<rr?s&  r)T)8rFrtypingrrrrGZtorch.utils.checkpointrZtorch.nnrrrZ activationsr Zmodeling_outputsr r r Zmodeling_utilsrutilsrrrrZconfiguration_efficientnetrZ get_loggerrCloggerrrrrrZ*EFFICIENTNET_PRETRAINED_MODEL_ARCHIVE_LISTZEFFICIENTNET_START_DOCSTRINGrrrrgr"Moduler#r2rJrQr^rhrsrrrrrrrrrsT      ''!QZ?