a  dV@sdZddlmZmZmZddlZddlZddlmZddlm Z m Z m Z ddl m Z ddlmZmZmZmZdd lmZdd lmZmZmZmZmZdd lmZd d lmZee Z!dZ"dZ#gdZ$dZ%dZ&dgZ'd0e(e)dddZ*Gdddej+Z,Gdddej+Z-Gdddej+Z.Gdddej+Z/Gdd d ej+Z0Gd!d"d"ej+Z1Gd#d$d$eZ2d%Z3d&Z4ed'e3Gd(d)d)e2Z5ed*e3Gd+d,d,e2Z6ed-e3Gd.d/d/e2eZ7dS)1z PyTorch ConvNext model.)OptionalTupleUnionN)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)ACT2FN)BackboneOutputBaseModelOutputWithNoAttention(BaseModelOutputWithPoolingAndNoAttention$ImageClassifierOutputWithNoAttention)PreTrainedModel)add_code_sample_docstringsadd_start_docstrings%add_start_docstrings_to_model_forwardloggingreplace_return_docstrings) BackboneMixin)ConvNextConfigrzfacebook/convnext-tiny-224)rirztabby, tabby catF) drop_probtrainingcCsd|dks |s|Sd|}|jdfd|jd}|tj||j|jd}||||}|S)aF Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). Comment by Ross Wightman: This is the same as the DropConnect impl I created for EfficientNet, etc networks, however, the original name is misleading as 'Drop Connect' is a different form of dropout in a separate paper... See discussion: https://github.com/tensorflow/tpu/issues/494#issuecomment-532968956 ... I've opted for changing the layer and argument names to 'drop path' rather than mix DropConnect as a layer name and use 'survival rate' as the argument. rrr)r)dtypedevice)shapendimtorchZrandrrZfloor_div)inputrrZ keep_probrZ random_tensoroutputr$w/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/transformers/models/convnext/modeling_convnext.py drop_path@s r&csPeZdZdZd eeddfdd ZejejdddZ e d d d Z Z S) ConvNextDropPathzXDrop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).N)rreturncst||_dSN)super__init__r)selfr __class__r$r%r+Xs zConvNextDropPath.__init__ hidden_statesr(cCst||j|jSr))r&rrr,r0r$r$r%forward\szConvNextDropPath.forward)r(cCs d|jS)Nzp={})formatr)r,r$r$r% extra_repr_szConvNextDropPath.extra_repr)N) __name__ __module__ __qualname____doc__rfloatr+r Tensorr2strr4 __classcell__r$r$r-r%r'Usr'cs6eZdZdZd fdd ZejejdddZZS) ConvNextLayerNormaALayerNorm that supports two data formats: channels_last (default) or channels_first. The ordering of the dimensions in the inputs. channels_last corresponds to inputs with shape (batch_size, height, width, channels) while channels_first corresponds to inputs with shape (batch_size, channels, height, width). ư> channels_lastcs`ttt||_tt||_||_ ||_ |j dvrTt d|j |f|_ dS)N)r?channels_firstzUnsupported data format: ) r*r+r Parameterr onesweightzerosbiaseps data_formatNotImplementedErrornormalized_shape)r,rIrFrGr-r$r%r+is  zConvNextLayerNorm.__init__)xr(cCs|jdkr*tjj||j|j|j|j}n|jdkr|j }| }|j ddd}|| dj ddd}||t ||j}|j|d}|jddddf||jddddf}|S)Nr?r@rT)Zkeepdim)r)rGr rZ functionalZ layer_normrIrCrErFrr9meanpowsqrtto)r,rJZ input_dtypeusr$r$r%r2ss    ,zConvNextLayerNorm.forward)r>r?) r5r6r7r8r+r r:r2r<r$r$r-r%r=cs r=cs4eZdZdZfddZejejdddZZ S)ConvNextEmbeddingszThis class is comparable to (and inspired by) the SwinEmbeddings class found in src/transformers/models/swin/modeling_swin.py. csLttj|j|jd|j|jd|_t|jdddd|_ |j|_dS)Nr kernel_sizestrider>r@rFrG) r*r+rConv2d num_channels hidden_sizesZ patch_sizepatch_embeddingsr= layernormr,configr-r$r%r+s  zConvNextEmbeddings.__init__) pixel_valuesr(cCs4|jd}||jkrtd||}||}|S)NrzeMake sure that the channel dimension of the pixel values match with the one set in the configuration.)rrX ValueErrorrZr[)r,r^rX embeddingsr$r$r%r2s    zConvNextEmbeddings.forward r5r6r7r8r+r FloatTensorr:r2r<r$r$r-r%rRs rRcs6eZdZdZdfdd ZejejdddZZ S) ConvNextLayera3This corresponds to the `Block` class in the original implementation. There are two equivalent implementations: [DwConv, LayerNorm (channels_first), Conv, GELU,1x1 Conv]; all in (N, C, H, W) (2) [DwConv, Permute to (N, H, W, C), LayerNorm (channels_last), Linear, GELU, Linear]; Permute back The authors used (2) as they find it slightly faster in PyTorch. Args: config ([`ConvNextConfig`]): Model configuration class. dim (`int`): Number of input channels. drop_path (`float`): Stochastic depth rate. Default: 0.0. rcsttj||dd|d|_t|dd|_t|d||_t |j |_ td|||_ |j dkrtj|j t|dd nd|_|d krt|nt|_dS) Nrr )rTpaddinggroupsr>rFrT)Z requires_gradr)r*r+rrWdwconvr=r[Linearpwconv1r Z hidden_actactpwconv2Zlayer_scale_init_valuerAr rBlayer_scale_parameterr'Identityr&)r,r]dimr&r-r$r%r+s  zConvNextLayer.__init__r/cCs||}||}|dddd}||}||}||}||}|jdurZ|j|}|dddd}|||}|S)NrrKr r)rhZpermuter[rjrkrlrmr&)r,r0r"rJr$r$r%r2s       zConvNextLayer.forward)rrar$r$r-r%rcs rccs6eZdZdZd fdd ZejejdddZZ S) ConvNextStageaConvNeXT stage, consisting of an optional downsampling layer + multiple residual blocks. Args: config ([`ConvNextConfig`]): Model configuration class. in_channels (`int`): Number of input channels. out_channels (`int`): Number of output channels. depth (`int`): Number of residual blocks. drop_path_rates(`List[float]`): Stochastic depth rates for each layer. rKNc st|ks|dkrBtt|dddtj|||d|_n t|_pXdg|tjfddt|D|_ dS) Nrr>r@rVrSrcsg|]}t|dqS))ror&)rc).0jr]drop_path_rates out_channelsr$r% z*ConvNextStage.__init__..) r*r+rZ Sequentialr=rWdownsampling_layerrnrangelayers)r,r] in_channelsrurTrUdepthrtr-rsr%r+s   zConvNextStage.__init__r/cCs||}||}|Sr))rxrzr1r$r$r%r2s  zConvNextStage.forward)rKrKrKNrar$r$r-r%rps rpcsDeZdZfddZdejeeeeee e fdddZ Z S) ConvNextEncoderc stt|_ddtd|jt|j  |j D}|j d}t |j D]H}|j |}t||||dkrrdnd|j |||d}|j||}qPdS)NcSsg|] }|qSr$)tolist)rqrJr$r$r%rvsz,ConvNextEncoder.__init__..rrKr)r{rurUr|rt)r*r+rZ ModuleListstagesr ZlinspaceZdrop_path_ratesumZdepthssplitrYryZ num_stagesrpappend)r,r]rtZprev_chsiZout_chsstager-r$r%r+s$     zConvNextEncoder.__init__FT)r0output_hidden_states return_dictr(cCsj|rdnd}t|jD]\}}|r,||f}||}q|rD||f}|s^tdd||fDSt||dS)Nr$css|]}|dur|VqdSr)r$)rqvr$r$r% rwz*ConvNextEncoder.forward..)last_hidden_stater0) enumeratertupler )r,r0rrZall_hidden_statesrZ layer_moduler$r$r%r2s    zConvNextEncoder.forward)FT) r5r6r7r+r rbrboolrrr r2r<r$r$r-r%r}s  r}c@s2eZdZdZeZdZdZdZddZ d dd Z d S) ConvNextPreTrainedModelz An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models. convnextr^TcCsjt|tjtjfr@|jjjd|jjd|j durf|j j n&t|tj rf|j j |jj ddS)zInitialize the weightsr)rLZstdNg?) isinstancerrirWrCdataZnormal_r]Zinitializer_rangerEZzero_ LayerNormZfill_)r,moduler$r$r% _init_weightss   z%ConvNextPreTrainedModel._init_weightsFcCst|tr||_dSr))rr}Zgradient_checkpointing)r,rvaluer$r$r%_set_gradient_checkpointing+s z3ConvNextPreTrainedModel._set_gradient_checkpointingN)F) r5r6r7r8r config_classZbase_model_prefixZmain_input_nameZsupports_gradient_checkpointingrrr$r$r$r%rs raJ This model is a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. Parameters: config ([`ConvNextConfig`]): Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [`~PreTrainedModel.from_pretrained`] method to load the model weights. aF Args: pixel_values (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)`): Pixel values. Pixel values can be obtained using [`AutoImageProcessor`]. See [`ConvNextImageProcessor.__call__`] for details. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. zQThe bare ConvNext model outputting raw features without any specific head on top.c s^eZdZfddZeeeeee de dd e j e ee eeeefdddZZS) ConvNextModelcsJt|||_t||_t||_tj|j d|j d|_ | dS)Nrf) r*r+r]rRr`r}encoderrrrYZlayer_norm_epsr[ post_initr\r-r$r%r+Ns    zConvNextModel.__init__Zvision) checkpoint output_typerZmodalityexpected_outputNr^rrr(cCs|dur |n|jj}|dur |n|jj}|dur8td||}|j|||d}|d}||ddg}|s||f|ddSt|||j dS)Nz You have to specify pixel_valuesrrrrr)r pooler_outputr0) r]ruse_return_dictr_r`rr[rLr r0)r,r^rrembedding_outputZencoder_outputsr pooled_outputr$r$r%r2[s( zConvNextModel.forward)NNN)r5r6r7r+rCONVNEXT_INPUTS_DOCSTRINGr_CHECKPOINT_FOR_DOCr _CONFIG_FOR_DOC_EXPECTED_OUTPUT_SHAPEr rbrrrrr2r<r$r$r-r%rIs$   rz ConvNext Model with an image classification head on top (a linear layer on top of the pooled features), e.g. for ImageNet. c sdeZdZfddZeeeeee e dde j e e je ee eeeefdddZZS) ConvNextForImageClassificationcsRt||j|_t||_|jdkr 1` a classification loss is computed (Cross-Entropy). NrrZ regressionZsingle_label_classificationZmulti_label_classificationrrK)losslogitsr0)r]rrrrZ problem_typerrr longintrZsqueezerviewrrr0) r,r^rrroutputsrrrZloss_fctr#r$r$r%r2s>     "     z&ConvNextForImageClassification.forward)NNNN)r5r6r7r+rrr_IMAGE_CLASS_CHECKPOINTrr_IMAGE_CLASS_EXPECTED_OUTPUTr rbrZ LongTensorrrrr2r<r$r$r-r%rs&  rzQ ConvNeXt backbone, to be used with frameworks like DETR and MaskFormer. csPeZdZfddZeeeeedde j e e e e edddZ ZS) ConvNextBackbonecst|t|t||_t||_|jdg|j|_i}t |j |j D]\}}t |dd||<qRt ||_|dS)Nrr@)rG)r*r+Z_init_backbonerRr`r}rrYZ num_featureszipZ _out_featuresZchannelsr=rZ ModuleDicthidden_states_normsr)r,r]rrrXr-r$r%r+s     zConvNextBackbone.__init__)rrNrc Cs|dur |n|jj}|dur |n|jj}||}|j|ddd}|j}d}tt|jdd|ddD].\}\} } | |j vrl|j | | } || f7}ql|s|f} |r| |jf7} | St ||r|jndddS)az Returns: Examples: ```python >>> from transformers import AutoImageProcessor, AutoBackbone >>> import torch >>> from PIL import Image >>> import requests >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> processor = AutoImageProcessor.from_pretrained("facebook/convnext-tiny-224") >>> model = AutoBackbone.from_pretrained("facebook/convnext-tiny-224") >>> inputs = processor(image, return_tensors="pt") >>> outputs = model(**inputs) ```NTrr$r) feature_mapsr0Z attentions) r]rrr`rr0rrZ stage_namesZ out_featuresrr ) r,r^rrrrr0ridxrZ hidden_stater#r$r$r%r2s2 ,    zConvNextBackbone.forward)NN)r5r6r7r+rrrr rr r:rrr2r<r$r$r-r%rs  r)rF)8r8typingrrrr Ztorch.utils.checkpointrZtorch.nnrrrZ activationsr Zmodeling_outputsr r r rZmodeling_utilsrutilsrrrrrZutils.backbone_utilsrZconfiguration_convnextrZ get_loggerr5loggerrrrrrZ&CONVNEXT_PRETRAINED_MODEL_ARCHIVE_LISTr9rr&Moduler'r=rRrcrpr}rZCONVNEXT_START_DOCSTRINGrrrrr$r$r$r%sX      , / ;L