a  dl@sdZddlZddlZddlmZmZddlmZddlm Z e e Z ddd Z gd Zgd Zd gZgd ZddZddZddZddZeeeedZGdddeZGdddeZGdddeZdS)z Jukebox configurationN)ListUnion)PretrainedConfig)loggingzEhttps://huggingface.co/openai/jukebox-5b-lyrics/blob/main/config.jsonzEhttps://huggingface.co/openai/jukebox-1b-lyrics/blob/main/config.json)zopenai/jukebox-5b-lyricszopenai/jukebox-1b-lyrics)O block_attntranspose_block_attnprev_block_attnrrr rrr rrr rrr rrr cross_attentionrrr rrr rrr r rrr rrr rrr r rrr rrr rrr r rrr rrr rrr r rrr rrr rrr r rrr rrr rrr r )rrr Zdense_attention) prime_attnr Z dense_attncCstdS)Nr)_FullDenseAttentionZlayerrz/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/transformers/models/jukebox/configuration_jukebox.pyfull_dense_attentionvsrcCs t|dS)Nr)_RawColumnPreviousRowAttentionr rrr!raw_column_previous_row_attentionzsrcCs t|dS)NO)_LARGE_ATTENTIONr rrr large_separated_enc_dec_w_lyrics~srcCs$|ddkrt|dSt|dS)Nr)_PrimePrimeDenseAttentionrr rrrenc_dec_with_lyricss  r)rrrrc*@seZdZdZdZdddZdddd d d dd d dddddddddd ddddddgdddddddddddgdgddd dd d f*d!d"Zed)ee e j fd#d$d%d&Z d'd(Z dS)*JukeboxPriorConfiga" This is the configuration class to store the configuration of a [`JukeboxPrior`]. It is used to instantiate a `JukeboxPrior` according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the top level prior from the [openai/jukebox-1b-lyrics](https://huggingface.co/openai/jukebox -1b-lyrics) architecture. Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Args: act_fn (`str`, *optional*, defaults to `"quick_gelu"`): Activation function. alignment_head (`int`, *optional*, defaults to 2): Head that is responsible of the alignment between lyrics and music. Only used to compute the lyric to audio alignment alignment_layer (`int`, *optional*, defaults to 68): Index of the layer that is responsible of the alignment between lyrics and music. Only used to compute the lyric to audio alignment attention_multiplier (`float`, *optional*, defaults to 0.25): Multiplier coefficient used to define the hidden dimension of the attention layers. 0.25 means that 0.25*width of the model will be used. attention_pattern (`str`, *optional*, defaults to `"enc_dec_with_lyrics"`): Which attention pattern to use for the decoder/ attn_dropout (`int`, *optional*, defaults to 0): Dropout probability for the post-attention layer dropout in the decoder. attn_res_scale (`bool`, *optional*, defaults to `False`): Whether or not to scale the residuals in the attention conditioner block. blocks (`int`, *optional*, defaults to 64): Number of blocks used in the `block_attn`. A sequence of length seq_len is factored as `[blocks, seq_len // blocks]` in the `JukeboxAttention` layer. conv_res_scale (`int`, *optional*): Whether or not to scale the residuals in the conditioner block. Since the top level prior does not have a conditioner, the default value is to None and should not be modified. num_layers (`int`, *optional*, defaults to 72): Number of layers of the transformer architecture. emb_dropout (`int`, *optional*, defaults to 0): Embedding dropout used in the lyric decoder. encoder_config (`JukeboxPriorConfig`, *optional*) : Configuration of the encoder which models the prior on the lyrics. encoder_loss_fraction (`float`, *optional*, defaults to 0.4): Multiplication factor used in front of the lyric encoder loss. hidden_size (`int`, *optional*, defaults to 2048): Hidden dimension of the attention layers. init_scale (`float`, *optional*, defaults to 0.2): Initialization scales for the prior modules. is_encoder_decoder (`bool`, *optional*, defaults to `True`): Whether or not the prior is an encoder-decoder model. In case it is not, and `nb_relevant_lyric_tokens` is greater than 0, the `encoder` args should be specified for the lyric encoding. mask (`bool`, *optional*, defaults to `False`): Whether or not to mask the previous positions in the attention. max_duration (`int`, *optional*, defaults to 600): Maximum supported duration of the generated song in seconds. max_nb_genres (`int`, *optional*, defaults to 1): Maximum number of genres that can be used to condition the model. merged_decoder (`bool`, *optional*, defaults to `True`): Whether or not the decoder and the encoder inputs are merged. This is used for the separated encoder-decoder architecture metadata_conditioning (`bool`, *optional*, defaults to `True)`: Whether or not to condition on the artist and genre metadata. metadata_dims (`List[int]`, *optional*, defaults to `[604, 7898]`): Number of genres and the number of artists that were used to train the embedding layers of the prior models. min_duration (`int`, *optional*, defaults to 0): Minimum duration of the generated audio on which the model was trained. mlp_multiplier (`float`, *optional*, defaults to 1.0): Multiplier coefficient used to define the hidden dimension of the MLP layers. 0.25 means that 0.25*width of the model will be used. music_vocab_size (`int`, *optional*, defaults to 2048): Number of different music tokens. Should be similar to the `JukeboxVQVAEConfig.nb_discrete_codes`. n_ctx (`int`, *optional*, defaults to 6144): Number of context tokens for each prior. The context tokens are the music tokens that are attended to when generating music tokens. n_heads (`int`, *optional*, defaults to 2): Number of attention heads. nb_relevant_lyric_tokens (`int`, *optional*, defaults to 384): Number of lyric tokens that are used when sampling a single window of length `n_ctx` res_conv_depth (`int`, *optional*, defaults to 3): Depth of the `JukeboxDecoderConvBock` used to upsample the previously sampled audio in the `JukeboxMusicTokenConditioner`. res_conv_width (`int`, *optional*, defaults to 128): Width of the `JukeboxDecoderConvBock` used to upsample the previously sampled audio in the `JukeboxMusicTokenConditioner`. res_convolution_multiplier (`int`, *optional*, defaults to 1): Multiplier used to scale the `hidden_dim` of the `JukeboxResConv1DBlock`. res_dilation_cycle (`int`, *optional*): Dilation cycle used to define the `JukeboxMusicTokenConditioner`. Usually similar to the ones used in the corresponding level of the VQVAE. The first prior does not use it as it is not conditioned on upper level tokens. res_dilation_growth_rate (`int`, *optional*, defaults to 1): Dilation grow rate used between each convolutionnal block of the `JukeboxMusicTokenConditioner` res_downs_t (`List[int]`, *optional*, defaults to `[3, 2, 2]`): Downsampling rates used in the audio conditioning network res_strides_t (`List[int]`, *optional*, defaults to `[2, 2, 2]`): Striding used in the audio conditioning network resid_dropout (`int`, *optional*, defaults to 0): Residual dropout used in the attention pattern. sampling_rate (`int`, *optional*, defaults to 44100): Sampling rate used for training. spread (`int`, *optional*): Spread used in the `summary_spread_attention` pattern timing_dims (`int`, *optional*, defaults to 64): Dimension of the timing embedding. zero_out (`bool`, *optional*, defaults to `False`): Whether or not to zero out convolution weights when initializing. Z jukebox_priorZ n_positionsZn_head)Zmax_position_embeddingsZnum_attention_headsZ quick_gelurDg?rF@NHg?皙?TPiXi\ig?iirrrrrrrDc+,Ks||_||_||_||_||_||_||_| |_| |_| |_ | |_ ||_ | durbt fi| |_ nd|_ ||_||_||_||_||_||_||_||_||_||_||_||_||_||_||_||_||_| |_|!|_ |"|_!|#|_"|$|_#|%|_$|&|_%|'|_&|(|_'|)|_(||_)|*|_*dSN)+act_fnalignment_headalignment_layerattention_multiplierattention_pattern attn_dropoutattn_res_scaleblocksconv_res_scale num_layers emb_dropoutmusic_vocab_sizerencoder_configencoder_loss_fraction init_scaleis_encoder_decoderlyric_vocab_sizelevelmask max_duration max_nb_genresmerged_decodermetadata_conditioning metadata_dims min_durationmlp_multipliern_ctxn_headsnb_relevant_lyric_tokensres_conv_depthres_conv_widthres_convolution_multiplierres_dilation_cycleres_dilation_growth_rate res_downs_t res_strides_t resid_dropout sampling_ratespread timing_dims hidden_sizezero_out),selfr(r9r)r*r+r,r-r.r/r0r1r2r4r5rPr6r7r8r:r;r<r=r>r?r@rAr3rBrCrDrErFrGrHrIrJrKrLrMrNrOrQkwargsrrr__init__sX.zJukeboxPriorConfig.__init__rpretrained_model_name_or_pathreturncKs|j|fi|\}}|ddkr2|d|}d|vrpt|drp|d|jkrptd|dd|jd|j|fi|S)N model_typejukeboxprior_You are using a model of type to instantiate a model of type N. This is not supported for all configurations of models and can yield errors.Zget_config_dictgethasattrrXloggerwarning from_dict)clsrVr9rS config_dictrrrfrom_pretrained`s  z"JukeboxPriorConfig.from_pretrainedcCs8t|j}|jdur |jnd|d<|jj|d<|S) Serializes this instance to a Python dictionary. Override the default [`~PretrainedConfig.to_dict`]. Returns: `Dict[str, any]`: Dictionary of all the attributes that make up this configuration instance, Nr4rX)copydeepcopy__dict__r4to_dict __class__rX)rRoutputrrrrkrs  zJukeboxPriorConfig.to_dict)r)__name__ __module__ __qualname____doc__rXZ attribute_maprT classmethodrstrosPathLikerfrkrrrrrsnm \ rc@steZdZdZdZddddddgd d d gd d dddd gdgddddfddZeeee j fddddZ dS)JukeboxVQVAEConfiga This is the configuration class to store the configuration of a [`JukeboxVQVAE`]. It is used to instantiate a `JukeboxVQVAE` according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the VQVAE from [openai/jukebox-1b-lyrics](https://huggingface.co/openai/jukebox-1b-lyrics) architecture. Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Args: act_fn (`str`, *optional*, defaults to `"relu"`): Activation function of the model. nb_discrete_codes (`int`, *optional*, defaults to 2048): Number of codes of the VQVAE. commit (`float`, *optional*, defaults to 0.02): Commit loss multiplier. conv_input_shape (`int`, *optional*, defaults to 1): Number of audio channels. conv_res_scale (`bool`, *optional*, defaults to `False`): Whether or not to scale the residuals of the `JukeboxResConv1DBlock`. embed_dim (`int`, *optional*, defaults to 64): Embedding dimension of the codebook vectors. hop_fraction (`List[int]`, *optional*, defaults to `[0.125, 0.5, 0.5]`): Fraction of non-intersecting window used when continuing the sampling process. levels (`int`, *optional*, defaults to 3): Number of hierarchical levels that used in the VQVAE. lmu (`float`, *optional*, defaults to 0.99): Used in the codebook update, exponential moving average coefficient. For more detail refer to Appendix A.1 of the original [VQVAE paper](https://arxiv.org/pdf/1711.00937v2.pdf) multipliers (`List[int]`, *optional*, defaults to `[2, 1, 1]`): Depth and width multipliers used for each level. Used on the `res_conv_width` and `res_conv_depth` res_conv_depth (`int`, *optional*, defaults to 4): Depth of the encoder and decoder block. If no `multipliers` are used, this is the same for each level. res_conv_width (`int`, *optional*, defaults to 32): Width of the encoder and decoder block. If no `multipliers` are used, this is the same for each level. res_convolution_multiplier (`int`, *optional*, defaults to 1): Scaling factor of the hidden dimension used in the `JukeboxResConv1DBlock`. res_dilation_cycle (`int`, *optional*): Dilation cycle value used in the `JukeboxResnet`. If an int is used, each new Conv1 block will have a depth reduced by a power of `res_dilation_cycle`. res_dilation_growth_rate (`int`, *optional*, defaults to 3): Resnet dilation growth rate used in the VQVAE (dilation_growth_rate ** depth) res_downs_t (`List[int]`, *optional*, defaults to `[3, 2, 2]`): Downsampling rate for each level of the hierarchical VQ-VAE. res_strides_t (`List[int]`, *optional*, defaults to `[2, 2, 2]`): Stride used for each level of the hierarchical VQ-VAE. sample_length (`int`, *optional*, defaults to 1058304): Provides the max input shape of the VQVAE. Is used to compute the input shape of each level. init_scale (`float`, *optional*, defaults to 0.2): Initialization scale. zero_out (`bool`, *optional*, defaults to `False`): Whether or not to zero out convolution weights when initializing. Z jukebox_vqvaeZrelurg{Gz?r"Fr)g??rwrgGz?)rr"r" Nr$r%i&r cKs|||_||_||_||_||_||_| |_| |_| |_||_ ||_ | |_ ||_ ||_ | |_||_||_||_||_||_dSr') hop_fractionconv_input_shape sample_lengthlevels embed_dimnb_discrete_codesrFrErGrIrH multipliersrJrKlmucommitr0r(r6rQ)rRr(rrr{r0r~rzr}rrrErFrGrHrIrJrKr|r6rQrSrrrrTs(zJukeboxVQVAEConfig.__init__rrUcKs||j|fi|\}}|ddkr,|d}d|vrjt|drj|d|jkrjtd|dd|jd|j|fi|S)NrXrY vqvae_configr[r\r]r^)rdrVrSrerrrrfs  z"JukeboxVQVAEConfig.from_pretrained) rnrorprqrXrTrrrrsrtrurfrrrrrvs26 /rvc sJeZdZdZdZdZdfd d Zeee e d ddZ ddZ Z S) JukeboxConfigaW This is the configuration class to store the configuration of a [`JukeboxModel`]. Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Instantiating a configuration with the defaults will yield a similar configuration to that of [openai/jukebox-1b-lyrics](https://huggingface.co/openai/jukebox-1b-lyrics) architecture. The downsampling and stride are used to determine downsampling of the input sequence. For example, downsampling = (5,3), and strides = (2, 2) will downsample the audio by 2^5 = 32 to get the first level of codes, and 2**8 = 256 to get the second level codes. This is mostly true for training the top level prior and the upsamplers. Args: vqvae_config (`JukeboxVQVAEConfig`, *optional*): Configuration for the `JukeboxVQVAE` model. prior_config_list (`List[JukeboxPriorConfig]`, *optional*): List of the configs for each of the `JukeboxPrior` of the model. The original architecture uses 3 priors. nb_priors (`int`, *optional*, defaults to 3): Number of prior models that will sequentially sample tokens. Each prior is conditional auto regressive (decoder) model, apart from the top prior, which can include a lyric encoder. The available models were trained using a top prior and 2 upsampler priors. sampling_rate (`int`, *optional*, defaults to 44100): Sampling rate of the raw audio. timing_dims (`int`, *optional*, defaults to 64): Dimensions of the JukeboxRangeEmbedding layer which is equivalent to traditional positional embedding layer. The timing embedding layer converts the absolute and relative position in the currently sampled audio to a tensor of length `timing_dims` that will be added to the music tokens. min_duration (`int`, *optional*, defaults to 0): Minimum duration of the audios to generate max_duration (`float`, *optional*, defaults to 600.0): Maximum duration of the audios to generate max_nb_genres (`int`, *optional*, defaults to 5): Maximum number of genres that can be used to condition a single sample. metadata_conditioning (`bool`, *optional*, defaults to `True`): Whether or not to use metadata conditioning, corresponding to the artist, the genre and the min/maximum duration. Example: ```python >>> from transformers import JukeboxModel, JukeboxConfig >>> # Initializing a Jukebox configuration >>> configuration = JukeboxConfig() >>> # Initializing a model from the configuration >>> model = JukeboxModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ``` rYTNrr&rr@c  s|duri}tdtfi||_|dur@dd|D|_nZg|_t|D]J} | d| d} | duri} td| d|jtfi| qN|jj |_ ||_ ||_ ||_ ||_ ||_||_| |_tjfi| dS)NzHvqvae_config is None. initializing the JukeboxVQVAE with default values.cSsg|]}tfi|qSr)r).0 prior_configrrr Ez*JukeboxConfig.__init__..rZzQ's config is None. Initializing the JukeboxPriorConfig list with default values.)rainforvr prior_configsrangepopappendrrz nb_priorsr<rMrOr@r;r>superrT) rRrprior_config_listrrMrOr@r;r<r>rSZ prior_idxrrlrrrT2s0     zJukeboxConfig.__init__)rrcKs&dd|D}|f||d|S)z Instantiate a [`JukeboxConfig`] (or a derived class) from clip text model configuration and clip vision model configuration. Returns: [`JukeboxConfig`]: An instance of a configuration object cSsg|] }|qSrrk)rconfigrrrrirz.JukeboxConfig.from_configs..)rZvqvae_config_dictr)rdrrrSrrrr from_configs`s zJukeboxConfig.from_configscCsTt|j}t|dD]\}}||d|<q|j|d<|jj|d<|S)rgrrZrrX) rhrirj enumeraterrkrrlrX)rRrmirrrrrkls   zJukeboxConfig.to_dict) NNrr&rrrrT)rnrorprqrXZis_compositionrTrrrrrvrrk __classcell__rrrrrs 6. r)rqrhrttypingrrZconfiguration_utilsrutilsrZ get_loggerrnraZ%JUKEBOX_PRETRAINED_CONFIG_ARCHIVE_MAPrrr rrrrrZATTENTION_PATTERNSrrvrrrrrs4   Qpy