a  d@sRddlZddlZddlmZmZmZmZmZmZddl Z ddl Z ddl m Z ddlmZeeZdZGdddZGd d d ZGd d d eZGd ddeZGdddeZGdddeZGdddeZGdddeZGdddeZGdddeZGdddeZGdddeZGdd d eZ e!e j"e!d!d"d#Z#d$d%Z$e!e j"e!e!eee!d&d'd(Z%Gd)d*d*eZ&Gd+d,d,eZ'Gd-d.d.eZ(Gd/d0d0eZ)Gd1d2d2eZ*Gd3d4d4eZ+Gd5d6d6eZ,Gd7d8d8eZ-Gd9d:d:eZ.Gd;d<dd>eZ0Gd?d@d@eZ1GdAdBdBeZ2GdCdDdDeZ3dS)EN)CallableIterableListOptionalTupleUnion)add_start_docstrings) get_loggeraP Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Indices can be obtained using [`BertTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) scores (`torch.FloatTensor` of shape `(batch_size, config.vocab_size)`): Prediction scores of a language modeling head. These can be logits for each vocabulary when not using beam search or log softmax for each vocabulary token when using beam search kwargs: Additional logits processor specific kwargs. Return: `torch.FloatTensor` of shape `(batch_size, config.vocab_size)`: The processed prediction scores. c@s0eZdZdZeeejejejdddZ dS)LogitsProcessorzSAbstract base class for all logit processors that can be applied during generation. input_idsscoresreturncCst|jddS)z#Torch method for processing logits.H is an abstract class. Only classes inheriting this class can be called.NNotImplementedError __class__selfr rro/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/transformers/generation/logits_process.py__call__6s zLogitsProcessor.__call__N __name__ __module__ __qualname____doc__r !LOGITS_PROCESSOR_INPUTS_DOCSTRINGtorch LongTensor FloatTensorrrrrrr 3sr c@s0eZdZdZeeejejejdddZ dS) LogitsWarperzjAbstract base class for all logit warpers that can be applied during generation with multinomial sampling.r cCst|jddS)z Torch method for warping logits.rNrrrrrrAs zLogitsWarper.__call__Nrrrrrr">sr"c@s0eZdZdZeeejejejdddZ dS)LogitsProcessorLista This class can be used to create a list of [`LogitsProcessor`] or [`LogitsWarper`] to subsequently process a `scores` input tensor. This class inherits from list and adds a specific *__call__* method to apply each [`LogitsProcessor`] or [`LogitsWarper`] to the inputs. r c s|D]}t|jj}t|dkr|tfddt|ddDshtdt|d|j d|||fi}q|||}q|S)Nrc3s|]}|vVqdSNr).0argkwargsrr Uz/LogitsProcessorList.__call__..z,Make sure that all the required parameters: z for z$ are passed to the logits processor.) inspect signaturer parameterslenalllistkeys ValueErrorr)rr rr( processorZ function_argsrr'rrPs & zLogitsProcessorList.__call__Nrrrrrr#Isr#c@sDeZdZdZeeeeefdddZej ej ej dddZ dS) MinLengthLogitsProcessora [`LogitsProcessor`] enforcing a min-length by setting EOS probability to 0. Args: min_length (`int`): The minimum length below which the score of `eos_token_id` is set to `-float("Inf")`. eos_token_id (`Union[int, List[int]]`): The id of the *end-of-sequence* token. Optionally, use a list to set multiple *end-of-sequence* tokens. ) min_length eos_token_idcCstt|tr|dkr td|t|tr0|g}tdd|DrTtdd|Drdtd|||_||_dS)Nrz6`min_length` has to be a non-negative integer, but is cSsg|]}t|tqSr isinstanceintr%irrr qr*z5MinLengthLogitsProcessor.__init__..cSsg|] }|dkqSrrr:rrrr<qr*=`eos_token_id` has to be a list of positive integers, but is ) r8r9r2r/anyloggerwarningr5r6)rr5r6rrr__init__ks $z!MinLengthLogitsProcessor.__init__r cCs:|jd}||jkr6|jD]}td |dd|f<q|SNinf)shaper5r6floatrr rcur_lenr;rrrrws    z!MinLengthLogitsProcessor.__call__N rrrrr9rrrBrr r!rrrrrr4`s  r4c@sFeZdZdZeeeeeefdddZej ej ej dddZ dS) !MinNewTokensLengthLogitsProcessora [`LogitsProcessor`] enforcing a min-length of new tokens by setting EOS (End-Of-Sequence) token probability to 0. Args: prompt_length_to_skip (`int`): The input tokens length. min_new_tokens (`int`): The minimum *new* tokens length below which the score of `eos_token_id` is set to `-float("Inf")`. eos_token_id (`Union[int, List[int]]`): The id of the *end-of-sequence* token. Optionally, use a list to set multiple *end-of-sequence* tokens. )prompt_length_to_skipmin_new_tokensr6cCsd|fd|ffD].\}}t|tr*|dkrtd|d|qt|trP|g}tdd|Drttdd|Drtd |||_||_||_ dS) NrLrMr`z'` has to be a positive integer, but is cSsg|]}t|tqSrr7r:rrrr<r*z>MinNewTokensLengthLogitsProcessor.__init__..cSsg|] }|dkqSr=rr:rrrr<r*r>) r8r9r2r/r?r@rArLrMr6)rrLrMr6Zarg_name arg_valuerrrrBs  $z*MinNewTokensLengthLogitsProcessor.__init__r cCs@|jd|j}||jkr<|jD]}td |dd|f<q |SrC)rFrLrMr6rG)rr rZnew_tokens_lengthr;rrrrs   z*MinNewTokensLengthLogitsProcessor.__call__NrJrrrrrKs rKc@s6eZdZdZedddZejejejdddZ dS) TemperatureLogitsWarperz [`LogitsWarper`] for temperature (exponential scaling output probability distribution). Args: temperature (`float`): The value used to module the logits distribution.  temperaturecCs*t|tr|dks td|||_dS)Nrz:`temperature` has to be a strictly positive float, but is )r8rGr2rR)rrRrrrrBsz TemperatureLogitsWarper.__init__r cCs||j}|Sr$rQrrrrrs z TemperatureLogitsWarper.__call__N) rrrrrGrBrTensorr!rrrrrrPsrPc@s6eZdZdZedddZejejejdddZ dS) RepetitionPenaltyLogitsProcessora# [`LogitsProcessor`] enforcing an exponential penalty on repeated sequences. Args: repetition_penalty (`float`): The parameter for repetition penalty. 1.0 means no penalty. See [this paper](https://arxiv.org/pdf/1909.05858.pdf) for more details. )penaltycCs*t|tr|dks td|||_dS)Nr6`penalty` has to be a strictly positive float, but is )r8rGr2rU)rrUrrrrBsz)RepetitionPenaltyLogitsProcessor.__init__r cCs>t|d|}t|dk||j||j}|d|||SNr)rgatherwhererUscatter_rr rZscorerrrrsz)RepetitionPenaltyLogitsProcessor.__call__N) rrrrrGrBrr r!rrrrrrTs rTc@s:eZdZdZeejdddZejejejdddZ dS) 'EncoderRepetitionPenaltyLogitsProcessoraq [`LogitsProcessor`] enforcing an exponential penalty on tokens that are not in the original input. Args: hallucination_penalty (`float`): The parameter for hallucination penalty. 1.0 means no penalty. encoder_input_ids (`torch.LongTensor`): The encoder_input_ids that should not be repeated within the decoder ids. )rUencoder_input_idscCs4t|tr|dks td|d||_||_dS)NrrVrX)r8rGr2rUr^)rrUr^rrrrBs z0EncoderRepetitionPenaltyLogitsProcessor.__init__r cCsBt|d|j}t|dk||j||j}|d|j||SrW)rrYr^rZrUr[r\rrrrsz0EncoderRepetitionPenaltyLogitsProcessor.__call__N) rrrrrGrr rBr!rrrrrr]s r]c@sFeZdZdZed dfeeedddZejej ej ddd Z d S) TopPLogitsWarperaD [`LogitsWarper`] that performs top-p, i.e. restricting to top tokens summing to prob_cut_off <= prob_cut_off. Args: top_p (`float`): If set to < 1, only the smallest set of most probable tokens with probabilities that add up to `top_p` or higher are kept for generation. filter_value (`float`, *optional*, defaults to `-float("Inf")`): All filtered values will be set to this float value. min_tokens_to_keep (`int`, *optional*, defaults to 1): Minimum number of tokens that cannot be filtered. InfrX)top_p filter_valuemin_tokens_to_keepcCs<t|}|dks|dkr&td|||_||_||_dS)Nrg?z.`top_p` has to be a float > 0 and < 1, but is )rGr2rarbrc)rrarbrcrrrrBs zTopPLogitsWarper.__init__r cCsrtj|dd\}}|jddjdd}|d|jk}|jdkrRd|d|j df<|d||}|||j}|S)NFZ descendingrDZdimrXr.) rsortsoftmaxcumsumrarcscatter masked_fillrb)rr r sorted_logitssorted_indicescumulative_probssorted_indices_to_removeindices_to_removerrrrs zTopPLogitsWarper.__call__N rrrrrGr9rBrr r!rrrrrr_s  r_c@sFeZdZdZed dfeeedddZejej ej ddd Z d S) TopKLogitsWarpera [`LogitsWarper`] that performs top-k, i.e. restricting to the k highest probability elements. Args: top_k (`int`): The number of highest probability vocabulary tokens to keep for top-k-filtering. filter_value (`float`, *optional*, defaults to `-float("Inf")`): All filtered values will be set to this float value. min_tokens_to_keep (`int`, *optional*, defaults to 1): Minimum number of tokens that cannot be filtered. r`rX)top_krbrccCs6t|tr|dkr td|t|||_||_dS)Nrz6`top_k` has to be a strictly positive integer, but is )r8r9r2maxrrrb)rrrrbrcrrrrB$s zTopKLogitsWarper.__init__r cCs<t|j|d}|t||ddk}|||j}|S)NrDr.rDN)minrrsizertopkrjrb)rr rrrrorrrr+szTopKLogitsWarper.__call__Nrprrrrrqs rqc@sHeZdZdZded dfeeedddZejej ej dd d Z d S) TypicalLogitsWarpera" [`LogitsWarper`] that performs typical decoding. See [Typical Decoding for Natural Language Generation](https://arxiv.org/abs/2202.00666) for more information. Args: mass (`float`): Value of typical_p between 0 and 1 inclusive, defaults to 0.9. filter_value (`float`, *optional*, defaults to `-float("Inf")`): All filtered values will be set to this float value. min_tokens_to_keep (`int`, *optional*, defaults to 1): Minimum number of tokens that cannot be filtered. g?r`rX)massrbrccCs<t|}|dkr|dks&td|||_||_||_dS)NrrXz2`typical_p` has to be a float > 0 and < 1, but is )rGr2rbryrc)rryrbrcrrrrBAs zTypicalLogitsWarper.__init__r cCstjjj|dd}t|}||jddd }t| |}tj|dd\}}|d|} | j ddj dd} | |j kj dd} d| | dk<||d| ddk} |jdkrd| d d|jf<| d|| } || |j}|S) NrDreT)ZkeepdimFrdrXr.)rnn functional log_softmaxexpZnansumabsrfrYrgrhrysumviewrcrirjrb)rr r normalizedpentZshifted_scoresZ sorted_scoresrlrkrmZlast_indrnrorrrrJs    zTypicalLogitsWarper.__call__Nrprrrrrx3s rxc@sFeZdZdZed dfeeedddZejej ej ddd Z d S) EpsilonLogitsWarpera [`LogitsWarper`] that performs epsilon-sampling, i.e. restricting to tokens with `prob >= epsilon`. Takes the largest min_tokens_to_keep tokens if no tokens satisfy this constraint. See [Truncation Sampling as Language Model Desmoothing](https://arxiv.org/abs/2210.15191) for more information. Args: epsilon (`float`): If set to > 0, only the most tokens with probabilities `epsilon` or higher are kept for generation. filter_value (`float`, *optional*, defaults to `-float("Inf")`): All filtered values will be set to this float value. min_tokens_to_keep (`int`, *optional*, defaults to 1): Minimum number of tokens that cannot be filtered. r`rXepsilonrbrccCsZt|}|dks|dkr&td|t|}|dkrDtd|||_||_||_dS)NrrXz7`epsilon_cutoff` has to be a float > 0 and < 1, but is C`min_tokens_to_keep` has to be a strictly positive integer, but is )rGr2r9rrbrcrrrbrcrrrrBrszEpsilonLogitsWarper.__init__r cCsV|jdd}||jk}t|j|d}||t||ddk@}|||j}|S)NrDrerrt) rgrrurcrvrrwrjrb)rr r probabilitiesrorrrrrrs   zEpsilonLogitsWarper.__call__Nrprrrrrcsrc@sFeZdZdZed dfeeedddZejej ej ddd Z d S) EtaLogitsWarpera [`LogitsWarper`] that performs eta-sampling, i.e. calculates a dynamic cutoff `eta := min(epsilon, sqrt(epsilon, e^-entropy(probabilities)))` and restricts to tokens with `prob >= eta`. Takes the largest min_tokens_to_keep tokens if no tokens satisfy this constraint. See [Truncation Sampling as Language Model Desmoothing](https://arxiv.org/abs/2210.15191) for more information. Args: min_tokens_to_keep (`int`, *optional*, defaults to 1): Minimum number of tokens that cannot be filtered.r`rXrcCs`t|}|dks|dkr&td|t|}|dkrDtd|t||_||_||_dS)NrrXz3`eta_cutoff` has to be a float > 0 and < 1, but is r)rGr2r9rZtensorrrbrcrrrrrBs zEtaLogitsWarper.__init__r cCs|jdd}tjj|d}t|jt|jt| d}||k}t|j | d}||t ||ddk@}| ||j }|S)NrDre)Zlogits).Nrrt)rgr distributionsZ Categoricalentropyrursqrtr}rcrvrwrjrb)rr rrretarorrrrrrs &zEtaLogitsWarper.__call__Nrprrrrrs r) ngram_sizeprev_input_ids num_hyposcsddt|D}t|D]b}||||}tfddt|DD].}t|dd}||g|dg||<qLq|S)NcSsg|]}iqSrrr%_rrrr<r*z_get_ngrams..csg|]}|dqSr$rr:Z gen_tokensrrr<r*rD)rangetolistziptupleget)rrrgenerated_ngramsidxZgenerated_ngramZngramZprev_ngram_tuplerrr _get_ngramss  rcCs,|d|}t|||}||gS)NrX)rrr)Z banned_ngramsrrrIZ start_idxZ ngram_idxrrr_get_generated_ngramss r)rrrrIrcsJdkrddt|DSt|fddt|D}|S)z6Copied from fairseq for no_repeat_ngram in beam_searchrXcSsg|]}gqSrrrrrrr<r*z-_calc_banned_ngram_tokens..cs"g|]}t||qSr)rr%Zhypo_idxrIrrrrrr<s)rr)rrrrI banned_tokensrrr_calc_banned_ngram_tokenss  rc@s6eZdZdZedddZejejejdddZ dS) NoRepeatNGramLogitsProcessora5 [`LogitsProcessor`] that enforces no repetition of n-grams. See [Fairseq](https://github.com/pytorch/fairseq/blob/a07cb6f40480928c9e0548b737aadd36ee66ac76/fairseq/sequence_generator.py#L345). Args: ngram_size (`int`): All ngrams of size `ngram_size` can only occur once. )rcCs*t|tr|dkr td|||_dS)Nrz;`ngram_size` has to be a strictly positive integer, but is )r8r9r2r)rrrrrrBsz%NoRepeatNGramLogitsProcessor.__init__r cCsL|jd}|jd}t|j|||}t|D]\}}td |||f<q,|S)NrrDrE)rFrr enumeraterG)rr rZnum_batch_hypothesesrIbanned_batch_tokensr;rrrrrs   z%NoRepeatNGramLogitsProcessor.__call__N rrrrr9rBrr r!rrrrrrs rc@s:eZdZdZeejdddZejejejdddZ dS) #EncoderNoRepeatNGramLogitsProcessora [`LogitsProcessor`] that enforces no repetition of encoder input ids n-grams for the decoder ids. See [ParlAI](https://github.com/facebookresearch/ParlAI/blob/master/parlai/core/torch_generator_agent.py#L1350). Args: encoder_ngram_size (`int`): All ngrams of size `ngram_size` can only occur within the encoder input ids. encoder_input_ids (`int`): The encoder_input_ids that should not be repeated within the decoder ids. )encoder_ngram_sizer^cCs^t|tr|dkr td|||_t|jdkr>|d}|jd|_t|||j|_ dS)NrzC`encoder_ngram_size` has to be a strictly positive integer, but is rX) r8r9r2rr.rF unsqueeze batch_sizerr)rrr^rrrrBs  z,EncoderNoRepeatNGramLogitsProcessor.__init__r csb|jd}|jjdfddt|D}t|D]\}}td |||f<qB|S)NrrDcs*g|]"}tj||jqSr)rrrrrIr num_beamsrrrr<sz@EncoderNoRepeatNGramLogitsProcessor.__call__..rE)rFrrrrG)rr rrrr;rrrrr s   z,EncoderNoRepeatNGramLogitsProcessor.__call__N) rrrrr9rr rBr!rrrrrrs  rc@seZdZdZeeeeeeefdddZej ej ej dddZ ej ej dd d Z eeeeed d d ZeeeeedddZejeeeejdddZdS)NoBadWordsLogitsProcessoraI [`LogitsProcessor`] that enforces that specified sequences will never be sampled. Args: bad_words_ids (`List[List[int]]`): List of list of token ids that are not allowed to be generated. In order to get the token ids of the words that should not appear in the generated text, use `tokenizer(bad_words, add_prefix_space=True, add_special_tokens=False).input_ids`. eos_token_id (`Union[int, List[int]]`): The id of the *end-of-sequence* token. Optionally, use a list to set multiple *end-of-sequence* tokens. ) bad_words_idsr6cst|trt|dkr&td|dtdd|DrHtd|dtdd|Drjtd|ddurvgttrgttfd d |}g|_g|_ |D].}t|d kr|j |dq|j |qd|_ |j D] }t|dkrtd |d qdS)Nrz3`bad_words_ids` has to be a non-empty list, but is .css|]}t|t VqdSr$)r8r0r%Z bad_word_idsrrrr)-r*z5NoBadWordsLogitsProcessor.__init__..z2`bad_words_ids` has to be a list of lists, but is css |]}tdd|DVqdS)css(|] }t|ttjf p|dkVqdS)rN)r8r9npinteger)r%Ztoken_idrrrr)0r*z?NoBadWordsLogitsProcessor.__init__...N)r?rrrrr)/szKEach list in `bad_words_ids` has to be a list of positive integers, but is cstfddDS)Ncsg|]}|gkqSrrr:Z bad_token_seqrrr<=r*zHNoBadWordsLogitsProcessor.__init__....)r/rr6rr=r*z4NoBadWordsLogitsProcessor.__init__..rXzBanned words token sequences z cannot have an empty list) r8rr.r2r?r9r0filterbad_words_id_length_1"bad_words_id_length_greater_than_1appendstatic_bad_words_mask)rrr6wordbanned_token_seqrrrrB*s6     z"NoBadWordsLogitsProcessor.__init__r cCsB|jdur$t|jdkr$|||_||}|||}|S)Nr)rr.r_calc_static_bad_word_mask_calc_banned_bad_words_idsr$_set_scores_to_inf_for_banned_tokens)rr rZdynamic_banned_tokensrrrrMs   z"NoBadWordsLogitsProcessor.__call__)rrcCs0t|jd}d||j<|d|jSrW)rzerosrFrrtodevicebool)rrrrrrrVs z4NoBadWordsLogitsProcessor._calc_static_bad_word_mask) prev_tokenstokensrcCs>t|dkrdSt|t|kr$dS|t| d|kSdS)NrTF)r.)rrrrrr _tokens_match[s  z'NoBadWordsLogitsProcessor._tokens_match)rrcCsNg}|D]@}g}|jD]&}|||ddr||dq||q|SNrD)rrr)rrrZprev_input_ids_sliceZbanned_tokens_slicerrrrres  z4NoBadWordsLogitsProcessor._calc_banned_bad_words_ids)rrrc Csg}t|D]B\}}|D]4}||jdkr:|||gqtd|dqq |sb|jdurb|S|rtt|}tj ||j d}tj | || |j }|jdurt||j}n|j}||td }|SdS)a Modifies the scores in place by setting the banned token positions to `-inf`. Banned token is expected to be a list of list of banned tokens to ban in the format [[batch index, vocabulary position],... Args: scores: logits distribution of shape (batch size, vocabulary size) banned_tokens: list of list of tokens to ban of length (batch_size) rXz#An invalid bad word ID is defined: zG. This ID is not contained in the vocabulary, and is therefore ignored.N)rrE)rrFrr@errorrrZonesr.r rsparsetrvrZto_denserZ bitwise_orrjrG) rrrZbanned_mask_listrZbatch_banned_tokenstokenindicesZ banned_maskrrrrqs,    z>NoBadWordsLogitsProcessor._set_scores_to_inf_for_banned_tokensN)rrrrrr9rrBrr r!rZ BoolTensorrrrrrrSrrrrrrs $#   rc@sJeZdZdZeeejgeefedddZ ej ej ej dddZ dS) PrefixConstrainedLogitsProcessora [`LogitsProcessor`] that enforces constrained generation and is useful for prefix-conditioned constrained generation. See [Autoregressive Entity Retrieval](https://arxiv.org/abs/2010.00904) for more information. Args: prefix_allowed_tokens_fn (`Callable[[int, torch.Tensor], List[int]]`): This function constraints the beam search to allowed tokens only at each step. This function takes 2 arguments `inputs_ids` and the batch ID `batch_id`. It has to return a list with the allowed tokens for the next generation step conditioned on the previously generated tokens `inputs_ids` and the batch ID `batch_id`. )prefix_allowed_tokens_fnrcCs||_||_dSr$)_prefix_allowed_tokens_fn _num_beams)rrrrrrrBsz)PrefixConstrainedLogitsProcessor.__init__r c Cslt|tj }t|d|j|jdD]8\}}t|D]&\}}d|||j||||f<q:q*||S)NrDr) rZ full_likemathrErrrrFr)rr rmaskZbatch_idZ beam_sentZbeam_idsentrrrrs ""z)PrefixConstrainedLogitsProcessor.__call__N) rrrrrr9rrSrrBr r!rrrrrrs "rc@s@eZdZdZeeedddZejej ejeej dddZ dS) HammingDiversityLogitsProcessora [`LogitsProcessor`] that enforces diverse beam search. Note that this logits processor is only effective for [`PreTrainedModel.group_beam_search`]. See [Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models](https://arxiv.org/pdf/1610.02424.pdf) for more details. Args: diversity_penalty (`float`): This value is subtracted from a beam's score if it generates a token same as any beam from other group at a particular time. Note that `diversity_penalty` is only effective if `group beam search` is enabled. num_beams (`int`): Number of beams used for group beam search. See [this paper](https://arxiv.org/pdf/1610.02424.pdf) for more details. num_beam_groups (`int`): Number of groups to divide `num_beams` into in order to ensure diversity among different groups of beams. See [this paper](https://arxiv.org/pdf/1610.02424.pdf) for more details. )diversity_penaltyrnum_beam_groupscCsxt|tr|dkstd||_t|tr2|dkr:td||_t|trR|dkrZtd||krjtd|||_dS)Nz=`diversity_penalty` should be a float strictly larger than 0.rz8`num_beams` should be an integer strictly larger than 1.z>`num_beam_groups` should be an integer strictly larger than 1.z8`beam_groups` has to be smaller or equal to `num_beams`.)r8rGr2_diversity_penaltyr9r_num_sub_beams)rrrrrrrrBsz(HammingDiversityLogitsProcessor.__init__)r rcurrent_tokensbeam_group_idxrc Cs|jd|j}||j}t||j|j}||}|jd} |dkrJ|St|D]\} || |j| |j|} tj| | d|j} || || d||j | 8<qR|S)NrrD)Z minlengthrX) rFrrrurrZbincountrrr) rr rrrrZgroup_start_idxZ group_end_idxZ group_sizeZ vocab_sizeZ batch_idxZprevious_group_tokensZtoken_frequencyrrrrs    (z(HammingDiversityLogitsProcessor.__call__Nrprrrrrsrc@s6eZdZdZedddZejejejdddZ dS) ForcedBOSTokenLogitsProcessorz [`LogitsProcessor`] that enforces the specified token as the first generated token. Args: bos_token_id (`int`): The id of the token to force as the first generated token.  bos_token_idcCs ||_dSr$r)rrrrrrBsz&ForcedBOSTokenLogitsProcessor.__init__r csZ|jd}|dkrV|jd}td |ddfddt|Df<d|ddjf<|S)NrDrXrEcsg|]}|jkr|qSrrr:rrrr< r*z:ForcedBOSTokenLogitsProcessor.__call__..r)rFrGrr)rr rrI num_tokensrrrrs   (z&ForcedBOSTokenLogitsProcessor.__call__Nrrrrrrsrc@sDeZdZdZeeeeefdddZej ej ej dddZ dS) ForcedEOSTokenLogitsProcessora [`LogitsProcessor`] that enforces the specified token as the last generated token when `max_length` is reached. Args: max_length (`int`): The maximum length of the sequence to be generated. eos_token_id (`Union[int, List[int]]`): The id of the token to force as the last generated token when `max_length` is reached. Optionally, use a list to set multiple *end-of-sequence* tokens. ) max_lengthr6cCs ||_t|tr|g}||_dSr$)rr8r9r6)rrr6rrrrBs z&ForcedEOSTokenLogitsProcessor.__init__r csj|jd}|jdkrf|jd}td |ddfddt|Df<jD]}d|dd|f<qP|S)NrDrXrEcsg|]}|jvr|qSrrr:rrrr<$r*z:ForcedEOSTokenLogitsProcessor.__call__..r)rFrrGrr6)rr rrIrr;rrrr s  ( z&ForcedEOSTokenLogitsProcessor.__call__NrJrrrrrs rc@s(eZdZdZejejejdddZdS)InfNanRemoveLogitsProcessorz [`LogitsProcessor`] that removes all `nan` and `inf` values to avoid the generation method to fail. Note that using the logits processor should only be used if necessary since it can slow down the generation method. r cCs*d|||k<t|jj||tdk<|S)NrrE)rZfinfoZdtypersrGrrrrr0s z$InfNanRemoveLogitsProcessor.__call__N)rrrrrr r!rrrrrr*src@sNeZdZdZeeefeeeefedddZ e j e j e j dddZ dS) ExponentialDecayLengthPenaltya [`LogitsProcessor`] that exponentially increases the score of the eos_token_id after regulation_start has been reached. Args: exponential_decay_length_penalty (`tuple(int, float)`): This tuple shall consist of: `(start_index, decay_factor)` where `start_index` indicates where penalty starts and `decay_factor` represents the factor of exponential decay eos_token_id (`Union[int, List[int]]`): The id of the *end-of-sequence* token. Optionally, use a list to set multiple *end-of-sequence* tokens. input_ids_seq_length (`int`): The length of the input sequence. ) exponential_decay_length_penaltyr6input_ids_seq_lengthcCs2|d||_|d|_t|tr(|g}||_dS)NrrX)regulation_startregulation_factorr8r9r6)rrr6rrrrrBIs   z&ExponentialDecayLengthPenalty.__init__r cCsR|jd}||jkrN|jD]2}|dd|ft|j||j|dd|f<q|Sr)rFrr6powrrHrrrrUs    0z&ExponentialDecayLengthPenalty.__call__N)rrrrrr9rGrrrBrrSr!rrrrrr:s   rc@s(eZdZdZejejejdddZdS)LogitNormalizationa [`LogitsWarper`] and [`LogitsProcessor`] for normalizing the scores using log-softmax. It's important to normalize the scores during beam search, after applying the logits processors or warpers, since the search algorithm used in this library doesn't do it (it only does it before, but they may need re-normalization) but it still supposes that the scores are normalized when comparing the hypotheses. r cCs|jdd}|S)NrDre)r|rrrrres zLogitNormalization.__call__N)rrrrrrSrrrrrr]src@s eZdZdZddZddZdS)$SuppressTokensAtBeginLogitsProcessora [`SuppressTokensAtBeginLogitsProcessor`] supresses a list of tokens as soon as the `generate` function starts generating using `begin_index` tokens. This should ensure that the tokens defined by `begin_suppress_tokens` at not sampled at the begining of the generation. cCst||_||_dSr$)r0begin_suppress_tokens begin_index)rrrrrrrBqs z-SuppressTokensAtBeginLogitsProcessor.__init__cCs,|jd|jkr(td |dd|jf<|S)NrXrE)rFrrGrrrrrrusz-SuppressTokensAtBeginLogitsProcessor.__call__NrrrrrBrrrrrrjsrc@s eZdZdZddZddZdS)SuppressTokensLogitsProcessorzThis processor can be used to suppress a list of tokens. The processor will set their log probs to `-inf` so that they are not sampled.cCst||_dSr$)r0suppress_tokens)rrrrrrBsz&SuppressTokensLogitsProcessor.__init__cCstd |dd|jf<|S)NrE)rGrrrrrrsz&SuppressTokensLogitsProcessor.__call__Nrrrrrr|src@s.eZdZdZeeedddZddZdS)ForceTokensLogitsProcessoraThis processor takes a list of pairs of integers which indicates a mapping from generation indices to token indices that will be forced before sampling. The processor will set their log probs to `inf` so that they are sampled at their corresponding index.)force_token_mapcCst||_dSr$)dictr)rrrrrrBsz#ForceTokensLogitsProcessor.__init__cCsN|jd}|j|d}|durJtd |ddddf<d|dd|f<|S)NrDrEr)rFrrrG)rr rZgeneration_idx current_tokenrrrrs  z#ForceTokensLogitsProcessor.__call__N)rrrrrr9rBrrrrrrsrc@s eZdZdZddZddZdS)WhisperTimeStampLogitsProcessora{ Whisper specific Processor. This processor can be used to force a list of tokens. The processor will set their log probs to `inf` so that they are sampled at their corresponding index. Args: generate_config (`GenerateConfig`): The generate config used to generate the output. The following parameters are required: eos_token_id (`int`, *optional*, defaults to 50257): The id of the *end-of-sequence* token. no_timestamps_token_id (`int`, *optional*, defaults to 50363): The id of the `"<|notimestamps|>"` token. max_initial_timestamp_index (`int`, *optional*, defaults to 1): Used to set the maximum value of the initial timestamp. This is used to prevent the model from predicting timestamps that are too far in the future. cCsZ|j|_|j|_|jd|_t|jd|_|jdd|jkrN|jd8_|j|_dS)NrXrrD)r6no_timestamps_token_idtimestamp_beginr.Zforced_decoder_idsrmax_initial_timestamp_index)rZgenerate_configrrrrBs z(WhisperTimeStampLogitsProcessor.__init__c Cstd |dd|jf<|jd|jdkr\td |ddddf<d|dd|jf<|St|jdD]}t|||jdf}t|dko|d|jk}t|dkp|d|jk}|r|rtd |||jdf<ntd ||d|j f<|jd|jkrj|j durj|j|j }td |dd|ddf<qjt j j j|dd}t|jdD]X}|||jdfjdd} ||d|jf} | | kr`td ||d|jf<q`|S)NrErXrrDrre)rGrrFrrrr0rr.r6rrrzr{r|Z logsumexprs) rr rkseqZlast_was_timestampZpenultimate_was_timestampZ last_allowedZlogprobsZtimestamp_logprobZmax_text_token_logprobrrrrs.   z(WhisperTimeStampLogitsProcessor.__call__Nrrrrrrs r)4r+rtypingrrrrrrnumpyrrutilsr Z utils.loggingr rr@rr r"r0r#r4rKrPrTr]r_rqrxrrr9rSrrrrrrrrrrrrrrrrrrrrrsR     ''0+)   )<#