a d|B@sddlmZddlmZddlmZmZmZddlm Z ddl m Z e rdddl m Z ddlmZnddlmZd d Z eGd d d ZeGd ddZeGdddZeGdddZGdddeeZeGdddZeGdddZeGdddZeGdddZeGdddZGdd d e ZGd!d"d"eZGd#d$d$eZGd%d&d&eZGd'd(d(eZe ed)d*d+Z d,S)-)field)Enum)ListNoReturnOptional) HTTPError)is_pydantic_available) validator) dataclasscCsddS)NcSs|SN)yr r s/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/huggingface_hub/inference/_text_generation.py,zvalidator..r )xr r rr +sr c@steZdZUdZdZeed<dZeed<dZ e e ed<dZ eed<e d d d Zeeed <dZe eed <dZe e ed<dZe eed<dZe e ed<dZe eed<dZe e ed<dZe eed<dZeed<dZeed<dZeed<edddZedddZed ddZedddZeddd Z edd!d"Z!edd#d$Z"edd%d&Z#dS)'TextGenerationParametersa Parameters for text generation. Args: do_sample (`bool`, *optional*): Activate logits sampling. Defaults to False. max_new_tokens (`int`, *optional*): Maximum number of generated tokens. Defaults to 20. repetition_penalty (`Optional[float]`, *optional*): The parameter for repetition penalty. A value of 1.0 means no penalty. See [this paper](https://arxiv.org/pdf/1909.05858.pdf) for more details. Defaults to None. return_full_text (`bool`, *optional*): Whether to prepend the prompt to the generated text. Defaults to False. stop (`List[str]`, *optional*): Stop generating tokens if a member of `stop_sequences` is generated. Defaults to an empty list. seed (`Optional[int]`, *optional*): Random sampling seed. Defaults to None. temperature (`Optional[float]`, *optional*): The value used to modulate the logits distribution. Defaults to None. top_k (`Optional[int]`, *optional*): The number of highest probability vocabulary tokens to keep for top-k-filtering. Defaults to None. top_p (`Optional[float]`, *optional*): If set to a value less than 1, only the smallest set of most probable tokens with probabilities that add up to `top_p` or higher are kept for generation. Defaults to None. truncate (`Optional[int]`, *optional*): Truncate input tokens to the given size. Defaults to None. typical_p (`Optional[float]`, *optional*): Typical Decoding mass. See [Typical Decoding for Natural Language Generation](https://arxiv.org/abs/2202.00666) for more information. Defaults to None. best_of (`Optional[int]`, *optional*): Generate `best_of` sequences and return the one with the highest token logprobs. Defaults to None. watermark (`bool`, *optional*): Watermarking with [A Watermark for Large Language Models](https://arxiv.org/abs/2301.10226). Defaults to False. details (`bool`, *optional*): Get generation details. Defaults to False. decoder_input_details (`bool`, *optional*): Get decoder input token logprobs and ids. Defaults to False. F do_samplemax_new_tokensNrepetition_penaltyreturn_full_textcCsgSr r r r r rrbrz!TextGenerationParameters.default_factorystopseed temperaturetop_ktop_ptruncate typical_pbest_of watermarkdetailsdecoder_input_detailscCs|dur|dkrtd|dkr4|ddur4td|d|dduB|dduB|d duB|d duB}|dkr|std |S) Nrz#`best_of` must be strictly positiverz,`seed` must not be set when `best_of` is > 1rrrrr!z+you must use sampling when `best_of` is > 1 ValueError)cls field_valuevaluesZsamplingr r r valid_best_ofzs$     z&TextGenerationParameters.valid_best_ofcCs|dur|dkrtd|S)Nrz.`repetition_penalty` must be strictly positiver'r)vr r rvalid_repetition_penaltysz1TextGenerationParameters.valid_repetition_penaltycCs|dur|dkrtd|S)Nrz`seed` must be positiver'r-r r r valid_seedsz#TextGenerationParameters.valid_seedcCs|dur|dkrtd|S)Nrz'`temperature` must be strictly positiver'r-r r r valid_tempsz#TextGenerationParameters.valid_tempcCs|dur|dkrtd|S)Nrz!`top_k` must be strictly positiver'r-r r r valid_top_ksz$TextGenerationParameters.valid_top_kcCs$|dur |dks|dkr td|S)Nr?z`top_p` must be > 0.0 and < 1.0r'r-r r r valid_top_psz$TextGenerationParameters.valid_top_pcCs|dur|dkrtd|S)Nrz$`truncate` must be strictly positiver'r-r r rvalid_truncatesz'TextGenerationParameters.valid_truncatecCs$|dur |dks|dkr td|S)Nrr3z#`typical_p` must be > 0.0 and < 1.0r'r-r r rvalid_typical_psz(TextGenerationParameters.valid_typical_p)$__name__ __module__ __qualname____doc__rbool__annotations__rintrrfloatrrrrstrrrrrr r!r"r#r$r%r r,r/r0r1r2r4r5r6r r r rr/s@ (             rc@sVeZdZUdZeed<dZeeed<dZ e ed<e dddZ e dd d Z dS) TextGenerationRequestac Request object for text generation (only for internal use). Args: inputs (`str`): The prompt for text generation. parameters (`Optional[TextGenerationParameters]`, *optional*): Generation parameters. stream (`bool`, *optional*): Whether to stream output tokens. Defaults to False. inputsN parametersFstreamcCs|s td|S)Nz`inputs` cannot be emptyr'r-r r r valid_inputsz!TextGenerationRequest.valid_inputcCs4|d}|dur0|jdur0|jdkr0|r0td|S)NrBr&z5`best_of` != 1 is not supported when `stream` == True)r"r()r)r*r+rBr r rvalid_best_of_streams z*TextGenerationRequest.valid_best_of_stream)r7r8r9r:r?r<rBrrrCr;r rDrEr r r rr@s    r@c@s2eZdZUdZeed<eed<dZee ed<dS) InputTokena1 Represents an input token. Args: id (`int`): Token ID from the model tokenizer. text (`str`): Token text. logprob (`float` or `None`): Log probability of the token. Optional since the logprob of the first token cannot be computed. idtextNlogprob) r7r8r9r:r=r<r?rIrr>r r r rrFs  rFc@s2eZdZUdZeed<eed<eed<eed<dS)Tokenat Represents a token. Args: id (`int`): Token ID from the model tokenizer. text (`str`): Token text. logprob (`float`): Log probability of the token. special (`bool`): Indicates whether the token is a special token. It can be used to ignore tokens when concatenating. rGrHrIZspecialN) r7r8r9r:r=r<r?r>r;r r r rrJs rJc@seZdZdZdZdZdS) FinishReasonlengthZ eos_tokenZ stop_sequenceN)r7r8r9ZLengthZEndOfSequenceTokenZ StopSequencer r r rrKsrKc@sneZdZUdZeed<eed<eed<dZe eed<e ddd Z e e ed <e d dd Ze eed <dS) BestOfSequencea Represents a best-of sequence generated during text generation. Args: generated_text (`str`): The generated text. finish_reason (`FinishReason`): The reason for the generation to finish, represented by a `FinishReason` value. generated_tokens (`int`): The number of generated tokens in the sequence. seed (`Optional[int]`): The sampling seed if sampling was activated. prefill (`List[InputToken]`): The decoder input tokens. Empty if `decoder_input_details` is False. Defaults to an empty list. tokens (`List[Token]`): The generated tokens. Defaults to an empty list. generated_text finish_reasongenerated_tokensNrcCsgSr r r r r rr8rzBestOfSequence.rprefillcCsgSr r r r r rr:rtokens)r7r8r9r:r?r<rKr=rrrrQrrFrRrJr r r rrMs rMc@szeZdZUdZeed<eed<dZeeed<e dddZ e e ed <e d ddZ e eed <dZee eed <dS) Detailsa Represents details of a text generation. Args: finish_reason (`FinishReason`): The reason for the generation to finish, represented by a `FinishReason` value. generated_tokens (`int`): The number of generated tokens. seed (`Optional[int]`): The sampling seed if sampling was activated. prefill (`List[InputToken]`, *optional*): The decoder input tokens. Empty if `decoder_input_details` is False. Defaults to an empty list. tokens (`List[Token]`): The generated tokens. Defaults to an empty list. best_of_sequences (`Optional[List[BestOfSequence]]`): Additional sequences when using the `best_of` parameter. rOrPNrcCsgSr r r r r rrYrzDetails.rrQcCsgSr r r r r rr[rrRbest_of_sequences)r7r8r9r:rKr<r=rrrrQrrFrRrJrTrMr r r rrS>s rSc@s*eZdZUdZeed<dZeeed<dS)TextGenerationResponsea\ Represents a response for text generation. In practice, if `details=False` is passed (default), only the generated text is returned. Args: generated_text (`str`): The generated text. details (`Optional[Details]`): Generation details. Returned only if `details=True` is sent to the server. rNNr$) r7r8r9r:r?r<r$rrSr r r rrUas  rUc@s2eZdZUdZeed<eed<dZeeed<dS) StreamDetailsao Represents details of a text generation stream. Args: finish_reason (`FinishReason`): The reason for the generation to finish, represented by a `FinishReason` value. generated_tokens (`int`): The number of generated tokens. seed (`Optional[int]`): The sampling seed if sampling was activated. rOrPNr) r7r8r9r:rKr<r=rrr r r rrVvs  rVc@s:eZdZUdZeed<dZeeed<dZ ee ed<dS)TextGenerationStreamResponsea Represents a response for text generation when `stream=True` is passed Args: token (`Token`): The generated token. generated_text (`Optional[str]`, *optional*): The complete generated text. Only available when the generation is finished. details (`Optional[StreamDetails]`, *optional*): Generation details. Only available when the generation is finished. tokenNrNr$) r7r8r9r:rJr<rNrr?r$rVr r r rrWs  rWc@seZdZdZdS)TextGenerationErrorz3Generic error raised if text-generation went wrong.Nr7r8r9r:r r r rrYsrYc@seZdZdZdS)ValidationErrorzServer-side validation error.NrZr r r rr[sr[c@s eZdZdS)GenerationErrorNr7r8r9r r r rr\sr\c@s eZdZdS)OverloadedErrorNr]r r r rr^sr^c@s eZdZdS)IncompleteGenerationErrorNr]r r r rr_sr_) http_errorreturncCsz.t|ddp|j}|d}|d}WntyD|Yn0|dur|dkr`t|||dkrrt|||dkrt|||dkrt|||dS) z Try to parse text-generation-inference error message and raise HTTPError in any case. Args: error (`HTTPError`): The HTTPError that have been raised. Zresponse_error_payloadNerror error_typeZ generationZincomplete_generationZ overloadedZ validation) getattrresponsejsonget Exceptionr\r_r^r[)r`payloadmessagercr r rraise_text_generation_errors        rkN)!Z dataclassesrenumrtypingrrrrequestsrutilsr Zpydanticr Zpydantic.dataclassesr rr@rFrJr?rKrMrSrUrVrWrYr[r\r^r_rkr r r rsF       # ""