a d@s2ddlZddlZddlZddlmZddlmZmZmZm Z m Z m Z m Z m Z ddlmZddlmZddlmZddlmZmZmZmZmZmZmZmZmZmZmZm Z ddl!m"Z"m#Z#m$Z$m%Z%m&Z&dd l'm(Z(m)Z)m*Z*dd l+m,Z,m-Z-m.Z.m/Z/dd l0m1Z1erddl2Z3dd l4m5Z5e6e7Z8Gd ddZ9dS)N)asdict) TYPE_CHECKINGAnyDictIterableListOptionalUnionoverload) HTTPError)CaseInsensitiveDict)INFERENCE_ENDPOINT) ContentTInferenceTimeoutError _b64_encode _b64_to_image_bytes_to_dict_bytes_to_image_get_recommended_model _import_numpy_is_tgi_server_open_as_binary_set_as_non_tgi _stream_text_generation_response)TextGenerationParametersTextGenerationRequestTextGenerationResponseTextGenerationStreamResponseraise_text_generation_error)ClassificationOutputConversationalOutputImageSegmentationOutput)BadRequestErrorbuild_hf_headers get_sessionhf_raise_for_status)Literal)Imagec@sBeZdZdZdGeeeeedfeeee eefee eefddddZ ddZ e dddddd eeee e feeeeeeed ed d d Ze dddddd eeee e feeeeeeedeed dd Zddddd d eeee e feeeeeeeeeeefd dd Zddeeee edddZddeeeedddZdHdddeee eee eee eefeeedddZddeeeddddZddeeee edd d!Zddeeee edd"d#ZdIddddddd$eeeeeeeeeeeeeeed%d& d'd(Zddeeeedd)d*Zddee eeee ed+d,d-Z dddeee eefeeed.d/d0Z!e dddd d1ddd dddddddd d2eed ed eeeeeeeeeeeee eeeeeeeeeeeeed3d4d5Z"e dddd d1ddd dddddddd d2eeded eeeeeeeeeeeee eeeeeeeeeeeee#d3d6d5Z"e dddd d1ddd dddddddd d2eed edeeeeeeeeeeeee eeeeeeeeeeeeeed3d7d5Z"e dddd d1ddd dddddddd d2eededeeeeeeeeeeeee eeeeeeeeeeeeee$d3d8d5Z"d d dd d1ddd dddddddd d d9eeeeeeeeeeeeeeee eeeeeeeeeeeeeeee#eeee$fd:d;d5Z"ddddddd$eeeeeeeeeeeeed%d<d=d>Z%ddeeeedd?d@Z&ddee eeee edAdBdCZ'dJeeeeedDdEdFZ(dS)KInferenceClientaZ Initialize a new Inference Client. [`InferenceClient`] aims to provide a unified experience to perform inference. The client can be used seamlessly with either the (free) Inference API or self-hosted Inference Endpoints. Args: model (`str`, `optional`): The model to run inference with. Can be a model id hosted on the Hugging Face Hub, e.g. `bigcode/starcoder` or a URL to a deployed Inference Endpoint. Defaults to None, in which case a recommended model is automatically selected for the task. token (`str`, *optional*): Hugging Face token. Will default to the locally saved token. Pass `token=False` if you don't want to send your token to the server. timeout (`float`, `optional`): The maximum number of seconds to wait for a response from the server. Loading a new model in Inference API can take up to several minutes. Defaults to None, meaning it will loop until the server is available. headers (`Dict[str, str]`, `optional`): Additional headers to send to the server. By default only the authorization and user-agent headers are sent. Values in this dictionary will override the default values. cookies (`Dict[str, str]`, `optional`): Additional cookies to send to the server. N)modeltokentimeoutheaderscookiesreturncCs:||_tt|d|_|dur*|j|||_||_dS)N)r*)r)r r#r,updater-r+)selfr)r*r+r,r-r1j/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/huggingface_hub/inference/_client.py__init__vs  zInferenceClient.__init__cCs d|jr|jndd|jdS)Nz)r)r+)r0r1r1r2__repr__szInferenceClient.__repr__.)jsondatar)taskstreamF)r6r7r)r8r9r.cCsdSNr1r0r6r7r)r8r9r1r1r2posts zInferenceClient.postTcCsdSr:r1r;r1r1r2r<s c Cs|||}|dur&|dur&tdt}|j}t|j} z$tj||| |j|j |j|d} Wn4t y} zt d|| WYd} ~ n d} ~ 00Wdn1s0Yzt | |r| n| jWSty} z| jjdkrp|dur(t||kr(t d|d|jd| td | td |durdt|jt|d }WYd} ~ q4WYd} ~ q4d} ~ 00q4dS) a  Make a POST request to the inference server. Args: json (`Union[str, Dict, List]`, *optional*): The JSON data to send in the request body. Defaults to None. data (`Union[str, Path, bytes, BinaryIO]`, *optional*): The content to send in the request body. It can be raw bytes, a pointer to an opened file, a local file path, or a URL to an online resource (image, audio file,...). If both `json` and `data` are passed, `data` will take precedence. At least `json` or `data` must be provided. Defaults to None. model (`str`, *optional*): The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. Will override the model defined at the instance level. Defaults to None. task (`str`, *optional*): The task to perform on the inference. Used only to default to a recommended model if `model` is not provided. At least `model` or `task` must be provided. Defaults to None. stream (`bool`, *optional*): Whether to iterate over streaming APIs. Returns: bytes: The raw bytes returned by the server. Raises: [`InferenceTimeoutError`]: If the model is unavailable or the request times out. `HTTPError`: If the request fails with an HTTP error status code other than HTTP 503. Nz.Ignoring `json` as `data` is passed as binary.)r6r7r,r-r+r9zInference call timed out: iz Model not loaded on the server: z/. Please retry with a higher timeout (current: z).z.Waiting for model to be loaded on the server: ) _resolve_urlwarningswarntimer+rr$r<r,r- TimeoutErrorrr% iter_linescontentr response status_codeloggerinfosleepmax) r0r6r7r)r8r9urlt0r+Zdata_as_binaryrEerrorr1r1r2r<sJ%    D   )r))audior)r.cCs|j||dd}t|S)a Perform audio classification on the provided audio content. Args: audio (Union[str, Path, bytes, BinaryIO]): The audio content to classify. It can be raw audio bytes, a local audio file, or a URL pointing to an audio file. model (`str`, *optional*): The model to use for audio classification. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended model for audio classification will be used. Returns: `List[Dict]`: The classification output containing the predicted label and its confidence. Raises: [`InferenceTimeoutError`]: If the model is unavailable or the request times out. `HTTPError`: If the request fails with an HTTP error status code other than HTTP 503. Example: ```py >>> from huggingface_hub import InferenceClient >>> client = InferenceClient() >>> client.audio_classification("audio.flac") [{'score': 0.4976358711719513, 'label': 'hap'}, {'score': 0.3677836060523987, 'label': 'neu'},...] ``` zaudio-classificationr7r)r8r<rr0rNr)rEr1r1r2audio_classifications#z$InferenceClient.audio_classificationcCs|j||dd}t|dS)a2 Perform automatic speech recognition (ASR or audio-to-text) on the given audio content. Args: audio (Union[str, Path, bytes, BinaryIO]): The content to transcribe. It can be raw audio bytes, local audio file, or a URL to an audio file. model (`str`, *optional*): The model to use for ASR. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended model for ASR will be used. Returns: str: The transcribed text. Raises: [`InferenceTimeoutError`]: If the model is unavailable or the request times out. `HTTPError`: If the request fails with an HTTP error status code other than HTTP 503. Example: ```py >>> from huggingface_hub import InferenceClient >>> client = InferenceClient() >>> client.automatic_speech_recognition("hello_world.flac") "hello world" ``` zautomatic-speech-recognitionrOtextrPrQr1r1r2automatic_speech_recognitions!z,InferenceClient.automatic_speech_recognition) parametersr))rSgenerated_responsespast_user_inputsrUr)r.cCs\dd|ii}|dur ||dd<|dur4||dd<|durD||d<|j||dd}t|S) a Generate conversational responses based on the given input text (i.e. chat with the API). Args: text (`str`): The last input from the user in the conversation. generated_responses (`List[str]`, *optional*): A list of strings corresponding to the earlier replies from the model. Defaults to None. past_user_inputs (`List[str]`, *optional*): A list of strings corresponding to the earlier replies from the user. Should be the same length as `generated_responses`. Defaults to None. parameters (`Dict[str, Any]`, *optional*): Additional parameters for the conversational task. Defaults to None. For more details about the available parameters, please refer to [this page](https://huggingface.co/docs/api-inference/detailed_parameters#conversational-task) model (`str`, *optional*): The model to use for the conversational task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended conversational model will be used. Defaults to None. Returns: `Dict`: The generated conversational output. Raises: [`InferenceTimeoutError`]: If the model is unavailable or the request times out. `HTTPError`: If the request fails with an HTTP error status code other than HTTP 503. Example: ```py >>> from huggingface_hub import InferenceClient >>> client = InferenceClient() >>> output = client.conversational("Hi, who are you?") >>> output {'generated_text': 'I am the one who knocks.', 'conversation': {'generated_responses': ['I am the one who knocks.'], 'past_user_inputs': ['Hi, who are you?']}, 'warnings': ['Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.']} >>> client.conversational( ... "Wow, that's scary!", ... generated_responses=output["conversation"]["generated_responses"], ... past_user_inputs=output["conversation"]["past_user_inputs"], ... ) ``` inputsrSNrVrWrUconversationalr6r)r8rP)r0rSrVrWrUr)payloadrEr1r1r2rY9s3   zInferenceClient.conversationalz np.ndarray)rSr)r.cCs0|jd|i|dd}t}|jt|dddS)a  Generate embeddings for a given text. Args: text (`str`): The text to embed. model (`str`, *optional*): The model to use for the conversational task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended conversational model will be used. Defaults to None. Returns: `np.ndarray`: The embedding representing the input text as a float32 numpy array. Raises: [`InferenceTimeoutError`]: If the model is unavailable or the request times out. `HTTPError`: If the request fails with an HTTP error status code other than HTTP 503. Example: ```py >>> from huggingface_hub import InferenceClient >>> client = InferenceClient() >>> client.feature_extraction("Hi, who are you?") array([[ 2.424802 , 2.93384 , 1.1750331 , ..., 1.240499, -0.13776633, -0.7889173 ], [-0.42943227, -0.6364878 , -1.693462 , ..., 0.41978157, -2.4336355 , 0.6162071 ], ..., [ 0.28552425, -0.928395 , -1.2077185 , ..., 0.76810825, -2.1069427 , 0.6236161 ]], dtype=float32) ``` rXfeature-extractionrZrfloat32)Zdtype)r<rarrayr)r0rSr)rEnpr1r1r2feature_extractionvs z"InferenceClient.feature_extraction)imager)r.cCs|j||dd}t|S)a Perform image classification on the given image using the specified model. Args: image (`Union[str, Path, bytes, BinaryIO]`): The image to classify. It can be raw bytes, an image file, or a URL to an online image. model (`str`, *optional*): The model to use for image classification. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended model for image classification will be used. Returns: `List[Dict]`: a list of dictionaries containing the predicted label and associated probability. Raises: [`InferenceTimeoutError`]: If the model is unavailable or the request times out. `HTTPError`: If the request fails with an HTTP error status code other than HTTP 503. Example: ```py >>> from huggingface_hub import InferenceClient >>> client = InferenceClient() >>> client.image_classification("https://upload.wikimedia.org/wikipedia/commons/thumb/4/43/Cute_dog.jpg/320px-Cute_dog.jpg") [{'score': 0.9779096841812134, 'label': 'Blenheim spaniel'}, ...] ``` zimage-classificationrOrPr0rar)rEr1r1r2image_classifications!z$InferenceClient.image_classificationcCsf|j||dd}t|}t|tsHtdt|dt|ddd|D]}t|d|d<qL|S) ap Perform image segmentation on the given image using the specified model. You must have `PIL` installed if you want to work with images (`pip install Pillow`). Args: image (`Union[str, Path, bytes, BinaryIO]`): The image to segment. It can be raw bytes, an image file, or a URL to an online image. model (`str`, *optional*): The model to use for image segmentation. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended model for image segmentation will be used. Returns: `List[Dict]`: A list of dictionaries containing the segmented masks and associated attributes. Raises: [`InferenceTimeoutError`]: If the model is unavailable or the request times out. `HTTPError`: If the request fails with an HTTP error status code other than HTTP 503. Example: ```py >>> from huggingface_hub import InferenceClient >>> client = InferenceClient() >>> client.image_segmentation("cat.jpg"): [{'score': 0.989008, 'label': 'LABEL_184', 'mask': }, ...] ``` zimage-segmentationrOz"Server output must be a list. Got z: Nz...mask)r<r isinstancelist ValueErrortypestrr)r0rar)rEoutputitemr1r1r2image_segmentations) &z"InferenceClient.image_segmentation)negative_promptheightwidthnum_inference_stepsguidance_scaler)r') rapromptrnrorprqrrr)r.cKs||||||d| } tdd| Dr6|} d} n2d} dt|i} | D]\} }|durN|| | <qN|j| | |dd}t|S)a; Perform image-to-image translation using a specified model. You must have `PIL` installed if you want to work with images (`pip install Pillow`). Args: image (`Union[str, Path, bytes, BinaryIO]`): The input image for translation. It can be raw bytes, an image file, or a URL to an online image. prompt (`str`, *optional*): The text prompt to guide the image generation. negative_prompt (`str`, *optional*): A negative prompt to guide the translation process. height (`int`, *optional*): The height in pixels of the generated image. width (`int`, *optional*): The width in pixels of the generated image. num_inference_steps (`int`, *optional*): The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference. guidance_scale (`float`, *optional*): Higher guidance scale encourages to generate images that are closely linked to the text `prompt`, usually at the expense of lower image quality. model (`str`, *optional*): The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None. Returns: `Image`: The translated image. Raises: [`InferenceTimeoutError`]: If the model is unavailable or the request times out. `HTTPError`: If the request fails with an HTTP error status code other than HTTP 503. Example: ```py >>> from huggingface_hub import InferenceClient >>> client = InferenceClient() >>> image = client.image_to_image("cat.jpg", prompt="turn the cat into a tiger") >>> image.save("tiger.jpg") ``` )rsrnrorprqrrcss|]}|duVqdSr:r1).0Z parameterr1r1r2 6z1InferenceClient.image_to_image..NrXzimage-to-image)r6r7r)r8)allvaluesritemsr<r)r0rarsrnrorprqrrr)kwargsrUr7r[keyvaluerEr1r1r2image_to_images&=   zInferenceClient.image_to_imagecCs |j||dd}t|ddS)a Takes an input image and return text. Models can have very different outputs depending on your use case (image captioning, optical character recognition (OCR), Pix2Struct, etc). Please have a look to the model card to learn more about a model's specificities. Args: image (`Union[str, Path, bytes, BinaryIO]`): The input image to caption. It can be raw bytes, an image file, or a URL to an online image.. model (`str`, *optional*): The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None. Returns: `str`: The generated text. Raises: [`InferenceTimeoutError`]: If the model is unavailable or the request times out. `HTTPError`: If the request fails with an HTTP error status code other than HTTP 503. Example: ```py >>> from huggingface_hub import InferenceClient >>> client = InferenceClient() >>> client.image_to_text("cat.jpg") 'a cat standing in a grassy field ' >>> client.image_to_text("https://upload.wikimedia.org/wikipedia/commons/thumb/4/43/Cute_dog.jpg/320px-Cute_dog.jpg") 'a dog laying on the grass next to a flower pot ' ``` z image-to-textrOrgenerated_textrPrbr1r1r2 image_to_textEs!zInferenceClient.image_to_text)sentenceother_sentencesr)r.cCs"|jd||di|dd}t|S)a Compute the semantic similarity between a sentence and a list of other sentences by comparing their embeddings. Args: sentence (`str`): The main sentence to compare to others. other_sentences (`List[str]`): The list of sentences to compare to. model (`str`, *optional*): The model to use for the conversational task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended conversational model will be used. Defaults to None. Returns: `List[float]`: The embedding representing the input text. Raises: [`InferenceTimeoutError`]: If the model is unavailable or the request times out. `HTTPError`: If the request fails with an HTTP error status code other than HTTP 503. Example: ```py >>> from huggingface_hub import InferenceClient >>> client = InferenceClient() >>> client.sentence_similarity( ... "Machine learning is so easy.", ... other_sentences=[ ... "Deep learning is so straightforward.", ... "This is so difficult, like rocket science.", ... "I can't believe how much I struggled with this.", ... ], ... ) [0.7785726189613342, 0.45876261591911316, 0.2906220555305481] ``` rX)Zsource_sentence sentencessentence-similarityrZrP)r0rrr)rEr1r1r2sentence_similarityis ( z#InferenceClient.sentence_similarity)rSrUr)r.cCs8d|i}|dur||d<|j||dd}t|ddS)a Generate a summary of a given text using a specified model. Args: text (`str`): The input text to summarize. parameters (`Dict[str, Any]`, *optional*): Additional parameters for summarization. Check out this [page](https://huggingface.co/docs/api-inference/detailed_parameters#summarization-task) for more details. model (`str`, *optional*): The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None. Returns: `str`: The generated summary text. Raises: [`InferenceTimeoutError`]: If the model is unavailable or the request times out. `HTTPError`: If the request fails with an HTTP error status code other than HTTP 503. Example: ```py >>> from huggingface_hub import InferenceClient >>> client = InferenceClient() >>> client.summarization("The Eiffel tower...") 'The Eiffel tower is one of the most famous landmarks in the world....' ``` rXNrU summarizationrZrZ summary_textrP)r0rSrUr)r[rEr1r1r2rs %zInferenceClient.summarization)detailsr9r) do_samplemax_new_tokensbest_ofrepetition_penaltyreturn_full_textseedstop_sequences temperaturetop_ktop_ptruncate typical_p watermark)rsrr9r)rrrrrrrrrrrrrr.cCsdSr:r1r0rsrr9r)rrrrrrrrrrrrrr1r1r2text_generationszInferenceClient.text_generationcCsdSr:r1rr1r1r2rscCsdSr:r1rr1r1r2rscCsdSr:r1rr1r1r2r s)rr9r)rrrrrrrrrrrrrdecoder_input_details)rsrr9r)rrrrrrrrrrrrrrr.cCs|r|stdd}t|||||| | | dur2| ng| | |||||d}t|||d}t|}t|sg}dD](}|d|dur|||d|=qnt|dkrtd |d t|rtd td}|rt d z|j ||d |d}Wnt yx}zpt |t r\dt|vr\t||j||||||||| | | | | |||||dWYd}~St|WYd}~n d}~00|rt||St|d}|rtfi|S|dS)a" Given a prompt, generate the following text. It is recommended to have Pydantic installed in order to get inputs validated. This is preferable as it allow early failures. API endpoint is supposed to run with the `text-generation-inference` backend (TGI). This backend is the go-to solution to run large language models at scale. However, for some smaller models (e.g. "gpt2") the default `transformers` + `api-inference` solution is still in use. Both approaches have very similar APIs, but not exactly the same. This method is compatible with both approaches but some parameters are only available for `text-generation-inference`. If some parameters are ignored, a warning message is triggered but the process continues correctly. To learn more about the TGI project, please refer to https://github.com/huggingface/text-generation-inference. Args: prompt (`str`): Input text. details (`bool`, *optional*): By default, text_generation returns a string. Pass `details=True` if you want a detailed output (tokens, probabilities, seed, finish reason, etc.). Only available for models running on with the `text-generation-inference` backend. stream (`bool`, *optional*): By default, text_generation returns the full generated text. Pass `stream=True` if you want a stream of tokens to be returned. Only available for models running on with the `text-generation-inference` backend. model (`str`, *optional*): The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None. do_sample (`bool`): Activate logits sampling max_new_tokens (`int`): Maximum number of generated tokens best_of (`int`): Generate best_of sequences and return the one if the highest token logprobs repetition_penalty (`float`): The parameter for repetition penalty. 1.0 means no penalty. See [this paper](https://arxiv.org/pdf/1909.05858.pdf) for more details. return_full_text (`bool`): Whether to prepend the prompt to the generated text seed (`int`): Random sampling seed stop_sequences (`List[str]`): Stop generating tokens if a member of `stop_sequences` is generated temperature (`float`): The value used to module the logits distribution. top_k (`int`): The number of highest probability vocabulary tokens to keep for top-k-filtering. top_p (`float`): If set to < 1, only the smallest set of most probable tokens with probabilities that add up to `top_p` or higher are kept for generation. truncate (`int`): Truncate inputs tokens to the given size typical_p (`float`): Typical Decoding mass See [Typical Decoding for Natural Language Generation](https://arxiv.org/abs/2202.00666) for more information watermark (`bool`): Watermarking with [A Watermark for Large Language Models](https://arxiv.org/abs/2301.10226) decoder_input_details (`bool`): Return the decoder input token logprobs and ids. You must set `details=True` as well for it to be taken into account. Defaults to `False`. Returns: `Union[str, TextGenerationResponse, Iterable[str], Iterable[TextGenerationStreamResponse]]`: Generated text returned from the server: - if `stream=False` and `details=False`, the generated text is returned as a `str` (default) - if `stream=True` and `details=False`, the generated text is returned token by token as a `Iterable[str]` - if `stream=False` and `details=True`, the generated text is returned with more details as a [`~huggingface_hub.inference._text_generation.TextGenerationResponse`] - if `details=True` and `stream=True`, the generated text is returned token by token as a iterable of [`~huggingface_hub.inference._text_generation.TextGenerationStreamResponse`] Raises: `ValidationError`: If input values are not valid. No HTTP call is made to the server. [`InferenceTimeoutError`]: If the model is unavailable or the request times out. `HTTPError`: If the request fails with an HTTP error status code other than HTTP 503. Example: ```py >>> from huggingface_hub import InferenceClient >>> client = InferenceClient() # Case 1: generate text >>> client.text_generation("The huggingface_hub library is ", max_new_tokens=12) '100% open source and built to be easy to use.' # Case 2: iterate over the generated tokens. Useful for large generation. >>> for token in client.text_generation("The huggingface_hub library is ", max_new_tokens=12, stream=True): ... print(token) 100 % open source and built to be easy to use . # Case 3: get more details about the generation process. >>> client.text_generation("The huggingface_hub library is ", max_new_tokens=12, details=True) TextGenerationResponse( generated_text='100% open source and built to be easy to use.', details=Details( finish_reason=, generated_tokens=12, seed=None, prefill=[ InputToken(id=487, text='The', logprob=None), InputToken(id=53789, text=' hugging', logprob=-13.171875), (...) InputToken(id=204, text=' ', logprob=-7.0390625) ], tokens=[ Token(id=1425, text='100', logprob=-1.0175781, special=False), Token(id=16, text='%', logprob=-0.0463562, special=False), (...) Token(id=25, text='.', logprob=-0.5703125, special=False) ], best_of_sequences=None ) ) # Case 4: iterate over the generated tokens with more details. # Last object is more complete, containing the full generated text and the finish reason. >>> for details in client.text_generation("The huggingface_hub library is ", max_new_tokens=12, details=True, stream=True): ... print(details) ... TextGenerationStreamResponse(token=Token(id=1425, text='100', logprob=-1.0175781, special=False), generated_text=None, details=None) TextGenerationStreamResponse(token=Token(id=16, text='%', logprob=-0.0463562, special=False), generated_text=None, details=None) TextGenerationStreamResponse(token=Token(id=1314, text=' open', logprob=-1.3359375, special=False), generated_text=None, details=None) TextGenerationStreamResponse(token=Token(id=3178, text=' source', logprob=-0.28100586, special=False), generated_text=None, details=None) TextGenerationStreamResponse(token=Token(id=273, text=' and', logprob=-0.5961914, special=False), generated_text=None, details=None) TextGenerationStreamResponse(token=Token(id=3426, text=' built', logprob=-1.9423828, special=False), generated_text=None, details=None) TextGenerationStreamResponse(token=Token(id=271, text=' to', logprob=-1.4121094, special=False), generated_text=None, details=None) TextGenerationStreamResponse(token=Token(id=314, text=' be', logprob=-1.5224609, special=False), generated_text=None, details=None) TextGenerationStreamResponse(token=Token(id=1833, text=' easy', logprob=-2.1132812, special=False), generated_text=None, details=None) TextGenerationStreamResponse(token=Token(id=271, text=' to', logprob=-0.08520508, special=False), generated_text=None, details=None) TextGenerationStreamResponse(token=Token(id=745, text=' use', logprob=-0.39453125, special=False), generated_text=None, details=None) TextGenerationStreamResponse(token=Token( id=25, text='.', logprob=-0.5703125, special=False), generated_text='100% open source and built to be easy to use.', details=StreamDetails(finish_reason=, generated_tokens=12, seed=None) ) ``` z`decoder_input_details=True` has been passed to the server but `details=False` is set meaning that the output from the server will be truncated.FN)rrrrrrrstoprrrrrrr)rXr9rU)rrrrrUrzRAPI endpoint/model for text-generation is not served via TGI. Ignoring parameters .zAPI endpoint/model for text-generation is not served via TGI. Parameter `details=True` will be ignored meaning only the generated text will be returned.zAPI endpoint/model for text-generation is not served via TGI. Cannot return output as a stream. Please pass `stream=False` as input.ztext-generation)r6r)r8r9z6The following `model_kwargs` are not used by the model)rsrr9r)rrrrrrrrrrrrrrr~)r?r@rrrrappendlen UserWarningrhr<r rfr"rjrrrrrr)r0rsrr9r)rrrrrrrrrrrrrrrUrequestr[Zignored_parametersr{Z bytes_outputer7r1r1r2r#s3     )rsrnrorprqrrr)r.cKsT||||||d|} i} | D]\} } | dur"| | | <q"|j| |dd} t| S)a Generate an image based on a given text using a specified model. You must have `PIL` installed if you want to work with images (`pip install Pillow`). Args: prompt (`str`): The prompt to generate an image from. negative_prompt (`str`, *optional*): An optional negative prompt for the image generation. height (`float`, *optional*): The height in pixels of the image to generate. width (`float`, *optional*): The width in pixels of the image to generate. num_inference_steps (`int`, *optional*): The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference. guidance_scale (`float`, *optional*): Higher guidance scale encourages to generate images that are closely linked to the text `prompt`, usually at the expense of lower image quality. model (`str`, *optional*): The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None. Returns: `Image`: The generated image. Raises: [`InferenceTimeoutError`]: If the model is unavailable or the request times out. `HTTPError`: If the request fails with an HTTP error status code other than HTTP 503. Example: ```py >>> from huggingface_hub import InferenceClient >>> client = InferenceClient() >>> image = client.text_to_image("An astronaut riding a horse on the moon.") >>> image.save("astronaut.png") >>> image = client.text_to_image( ... "An astronaut riding a horse on the moon.", ... negative_prompt="low resolution, blurry", ... model="stabilityai/stable-diffusion-2-1", ... ) >>> image.save("better_astronaut.png") ``` )rXrnrorprqrrNz text-to-imagerZ)ryr<r)r0rsrnrorprqrrr)rzrUr[r{r|rEr1r1r2 text_to_image2sB  zInferenceClient.text_to_imagecCs|jd|i|ddS)a Synthesize an audio of a voice pronouncing a given text. Args: text (`str`): The text to synthesize. model (`str`, *optional*): The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None. Returns: `bytes`: The generated audio. Raises: [`InferenceTimeoutError`]: If the model is unavailable or the request times out. `HTTPError`: If the request fails with an HTTP error status code other than HTTP 503. Example: ```py >>> from pathlib import Path >>> from huggingface_hub import InferenceClient >>> client = InferenceClient() >>> audio = client.text_to_speech("Hello world") >>> Path("hello_world.flac").write_bytes(audio) ``` rXztext-to-speechrZ)r<)r0rSr)r1r1r2text_to_speechszInferenceClient.text_to_speech)ralabelsr)r.cCs@t|dkrtd|jt|dd|id|dd}t|S)a Provide input image and text labels to predict text labels for the image. Args: image (`Union[str, Path, bytes, BinaryIO]`): The input image to caption. It can be raw bytes, an image file, or a URL to an online image. labels (`List[str]`): List of string possible labels. The `len(labels)` must be greater than 1. model (`str`, *optional*): The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None. Returns: `List[Dict]`: List of classification outputs containing the predicted labels and their confidence. Raises: [`InferenceTimeoutError`]: If the model is unavailable or the request times out. `HTTPError`: If the request fails with an HTTP error status code other than HTTP 503. Example: ```py >>> from huggingface_hub import InferenceClient >>> client = InferenceClient() >>> client.zero_shot_image_classification( ... "https://upload.wikimedia.org/wikipedia/commons/thumb/4/43/Cute_dog.jpg/320px-Cute_dog.jpg", ... labels=["dog", "cat", "horse"], ... ) [{"label": "dog", "score": 0.956}, ...] ``` zQYou must specify at least 2 classes to compare. Please specify more than 1 class.Zcandidate_labels,)rarUzzero-shot-image-classificationrZ)rrhr<rjoinr)r0rarr)rEr1r1r2zero_shot_image_classifications& z.InferenceClient.zero_shot_image_classification)r)r8r.cCst|p|j}|dur*|ds&|dr*|S|durJ|durBtdt|}|dvrftd|d|Std|S)Nzhttp://zhttps://zYou must specify at least a model (repo_id or URL) or a task, either when instantiating `InferenceClient` or when making a request.)r\rz /pipeline//z/models/)r) startswithrhrr )r0r)r8r1r1r2r>s  zInferenceClient._resolve_url)NNNNN)NN)N)NN))__name__ __module__ __qualname____doc__rrjr boolfloatrr3r5r rrr&bytesr<rrrRrTrr rYr`rcr!rmintr}rrrrrrrrrr>r1r1r1r2r(]s    S * '   =( ( 6 T% 3 +       Q! 0r():loggingrAr?Z dataclassesrtypingrrrrrrr r requestsr Zrequests.structuresr Zhuggingface_hub.constantsr Z!huggingface_hub.inference._commonrrrrrrrrrrrrZ*huggingface_hub.inference._text_generationrrrrrZ huggingface_hub.inference._typesrr r!Zhuggingface_hub.utilsr"r#r$r%Zhuggingface_hub.utils._typingr&numpyr_ZPILr' getLoggerrrGr(r1r1r1r2$s" (   8