a  d,v@sddlZddlZddlZddlZddlmZddlmZddl Z ddl m Z m Z m Z ddlmZddlmZmZmZdd lmZmZmZmZmZdd lmZmZdd lmZee Z!erddl"Z"erdd l#m$Z$m%Z%dd lm&Z&ne'Z$da(e)e*e+e,e-dZ.eGdddZ/ia0gdZ1d,ddZ2ddZ3d-ddZ4d.ddZ5ddZ6ddZ7Gd d!d!Z8Gd"d#d#e8Z9Gd$d%d%e8Z:Gd&d'd'e8Z;Gd(d)d)e8Zz"Agent.__init__..cSsi|]\}}|tvr||qSr%)r>rfr.rBr%r%r&rhrir rQcSs g|]\}}d|d|qSz- : r%)rfnrgr%r%r& riz"Agent.__init__..zSThe following tools have been replaced by the ones provided in `additional_tools`: .rz has been replaced by z# as provided in `additional_tools`.)rC __class__r!rchat_prompt_templaterun_prompt_templater>rG_toolboxrlogrHlisttupledictr.r<updaterXrSloggerwarnkeysprepare_for_new_chat)selfrqrradditional_toolsZ agent_name replacementsnamesr.r%r%r&__init__s,       zAgent.__init__returncCs|jS)z-Get all tool currently available to the agent)rsr}r%r%r&rJsz Agent.toolboxFcCsnddd|jD}|rP|jdur8|jd|}n|j}|td|7}n|jd|}|d|}|S)NrQcSs"g|]\}}d|d|jqSrk)rrjr%r%r&rnriz'Agent.format_prompt..z <>z<>z <>)rSrJr< chat_historyrqreplacerrr)r}r chat_moderpromptr%r%r& format_prompts  zAgent.format_promptcCs ||_dS)z Set the function use to stream results (which is `print` by default). Args: streamer (`callable`): The function to call when streaming results from the LLM. N)rt)r}Zstreamerr%r%r& set_streamszAgent.set_stream) return_coderEc Ks|j|dd}|j|ddgd}||d|_t|\}}|d||dur|d ||s|d t||j||jd |_|j |t ||j|j ddSt ||j|d } | d|SdS) a Sends a new request to the agent in a chat. Will use the previous ones in its history. Args: task (`str`): The task to perform return_code (`bool`, *optional*, defaults to `False`): Whether to just return code and not evaluate it. remote (`bool`, *optional*, defaults to `False`): Whether or not to use remote tools (inference endpoints) instead of local ones. kwargs (additional keyword arguments, *optional*): Any keyword argument to send to the agent when evaluating the code. Example: ```py from transformers import HfAgent agent = HfAgent("https://api-inference.huggingface.co/models/bigcode/starcoder") agent.chat("Draw me a picture of rivers and lakes") agent.chat("Transform the picture so that there is a rock in there") ``` T)rzHuman:z=====stoprQ==Explanation from the agent== N" ==Code generated by the agent== ==Result==rErKrD) r generate_oner[rr`rtrMrJrK chat_staterxrrV r}rrrEkwargsrr\r_rIZ tool_coder%r%r&rcs   z Agent.chatcCsd|_i|_d|_dS)zG Clears the history of prior calls to [`~Agent.chat`]. N)rrrKrr%r%r&r|0szAgent.prepare_for_new_chatc Ks||}|j|dgd}t|\}}|d||d||s~|dt||j||jd|_t||j|dSt ||j|d} | d |Sd S) a  Sends a request to the agent. Args: task (`str`): The task to perform return_code (`bool`, *optional*, defaults to `False`): Whether to just return code and not evaluate it. remote (`bool`, *optional*, defaults to `False`): Whether or not to use remote tools (inference endpoints) instead of local ones. kwargs (additional keyword arguments, *optional*): Any keyword argument to send to the agent when evaluating the code. Example: ```py from transformers import HfAgent agent = HfAgent("https://api-inference.huggingface.co/models/bigcode/starcoder") agent.run("Draw me a picture of rivers and lakes") ``` zTask:rrrrr)staterDrQN) rrrartrMrJrKrrGrVrr%r%r&re8s   z Agent.runcCstdSN)NotImplementedErrorr}rrr%r%r&r]szAgent.generate_onecsfdd|DS)Ncsg|]}|qSr%)rrfrr}rr%r&rncriz'Agent.generate_many..r%r}promptsrr%rr& generate_manyaszAgent.generate_many)NNN)F)r!r"r#__doc__rpropertyrrrrJrrrcr|rerrr%r%r%r&rbs   *%rbcsBeZdZdZdfdd ZddZdd Zd d Zd d ZZ S) OpenAiAgentu! Agent that uses the openai API to generate code. The openAI models are used in generation mode, so even for the `chat()` API, it's better to use models like `"text-davinci-003"` over the chat-GPT variant. Proper support for chat-GPT models will come in a next version. Args: model (`str`, *optional*, defaults to `"text-davinci-003"`): The name of the OpenAI model to use. api_key (`str`, *optional*): The API key to use. If unset, will look for the environment variable `"OPENAI_API_KEY"`. chat_prompt_template (`str`, *optional*): Pass along your own prompt if you want to override the default template for the `chat` method. Can be the actual prompt template or a repo ID (on the Hugging Face Hub). The prompt should be in a file named `chat_prompt_template.txt` in this repo in this case. run_prompt_template (`str`, *optional*): Pass along your own prompt if you want to override the default template for the `run` method. Can be the actual prompt template or a repo ID (on the Hugging Face Hub). The prompt should be in a file named `run_prompt_template.txt` in this repo in this case. additional_tools ([`Tool`], list of tools or dictionary with tool values, *optional*): Any additional tools to include on top of the default ones. If you pass along a tool with the same name as one of the default tools, that default tool will be overridden. Example: ```py from transformers import OpenAiAgent agent = OpenAiAgent(model="text-davinci-003", api_key=xxx) agent.run("Is the following `text` (in Spanish) positive or negative?", text="¡Este es un API muy agradable!") ``` text-davinci-003NcsXtstd|dur$tjdd}|dur6tdn|t_||_t j |||ddS)N.)r_completion_generaterr%rr&rs zOpenAiAgent.generate_manycCs,d|jvr|||S||g|dSdS)Nrr)rrrrr%r%r&rs  zOpenAiAgent.generate_onecCs2tjj|jd|dgd|d}|ddddS)Nuserrolecontentr)rmessages temperaturerchoicesmessager)rChatCompletioncreaterr}rrr\r%r%r&rs zOpenAiAgent._chat_generatecCs*tjj|j|d|dd}dd|dDS)Nr)rrrr max_tokenscSsg|] }|dqStextr%rfZanswerr%r%r&rnriz4OpenAiAgent._completion_generate..r)r Completionrrr}rrr\r%r%r&rsz OpenAiAgent._completion_generate)rNNNN r!r"r#rrrrrr __classcell__r%r%rr&rfs' rcsBeZdZdZdfdd ZddZdd Zd d Zd d ZZ S)AzureOpenAiAgentu Agent that uses Azure OpenAI to generate code. See the [official documentation](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/) to learn how to deploy an openAI model on Azure The openAI models are used in generation mode, so even for the `chat()` API, it's better to use models like `"text-davinci-003"` over the chat-GPT variant. Proper support for chat-GPT models will come in a next version. Args: deployment_id (`str`): The name of the deployed Azure openAI model to use. api_key (`str`, *optional*): The API key to use. If unset, will look for the environment variable `"AZURE_OPENAI_API_KEY"`. resource_name (`str`, *optional*): The name of your Azure OpenAI Resource. If unset, will look for the environment variable `"AZURE_OPENAI_RESOURCE_NAME"`. api_version (`str`, *optional*, default to `"2022-12-01"`): The API version to use for this agent. is_chat_mode (`bool`, *optional*): Whether you are using a completion model or a chat model (see note above, chat models won't be as efficient). Will default to `gpt` being in the `deployment_id` or not. chat_prompt_template (`str`, *optional*): Pass along your own prompt if you want to override the default template for the `chat` method. Can be the actual prompt template or a repo ID (on the Hugging Face Hub). The prompt should be in a file named `chat_prompt_template.txt` in this repo in this case. run_prompt_template (`str`, *optional*): Pass along your own prompt if you want to override the default template for the `run` method. Can be the actual prompt template or a repo ID (on the Hugging Face Hub). The prompt should be in a file named `run_prompt_template.txt` in this repo in this case. additional_tools ([`Tool`], list of tools or dictionary with tool values, *optional*): Any additional tools to include on top of the default ones. If you pass along a tool with the same name as one of the default tools, that default tool will be overridden. Example: ```py from transformers import AzureOpenAiAgent agent = AzureAiAgent(deployment_id="Davinci-003", api_key=xxx, resource_name=yyy) agent.run("Is the following `text` (in Spanish) positive or negative?", text="¡Este es un API muy agradable!") ``` N 2022-12-01c ststd||_dt_|dur0tjdd}|durBtdn|t_ |dur^tjdd}|durptdnd|dt_ |t_ |durd | v}||_ tj|||d dS) NrZazureZAZURE_OPENAI_API_KEYzYou need an Azure openAI key to use `AzureOpenAIAgent`. If you have one, set it in your env with `os.environ['AZURE_OPENAI_API_KEY'] = xxx.ZAZURE_OPENAI_RESOURCE_NAMEzYou need a resource_name to use `AzureOpenAIAgent`. If you have one, set it in your env with `os.environ['AZURE_OPENAI_RESOURCE_NAME'] = xxx.zhttps://z.openai.azure.comrr)r r deployment_idrZapi_typerrrr@rZapi_base api_versionlower is_chat_modelrr) r}rr resource_namerrrqrrr~rr%r&rs6  zAzureOpenAiAgent.__init__cs*jrfdd|DS|SdS)Ncsg|]}|qSr%rrrr%r&rn(riz2AzureOpenAiAgent.generate_many..)rrrr%rr&r&szAzureOpenAiAgent.generate_manycCs(|jr|||S||g|dSdS)Nr)rrrrr%r%r&r,s zAzureOpenAiAgent.generate_onecCs2tjj|jd|dgd|d}|ddddS)Nrrr)enginerrrrrr)rrrrrr%r%r&r2s zAzureOpenAiAgent._chat_generatecCs*tjj|j|d|dd}dd|dDS)Nrr)rrrrrcSsg|] }|dqSrr%rr%r%r&rnCriz9AzureOpenAiAgent._completion_generate..r)rrrrrr%r%r&r;sz%AzureOpenAiAgent._completion_generate)NNrNNNNrr%r%rr&rs2. rcs*eZdZdZdfdd ZddZZS)HfAgentuc Agent that uses an inference endpoint to generate code. Args: url_endpoint (`str`): The name of the url endpoint to use. token (`str`, *optional*): The token to use as HTTP bearer authorization for remote files. If unset, will use the token generated when running `huggingface-cli login` (stored in `~/.huggingface`). chat_prompt_template (`str`, *optional*): Pass along your own prompt if you want to override the default template for the `chat` method. Can be the actual prompt template or a repo ID (on the Hugging Face Hub). The prompt should be in a file named `chat_prompt_template.txt` in this repo in this case. run_prompt_template (`str`, *optional*): Pass along your own prompt if you want to override the default template for the `run` method. Can be the actual prompt template or a repo ID (on the Hugging Face Hub). The prompt should be in a file named `run_prompt_template.txt` in this repo in this case. additional_tools ([`Tool`], list of tools or dictionary with tool values, *optional*): Any additional tools to include on top of the default ones. If you pass along a tool with the same name as one of the default tools, that default tool will be overridden. Example: ```py from transformers import HfAgent agent = HfAgent("https://api-inference.huggingface.co/models/bigcode/starcoder") agent.run("Is the following `text` (in Spanish) positive or negative?", text="¡Este es un API muy agradable!") ``` Ncs`||_|dur"dt|_n(|ds6|dr>||_n d||_tj|||ddS)NzBearer ZBearerZBasicr) url_endpointr get_tokentokenrZrr)r}rrrqrrr~rr%r&rfs zHfAgent.__init__cCsd|ji}|dd|dd}tj|j||d}|jdkrVtdtd | |S|jdkrzt d |jd | | d d }|D]$}| |r|dt | Sq|S)N AuthorizationrF)max_new_tokensZreturn_full_textr)inputs parameters)r1headersiz=Getting rate-limited, waiting a tiny bit before trying again.r zError rlrZgenerated_text)rrequestspostr status_coderyinfotimesleepZ _generate_oner@r1endswithrX)r}rrrrresponser\stop_seqr%r%r&rvs         zHfAgent.generate_one)NNNN)r!r"r#rrrrr%r%rr&rFs rcsBeZdZdZd fdd ZeddZeddZd d Z Z S) LocalAgenta Agent that uses a local model and tokenizer to generate code. Args: model ([`PreTrainedModel`]): The model to use for the agent. tokenizer ([`PreTrainedTokenizer`]): The tokenizer to use for the agent. chat_prompt_template (`str`, *optional*): Pass along your own prompt if you want to override the default template for the `chat` method. Can be the actual prompt template or a repo ID (on the Hugging Face Hub). The prompt should be in a file named `chat_prompt_template.txt` in this repo in this case. run_prompt_template (`str`, *optional*): Pass along your own prompt if you want to override the default template for the `run` method. Can be the actual prompt template or a repo ID (on the Hugging Face Hub). The prompt should be in a file named `run_prompt_template.txt` in this repo in this case. additional_tools ([`Tool`], list of tools or dictionary with tool values, *optional*): Any additional tools to include on top of the default ones. If you pass along a tool with the same name as one of the default tools, that default tool will be overridden. Example: ```py import torch from transformers import AutoModelForCausalLM, AutoTokenizer, LocalAgent checkpoint = "bigcode/starcoder" model = AutoModelForCausalLM.from_pretrained(checkpoint, device_map="auto", torch_dtype=torch.bfloat16) tokenizer = AutoTokenizer.from_pretrained(checkpoint) agent = LocalAgent(model, tokenizer) agent.run("Draw me a picture of rivers and lakes.") ``` Ncs"||_||_tj|||ddS)Nr)r tokenizerrr)r}rrrqrrr~rr%r&rszLocalAgent.__init__cKs.tj|fi|}tj|fi|}|||S)a Convenience method to build a `LocalAgent` from a pretrained checkpoint. Args: pretrained_model_name_or_path (`str` or `os.PathLike`): The name of a repo on the Hub or a local path to a folder containing both model and tokenizer. kwargs: Keyword arguments passed along to [`~PreTrainedModel.from_pretrained`]. Example: ```py import torch from transformers import LocalAgent agent = LocalAgent.from_pretrained("bigcode/starcoder", device_map="auto", torch_dtype=torch.bfloat16) agent.run("Draw me a picture of rivers and lakes.") ``` )rfrom_pretrainedr)clsZpretrained_model_name_or_pathrrrr%r%r&rszLocalAgent.from_pretrainedcCs<t|jdr t|jjdS|jD] }|jSdS)N hf_device_mapr)hasattrrrurvaluesrdrZdevice)r}paramr%r%r& _model_devices zLocalAgent._model_devicec Cs|j|dd|j}|djd}tt||jg}|jj|dd|d}|j|d |d}|D] }| |rl|dt | }ql|S)Npt)Zreturn_tensors input_idsr r)rstopping_criteriar) rtorshaperStopSequenceCriteriargeneratedecodetolistrrX) r}rrZencoded_inputsZsrc_lenroutputsr\rr%r%r&rs  zLocalAgent.generate_one)NNN) r!r"r#rr classmethodrrrrrr%r%rr&rs#   rc@s&eZdZdZddZedddZdS)ra6 This class can be used to stop generation whenever a sequence of tokens is encountered. Args: stop_sequences (`str` or `List[str]`): The sequence (or list of sequences) on which to stop execution. tokenizer: The tokenizer used to decode the model outputs. cCs t|tr|g}||_||_dSr)rHrstop_sequencesr)r}rrr%r%r&rs zStopSequenceCriteria.__init__rc s,|j|dtfdd|jDS)Nrc3s|]}|VqdSr)r)rfZ stop_sequenceZdecoded_outputr%r& riz0StopSequenceCriteria.__call__..)rrranyr)r}rZscoresrr%rr&__call__szStopSequenceCriteria.__call__N)r!r"r#rrrrr%r%r%r&rs r)r')FN)F)>Zimportlib.utilr:r1rrZ dataclassesrtypingrrZhuggingface_hubrrrZ models.autorutilsr r r baser rrrrrrrZpython_interpreterrZ get_loggerr!ryrZ generationrrrobjectr9rrrrrrFrr>r?r8rCrMrVr`rarbrrrrrr%r%r%r&sV         )b~G]