a dM@sddlZddlZddlmZddlZddlmZddlmZddl m Z ddl m Z m Z dd lmZdd lmZe rdd lmZe d d rddlmmZddZd!edddZddZddZeddZddZddZd"e eddd Z!dS)#N)contextmanager)write_basic_config) PartialState)DistributedType)is_deepspeed_availableis_tpu_available) convert_model)is_torch_version)DeepSpeedEngineF)Z check_devicecCs(tddsttdsdSt|tjjjS)zD Check whether the module was compiled with torch.compile()  Make sure all processes will reach this instruction otherwise one of your processes will hang forever. N)rwait_for_everyonerrrrr!Zs r!cCs8tjtjkrt||ntjdkr4t||dS)z Save the data to disk. Use in place of `torch.save()`. Args: obj: The data to save f: The file (or file-like object) to use to save the data rN)rZdistributed_typerZTPUxmsaveZlocal_process_indexr)objfrrrr#gs r#cksR|D]\}}t|tj|<qdV|D]}|tjvr.tj|=q.dS)a A context manager that will add each keyword argument passed to `os.environ` and remove them when exiting. Will convert the values in `kwargs` to strings and upper-case all the keys. Example: ```python >>> import os >>> from accelerate.utils import patch_environment >>> with patch_environment(FOO="bar"): ... print(os.environ["FOO"]) # prints "bar" >>> print(os.environ["FOO"]) # raises KeyError ``` N)itemsstrosenvironupper)kwargskeyvaluerrrpatch_environmentus r.cCsHt|ds t|ds t|d|}t|dr0|jSt|dr@|jSt|S)z( Gets a pretty name from `obj`. __qualname____name__ __class__)rrr/r0r')r$rrrget_pretty_names   r2cCs@|D]2\}}t|tr2||i}t||q|||<q|S)z Recursively merges two dictionaries. Args: source (`dict`): The dictionary to merge into `destination`. destination (`dict`): The dictionary to merge `source` into. )r&rdict setdefault merge_dicts)sourceZ destinationr,r-noderrrr5s     r5)portreturncCsR|dur d}ttjtj"}|d|fdkWdS1sD0YdS)z Checks if a port is in use on `localhost`. Useful for checking if multiple `accelerate launch` commands have been run and need to see if the port is already in use. Nis,         -