a d5@sjddlZddlZddlmZddlmZddlmZmZm Z m Z m Z m Z ddl Z ddlmZddlmZddlmZmZGdd d eZd&ee ed d d Ze edddZee dddZededddZedddZe e ee efdddZedddZe e edddZ e ee ee eddd Z!edd!d"Z"e j#dd#d$d%Z$dS)'N)contextmanager) lru_cache)castDict GeneratorListOptionalUnion)rank_zero_info) Accelerator)_TORCH_GREATER_EQUAL_1_12_TORCH_GREATER_EQUAL_2_0c@seZdZdZejddddZddddZee e e e e fe e e d d d Zee e e ejd d d Zee dddZeedddZeeddddZdS)CUDAAcceleratorz$Accelerator for NVIDIA CUDA devices.NdevicereturncCs2|jdkrtd|dt|tj|dS)zm Raises: ValueError: If the selected device is not of type CUDA. cudazDevice should be CUDA, got z instead.N)type ValueError_check_cuda_matmul_precisiontorchrZ set_device)selfrrk/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/lightning_fabric/accelerators/cuda.py setup_devices zCUDAAccelerator.setup_devicercCstjdS)N)rrZ empty_cache)rrrrteardown)szCUDAAccelerator.teardown)devicesrcCsddlm}||ddS)z!Accelerator device parsing logic.r)_parse_gpu_idsT)Z include_cuda)Z(lightning_fabric.utilities.device_parserr)rrrrr parse_devices-s zCUDAAccelerator.parse_devicescCsdd|DS)z*Gets parallel devices for the Accelerator.cSsg|]}td|qS)r)rr).0irrr 7z8CUDAAccelerator.get_parallel_devices..r)rrrrget_parallel_devices4sz$CUDAAccelerator.get_parallel_devicescCstS)z!Get the devices when set to auto.num_cuda_devicesrrrrauto_device_count9sz!CUDAAccelerator.auto_device_countcCs tdkS)Nrr%rrrr is_available>szCUDAAccelerator.is_available)accelerator_registryrcCs|jd||jjddS)Nr) description)register __class____name__)clsr)rrrregister_acceleratorsBs z%CUDAAccelerator.register_accelerators)r- __module__ __qualname____doc__rrrr staticmethodr intstrrrrr$r'boolr( classmethodrr/rrrrrs (r) num_devicesrc Cst}|std|d|t|kr@td|dt|dg}g}|D]Z}ztjdtd|dWn ty||YqLYn0||t||krLqqL|dkrt||krtd|d t|d |d |S) aReturns a list of all available and usable CUDA GPU devices. A GPU is considered usable if we can successfully move a tensor to the device, and this is what this function tests for each GPU on the system until the target number of usable devices is found. A subset of GPUs on the system might be used by other processes, and if the GPU is configured to operate in 'exclusive' mode (configurable by the admin), then only one process is allowed to occupy it. Args: num_devices: The number of devices you want to request. By default, this function will return as many as there are usable CUDA GPU devices available. Warning: If multiple processes call this function at the same time, there can be race conditions in the case where both processes determine that the device is unoccupied, leading into one of them crashing later on. zYou requested to find z? devices but there are no visible CUDA devices on this machine.z# devices but this machine only has z GPUs.rr)rr8z devices but only z& are currently available. The devices zA are occupied by other processes and can't be used at the moment.)_get_all_visible_cuda_devicesrlenrZtensorr RuntimeErrorappend)r9visible_devicesZavailable_devicesZunavailable_devicesZgpu_idxrrrfind_usable_cuda_devicesKs6       r?rcCstttS)aReturns a list of all visible CUDA GPU devices. Devices masked by the environment variabale ``CUDA_VISIBLE_DEVICES`` won't be returned here. For example, assume you have 8 physical GPUs. If ``CUDA_VISIBLE_DEVICES="1,3,6"``, then this function will return the list ``[0, 1, 2]`` because these are the three visible GPUs after applying the mask ``CUDA_VISIBLE_DEVICES``. )listranger&rrrrr:}sr:ccsTttjdrJtdkrJtsJtjj}ttj_zdVW|tj_qP|tj_0ndVdS)zpContext manager that safely patches :func:`torch.cuda.is_available` with its NVML-based version if possible.Z_cuda_getDeviceCountrN)hasattrrZ_C_device_count_nvmlr rr(is_cuda_available)Z orig_checkrrr_patch_cuda_is_availablesrEcCs*trtjSt}|dkr&tjS|S)zReturns the number of available CUDA devices. Unlike :func:`torch.cuda.device_count`, this function does its best not to create a CUDA context for fork support, if the platform allows it. r)r rrZ device_countrC)Z nvml_countrrrr&s r&cCstrtjStdkS)zReturns a bool indicating if CUDA is currently available. Unlike :func:`torch.cuda.is_available`, this function does its best not to create a CUDA context for fork support, if the platform allows it. r)r rrr(r&rrrrrDsrDcCstd}|durttdSttddd}ttttddd }|d rX||d S|d rl||d Sg}|d D]@}|| }||vrt ttgS|d krq| |qz|S)z0Parse CUDA_VISIBLE_DEVICES environment variable.ZCUDA_VISIBLE_DEVICESN@)srcSsj|sdSt|D]<\}}|s4|dkr0|dvs4qN|dt|kr|d7}q|dkrft|d|SdS)z:Return -1 or positive integer sequence string starts with,r8rz+-rFN) enumerateisdigitr;r4)rHidxcrrr_strtouls z(_parse_visible_devices.._strtoul)lstprefixrcSsJg}|dD]6}||vr,tttgS||s:qF||q|S)N,)splitrrr5 startswithr=)rNrOrcselemrrrparse_list_with_prefixs  z6_parse_visible_devices..parse_list_with_prefixzGPU-MIG-rPr) osgetenvr@rAr5r4rrRrQstriprr=)varrMrUrcrTxrrr_parse_visible_devicess$         r]cCsnddlm}m}m}|d}|}|dkr:tddS|d}|||}|dkrftddS~|jS)zgReturn number of devices as reported by NVML or negative value if NVML discovery/initialization failed.r)byrefc_intCDLLlibnvidia-ml.so.1Can't initialize NVMLr8Can't get nvml device count) ctypesr^r_r`nvmlInitwarningswarnnvmlDeviceGetCount_v2value)r^r_r`nvml_hr[ dev_countrrr_raw_device_count_nvmls  rlc Csddlm}m}m}m}m}|d}|}|dkrBtddS|d}| ||}|dkrntddSg}t |j D]|} |} | | || }|dkrtddSd } || } | | | | }|dkrtd dS|| jd d q|~|S) z^Return list of device UUID as reported by NVML or None if NVM discovery/initialization failed.r)r^r_c_void_pr`create_string_bufferrarbNr8rczCan't get device handle`zCan't get device UUIDascii)rdr^r_rmr`rnrerfrgrhrAriZnvmlDeviceGetHandleByIndex_v2ZnvmlDeviceGetUUIDr=rawdecoderY) r^r_rmr`rnrjr[rkuuidsrKZdev_idZbuf_lenbufrrr_raw_device_uuid_nvmls6    rv) candidatesrtrcCsbttttddd}g}|D]>}|||}|dkr8q^||vrRtttgS||q|S)zuGiven the set of partial uuids and list of known uuids builds a set of ordinals excluding ambiguous partials IDs.) candidatertrcSs8d}t|D]&\}}||s q |dkr.dS|}q |S)Nr8)rIrR)rxrt best_matchrKuuidrrruuid_to_orinals z3_transform_uuid_to_ordinals..uuid_to_orinalr)r5rr4rr=)rwrtr{r[rxrKrrr_transform_uuid_to_ordinalss   r|cCst}|sdSzt|dtur^|ddr4WdSt}|durHWdStttt||}n>t}|dkrr|WSt |D] \}}tt ||krz|WSqzWn&t yYdSt yYdS0t |S)zReturn number of devices as reported by NVML taking CUDA_VISIBLE_DEVICES into account. Negative value is returned if NVML discovery or initialization has failed. rrVr8N)r]rr5rRrvr|rrrlrIr4OSErrorAttributeErrorr;)r>rtZraw_cntrKvalrrrrC6s,  rCrcCsPtsdStj|\}}|dk}|s(dStdkrLtdtj|ddS)NZhighestzYou are using a CUDA device (a5) that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision)r rrZget_device_capabilityZget_float32_matmul_precisionr Zget_device_name)rmajor_Zampere_or_laterrrrrVs r)r8)%rWrf contextlibr functoolsrtypingrrrrrr rZ"lightning_utilities.core.rank_zeror Z)lightning_fabric.accelerators.acceleratorr Z"lightning_fabric.utilities.importsr r rr4r?r:rEr&r6rDr5r]rlrvr|rCrrrrrrs,     02  1!