a d @sddlmZddlmZddlZddlmZddlmZddlm Z ddl m Z ddl m Z dd lmZmZdd lmZGd d d eZdS) ) annotations)AnyN)Tensor)Module) Accelerator) CheckpointIO) Precision)Strategy TBroadcast)_DEVICEcseZdZdZd+dddddfd d Zed d d dZedd ddZdddddZdddddddZ d,dddddd d!Z dddd"d#d$Z d-d&d'd&d(d)d*Z Z S).SingleDeviceStrategyz7Strategy that handles communication on a single device.cpuNr zAccelerator | NonezCheckpointIO | NonezPrecision | None)device accelerator checkpoint_io precisioncs4tj|||dt||_d|_d|_d|_dS)N)rrrr)super__init__torchr _root_deviceZ global_rankZ local_rankZ world_size)selfrrrr __class__r/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/lightning_fabric/strategies/single_device.pyr s  zSingleDeviceStrategy.__init__z torch.device)returncCs|jSN)rrrrr root_device-sz SingleDeviceStrategy.root_deviceboolcCsdS)NTrrrrris_global_zero1sz#SingleDeviceStrategy.is_global_zerorNone)modulercCs||jdSr)tor)rr#rrrmodule_to_device5sz%SingleDeviceStrategy.module_to_devicez Any | Tensorr)tensorargskwargsrcOs|S)aReduces a tensor from several distributed processes to one aggregated tensor. As this plugin only operates with a single device, the reduction is simply the identity. Args: tensor: the tensor to sync and reduce *args: ignored **kwargs: ignored Return: the unmodified input as reduction is not needed for single process operation r)rr&r'r(rrr all_reduce8s zSingleDeviceStrategy.all_reduceFrz Any | None)r&group sync_gradsrcCs|S)z&Perform a all_gather on all processes.r)rr&r*r+rrr all_gatherFszSingleDeviceStrategy.all_gather)r'r(rcOsdSrr)rr'r(rrrbarrierJszSingleDeviceStrategy.barrierrr int)objsrcrcCs|Srr)rr/r0rrr broadcastMszSingleDeviceStrategy.broadcast)r NNN)NF)r)__name__ __module__ __qualname____doc__rpropertyrr!r%r)r,r-r1 __classcell__rrrrr s r ) __future__rtypingrrrZtorch.nnrZlightning_fabric.acceleratorsrZ)lightning_fabric.plugins.io.checkpoint_iorZ"lightning_fabric.plugins.precisionrZ$lightning_fabric.strategies.strategyr r Z lightning_fabric.utilities.typesr r rrrrs