a d9 @sddlmZddlmZmZmZmZddlmZddl m Z ddl m Z ddl mZmZddl mZddl mZdd l mZdd l mZdd l mZdd l mZdd lmZddlmZ ddlm!Z"ddlm#Z$ddlm%Z&ddlm'Z(ddlm)Z)ddl*m+Z,ddl-m.Z/ddl-m0Z1ddl-m2Z3ddl-m4Z5ddl-m6Z7ddl-m8Z9dd l:mZ;ddl:mZ?ddl:m@ZAddl:mZBddl:m!ZCddl:m#ZDddl:m%ZEddl:m'ZFddl:mGZHddlImJZJmKZKee5eefZLeeLeMfZNGd d!d!eeZOeeeMe,feeeePeMePfeeeePeMePfeeeePeMePfeeeeePeMePfeeeMe,ffd"d#d$ZQeFe(d%d&d'ZRee5ed(d)d*ZSd+S),)ABC)ListOptionalTupleUnion)Fabric) _PLUGIN_INPUT)_PRECISION_INPUT) CheckpointIOClusterEnvironment)DeepSpeedPrecision)DoublePrecision)MixedPrecision) Precision)TPUBf16Precision) TPUPrecision)DataParallelStrategy) DDPStrategy)DeepSpeedStrategy)SingleDeviceStrategy)SingleTPUStrategy)Strategy) XLAStrategy) Accelerator)DeepSpeedPrecisionPlugin)DoublePrecisionPlugin)MixedPrecisionPlugin)PrecisionPlugin)TPUBf16PrecisionPlugin)TPUPrecisionPlugin)DDPShardedStrategy)DDPSpawnShardedStrategy)DDPSpawnStrategy)TPUSpawnStrategyrank_zero_deprecationrank_zero_warnc seZdZdZdeeeefeeeefeee e ee fe e eee e e feee e ee feee e ee fdd fdd Z ZS) LightningLiteaVLite accelerates your PyTorch training or inference code with minimal changes required. .. deprecated:: v1.9.0 The `pytorch_lightning.lite.LightningLite` class was deprecated in v1.9.0 and will be renamed to `lightning.fabric.Fabric` in v2.0.0. It is no longer part of the pure `pytorch_lightning` package, and now lives in the main `lightning` package. - Automatic placement of models and data onto the device. - Automatic support for mixed and double precision (smaller memory footprint). - Seamless switching between hardware (CPU, GPU, TPU) and distributed training strategies (data-parallel training, sharded training, etc.). - Automated spawning of processes, no launch utilities required. - Multi-node support. Args: accelerator: The hardware to run on. Possible choices are: ``"cpu"``, ``"cuda"``, ``"mps"``, ``"gpu"``, ``"tpu"``, ``"auto"``. strategy: Strategy for how to run across multiple devices. Possible choices are: ``"dp"``, ``"ddp"``, ``"ddp_spawn"``, ``"deepspeed"``, ``"fsdp"``. devices: Number of devices to train on (``int``), which GPUs to train on (``list`` or ``str``), or ``"auto"``. The value applies per node. num_nodes: Number of GPU nodes for distributed training. precision: Double precision (``64``), full precision (``32``), half precision (``16``), or bfloat16 precision (``"bf16"``). plugins: One or several custom plugins gpus: Provides the same function as the ``devices`` argument but implies ``accelerator="gpu"``. .. deprecated:: v1.8.0 ``gpus`` has been deprecated in v1.8.0 and will be removed in v2.0.0. Please use ``accelerator='gpu'`` and ``devices=x`` instead. tpu_cores: Provides the same function as the ``devices`` argument but implies ``accelerator="tpu"``. .. deprecated:: v1.8.0 ``tpu_cores`` has been deprecated in v1.8.0 and will be removed in v2.0.0. Please use ``accelerator='tpu'`` and ``devices=x`` instead. N ) acceleratorstrategydevices num_nodes precisionpluginsgpus tpu_coresreturnc std|dus|dur,t||||d\}}t|tr@t|} nt|trZdd|D} n|} t|ttfvsv|dvrd} t|tus|dkrd} t d | d t j |t|t rt |n||||| d dS) NzThe `pytorch_lightning.lite.LightningLite` class was deprecated in v1.9.0 and will be renamed to `lightning_fabric.Fabric` in v2.0.0.)r*r,r0r1cSs"g|]}t|trt|n|qS) isinstancePLPrecisionPlugin_to_lite_precision).0pluginr3r3d/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/pytorch_lightning/lite/lite.py ~sz*LightningLite.__init__..)Z ddp_shardedddp_sharded_spawnr;z, start_method='spawn'zLightningLite's sharded implementation using FairScale has been removed in favor of PyTorch's FSDP. You can try `Fabric(strategy=FSDPStrategy(sharding_strategy=ShardingStrategy.SHARD_GRAD_OPuj))` which implements optimizer-only sharding à la ZeRO-2. Or full sharding with `Fabric(strategy='fsdp')`)r*r+r,r-r.r/)r% _convert_deprecated_device_flagsr4r5r6listtypePLDDPShardedStrategyPLDDPSpawnShardedStrategy RuntimeErrorsuper__init__ PLStrategy_to_lite_strategy) selfr*r+r,r-r.r/r0r1Z lite_pluginsZ spawn_message __class__r3r9rDasD     zLightningLite.__init__)NNNr(r)NNN)__name__ __module__ __qualname____doc__rrstr PLAcceleratorrErintr _PL_PLUGIN_INPUTrD __classcell__r3r3rHr9r':s((r')r*r,r0r1r2cCs|durtd|d|d|durs^                                   c( +O