a do@sddlZddlZddlmZmZmZddlZddlmZddlm Z ddl Z ddl m Z mZddlmZdddd ZGd d d e Zed feeeejfed ddZeedddZdS)N)AnyOptionalUnion)apply_to_collection)Tensor)_LightningModuleWrapperBase$_LightningPrecisionModuleWrapperBase)rank_zero_warn)returncCstjddddS)NignorezvWas asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.)message)warningsfilterwarningsrrr/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/pytorch_lightning/overrides/data_parallel.py_ignore_scalar_return_in_dpsrcsjeZdZdZd eedefeedefddfdd Zeeedfdd Z edd d d Z Z S)LightningParallelModulea@Wraps the user's LightningModule and redirects the forward call to the appropriate method, either ``training_step``, ``validation_step``, ``test_step``, or ``predict_step``. This class is used in combination with :class:`~torch.nn.parallel.DataParallel` as shown in the example. It also takes care of converting Python scalars to Tensors and un-squeezes 0-dimensional Tensors as it is required by :class:`~torch.nn.parallel.DataParallel`. Example: dp_model = torch.nn.DataParallel( module=LightningParallelModule(lightning_module), device_ids=[3, 4], ... ) Args: pl_module: The module to wrap. See description for `forward_module`. .. deprecated:: v1.8.0 The argument ``pl_module`` is deprecated in v1.8.0 and will be removed in v2.0.0. Please use ``forward_module`` instead. forward_module: The module to wrap. If it's not a ``LightningModule``, it must have an attribute ``.module`` pointing to a ``LightningModule`` reference. Nzpl.LightningModule)forward_module pl_moduler cs(|||tj|p|dtdS)N)r)Z_validate_init_argumentssuper__init__r)selfrr __class__rrrAs z LightningParallelModule.__init__)inputskwargsr csH|tj|i|}ttdfdd }t|tjtf|d}|S)Ndatar csjj}t||}t|}|S)N)Zlightning_moduledevicepython_scalar_to_tensorunsqueeze_scalar_tensorrrrrroutput_transformOs z9LightningParallelModule.forward..output_transformZdtypefunction) update_replica_device_attributesrforwardrrnumbersNumberr)rrroutputr#rr"rr'Js  zLightningParallelModule.forward)rr csJdttdfdd }t|t|ddur>|jjdntddS)a/Updates the device information of LightningModule by reading the device from the inputs. In :class:`~torch.nn.data_parallel.DataParallel` changes to the state during the `forward` pass are lost when the replicas get discarded. The only way to know the current device is from the inputs passed into the model. Args: inputs: A collection of inputs (typically a tuple). If the inputs don't contain tensors, a warning is shown that accessing ``self.device`` will not return the correct device. N)tensorr cs"dur|jtdkr|j|S)Ncpu)rtorch)r+Zreplica_devicerrfind_tensor_with_devicedszYLightningParallelModule.update_replica_device_attributes..find_tensor_with_devicer$rzCould not determine on which device the inputs are. When using DataParallel (strategy='dp'), be aware that in case you are using self.device in your code, it will reference only the root device.)rrZ_forward_moduletor )rrr/rr.rr&Xs z8LightningParallelModule.update_replica_device_attributes)NN) __name__ __module__ __qualname____doc__rrrrrr'r& __classcell__rrrrr&s rr,)rrr cCs t|tjrtj|g|d}|S)zTConverts a Python scalar number to a torch tensor and places it on the given device.r0) isinstancer(r)r-r+r!rrrrws rrcCs$t|tr |dkr |d}|S)zUn-squeezes a 0-dim tensor.r)r7rZdimZ unsqueeze)rrrrr ~s r )r(r typingrrrr-Z#lightning_utilities.core.apply_funcrrZpytorch_lightningplZ pytorch_lightning.overrides.baserrZ%pytorch_lightning.utilities.rank_zeror rrrstrrr rrrrs    Q&