a þdœã@s¦ddlZddlmZmZmZmZmZddlZddlmZddl m Z ddl m Z ddl mZddlmZddlmZmZed Zed ZeeefZGd d „d ƒZdS) éN)ÚAnyÚDictÚ GeneratorÚOptionalÚUnion)ÚTensor)ÚModule)Ú Optimizer)ÚLiteral)Ú_convert_fp_tensor)Ú _PARAMETERSÚ Optimizable)é@é é)Z64Ú32Z16Zbf16c@súeZdZUdZdZeed<eedœdd„Ze j e ddœd d „ƒZ e e d œd d „Ze eeedœdd„Ze eeeeddœdd„Ze eeedœdd„Zeeedœdd„Zeedœdd„Zeeefdœdd„Zeeefddœd d!„Zddœd"d#„ZdS)$Ú PrecisionzÉBase class for all plugins handling the precision-specific parts of the training. The class attribute precision must be overwritten in child classes. The default value reflects fp32 training. rÚ precision)ÚmoduleÚreturncCs|S)z©Convert the module parameters to the precision type this plugin handles. This is optional and depends on the precision limitations during optimization. ©)Úselfrrrúu/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/lightning_fabric/plugins/precision/precision.pyÚconvert_module'szPrecision.convert_module)NNN)rccs dVdS)zWA contextmanager for managing model forward/training_step/evaluation_step/predict_step.Nr©rrrrÚforward_context.szPrecision.forward_context)ÚdatarcCs t|tjƒS)zÇConvert model inputs (forward) to the floating point precision type of this plugin. This is a no-op for tensors that are not of floating-point type or already have the desired type. )r ÚtorchÚfloat32)rrrrrÚ convert_input3szPrecision.convert_input)ÚtensorrrcCsdS)zýRuns before precision plugin executes backward. Args: tensor: The tensor that will be used for backpropagation module: The module that was involved in producing the tensor and whose parameters need the gradients Nr©rr rrrrÚ pre_backward:szPrecision.pre_backwardN)r ÚmodelÚargsÚkwargsrcOs|j|i|¤ŽdS)zñPerforms the actual backpropagation. Args: tensor: The tensor that will be used for backpropagation model: The module that was involved in producing the tensor and whose parameters need the gradients N)Úbackward)rr r#r$r%rrrr&BszPrecision.backwardcCsdS)züRuns after precision plugin executes backward. Args: tensor: The tensor that will be used for backpropagation module: The module that was involved in producing the tensor and whose parameters need the gradients Nrr!rrrÚ post_backwardKszPrecision.post_backward)Ú optimizerr%rcKs|jfi|¤ŽS)zHook to run the optimizer step.)Ústep)rr(r%rrrÚoptimizer_stepSszPrecision.optimizer_step)r(rccs|jD]}|dEdHqdS)z€The main params of the model. Returns the plain model params here. Maybe different in other precision plugins. ÚparamsN)Z param_groups)rr(ÚgrouprrrÚ main_params[s zPrecision.main_paramscCsiS)zªCalled when saving a checkpoint, implement to generate precision plugin state_dict. Returns: A dictionary containing precision plugin state. rrrrrÚ state_dictcszPrecision.state_dict)r.rcCsdS)zÞCalled when loading a checkpoint, implement to reload precision plugin state given precision plugin state_dict. Args: state_dict: the precision plugin state returned by ``state_dict``. Nr)rr.rrrÚload_state_dictkszPrecision.load_state_dictcCsdS)z‹This method is called to teardown the training process. It is the right place to release memory and free other resources. NrrrrrÚteardowntszPrecision.teardown)Ú__name__Ú __module__Ú __qualname__Ú__doc__rÚ_PRECISION_INPUT_STRÚ__annotations__rrÚ contextlibÚcontextmanagerrrrrrrr"r&r'r r*r r r-rÚstrr.r/r0rrrrrs"    ü  r)r7ÚtypingrrrrrrrZtorch.nnrZ torch.optimr Ztyping_extensionsr Z(lightning_fabric.plugins.precision.utilsr Z lightning_fabric.utilities.typesr r Z_PRECISION_INPUT_INTr5Z_PRECISION_INPUTrrrrrÚs