a d$@sdZddlZddlmZmZmZmZddlmZddl m Z ddl m Z ddl mZeeefZeeeeegeffdd d Zeed d d Zeed ddZeed ddZeed ddZedddZeed ddZdS)a7Contains migration functions to upgrade legacy checkpoints to the format of the current Lightning version. When Lightning loads a checkpoint, these migrations will be applied on the loaded checkpoint dictionary sequentially, see :func:`~pytorch_lightning.utilities.migration.utils.migrate_checkpoint`. For the Lightning developer: How to add a new migration? 1. Create a new function with a descriptive name and docstring that explains the details of this migration. Include version information as well as the specific commit or PR where the breaking change happened. 2. Add the function to the `_migration_index()` below. The key in the index is the version of Lightning in which the change happened. Any checkpoint with a version greater or equal to that version will apply the given function. Multiple migrations per version get executed in the provided list order. 3. You can test the migration on a checkpoint (backup your files first) by running: cp model.ckpt model.ckpt.backup python -m pytorch_lightning.utilities.upgrade_checkpoint --file model.ckpt N)AnyCallableDictList)PossibleUserWarning) EarlyStopping)ModelCheckpoint)rank_zero_warn)returncCstgttgtgtgdS)zQMigration functions returned here will get executed in the order they are listed.)z0.10.0z1.6.0z1.6.5z1.9.0)(_migrate_model_checkpoint_early_stopping._migrate_loop_global_step_to_progress_tracking0_migrate_loop_current_epoch_to_progress_tracking"_migrate_loop_batches_that_stepped9_migrate_model_checkpoint_save_on_train_epoch_end_defaultrrx/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/pytorch_lightning/utilities/migration/migration.py_migration_index*s r) checkpointr cCstdftdftdftdftdfd}|dp0i|d<|D]P\}}||vr>||}|\}}|d|pni|d|<||d||<||=q>|S)zbThe checkpoint and early stopping keys were renamed. Version: 0.10.0 Commit: a5d1176 Zbest_model_scoreZbest_model_pathZ wait_countZpatience)Z$checkpoint_callback_best_model_scoreZ#checkpoint_callback_best_model_pathZcheckpoint_callback_bestZearly_stop_callback_waitZearly_stop_callback_patience callbacks)rrgetitems)rZ keys_mappingkeynew_pathvalueZ callback_typeZ callback_keyrrrr 4sr cCsp|d}|ddti|ddt|ddd}||dddd<|ddd }||dd<|S) zSets the `global_step` value for checkpoints before v1.6 without the progress tracking state. It will be overwritten by the loop's state if it was also saved. Version: 1.6.0 Commit: c67b075 PR: #13645, #11805 global_steploopsfit_loop3epoch_loop.batch_loop.optimizer_loop.optim_progress optimizersteptotal completed5epoch_loop.batch_loop.manual_loop.optim_step_progress setdefault!_get_fit_loop_initial_state_1_6_0)rrZoptim_progressZoptim_step_progressrrrr Ms r cCsH|d}|ddti|ddt||ddddd<|S)zSets the `current_epoch` value for checkpoints before v1.6 without the progress tracking state. It will be overwritten by the loop's state if it was also saved. Version: 1.6.0 Commit: aea96e4 PR: #11805 epochrrepoch_progresscurrentr!r#)rr&rrrr as r cCs$|d}|dddd||S)zSets the `_batches_that_stepped` default value for checkpoints before v1.6.5 which don't have this key. Version: 1.6.5 Commit: c67b075 PR: #13645 rrrepoch_loop.state_dict_batches_that_stepped)r$)rrrrrrpsrcCsdddddddidddddddddddddddddddiidddddddddddd dddddddd diddddddddddddddddddd iidddddddddddid S) Nr)r!ready)r(r )r!r+started)rZ zero_grad)rZoptimizer_position)r! processedr+r,F)r(Z is_last_batchr r*)r"z,epoch_loop.batch_loop.manual_loop.state_dictrz/epoch_loop.batch_loop.optimizer_loop.state_dictz epoch_loop.batch_loop.state_dictzepoch_loop.batch_progresszepoch_loop.scheduler_progressr)z'epoch_loop.val_loop.dataloader_progressz-epoch_loop.val_loop.epoch_loop.batch_progressz)epoch_loop.val_loop.epoch_loop.state_dictzepoch_loop.val_loop.state_dictr'Z state_dictrrrrrr%|sF         r%csjd|vr |Sttdddt|d}fdd|dD}t||kr^tdtd|S||d<|S) a9The ``save_on_train_epoch_end`` was removed from the state-key of ``ModelCheckpoint`` in 1.9.0, and this migration drops it from the state-keys saved in the checkpoint dict so that the keys match when the Trainer loads the callback state. Version: 1.9.0 Commit: f4ca56 PR: #15300, #15606 r)old_keyr cSs|ds|Stdd|S)Nrz., 'save_on_train_epoch_end': (None|True|False)) startswithresub)r.rrrnew_keys zJ_migrate_model_checkpoint_save_on_train_epoch_end_default..new_keycs$i|]\}}t|tr||qSr) isinstancestr).0r.stater3rr szM_migrate_model_checkpoint_save_on_train_epoch_end_default..aNYou have multiple `ModelCheckpoint` callback states in this checkpoint, but we found state keys that would end up colliding with each other after an upgrade, which means we can't differentiate which of your checkpoint callbacks needs which states. At least one of your `ModelCheckpoint` callbacks will not be able to reload the state.)category)r5lenrr r)rZnum_keysZnew_callback_statesrr8rrs     r)__doc__r1typingrrrrZ#lightning_fabric.utilities.warningsrZ*pytorch_lightning.callbacks.early_stoppingrZ,pytorch_lightning.callbacks.model_checkpointrZ%pytorch_lightning.utilities.rank_zeror r5Z _CHECKPOINTrr r r rr%rrrrrs     $  0