a dR@sddlZddlZddlmZmZddlmZddlmZm Z m Z m Z m Z m Z mZmZmZmZddlZddlmZddlmZddlmZddlZddlmZdd lmZdd lmZm Z m!Z!m"Z"m#Z#dd l$m%Z%dd l&m'Z'dd l(m)Z)m*Z*edZ+e+r6ddl,Z,ddl-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3e2ee3ddne4e5d<e4e5d<Gdddej6j7j8Z8ee8fZ9eee8fZ:eeeee8fZ;e ee e<e ee egeee8ffZ?Gddde/Z@GdddeZAGdddZBee=4.17.0)ActionConfigFileArgumentParserclass_from_function Namespace"register_unresolvable_import_pathsset_config_read_modeT)Zfsspec_enabledrr cs*eZdZeeeeddfdd ZZS)ReduceLROnPlateauN) optimizermonitorargskwargsreturncs$tj|g|Ri|||_dSN)super__init__r%)selfr$r%r&r' __class__^/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/pytorch_lightning/cli.pyr+6szReduceLROnPlateau.__init__)__name__ __module__ __qualname__rstrrr+ __classcell__r/r/r-r0r#5sr#c seZdZdZddddeeeeeddfdd Zdee d ee e e e ffee ee ee ee feeeeed d dZefddfeeeeeed ffeeddddZeddfeeeed ffeeddddZZS)LightningArgumentParserzAExtension of jsonargparse's ArgumentParser for pytorch-lightning.z+pytorch-lightning trainer command line toolZPLF description env_prefix default_envN)r&r8r9r:r'r(csBtsttdtj||||d|g|_i|_i|_dS)aInitialize argument parser that supports configuration file input. For full details of accepted arguments see `ArgumentParser.__init__ `_. Args: description: Description of the tool shown when running ``--help``. env_prefix: Prefix for environment variables. Set ``default_env=True`` to enable env parsing. default_env: Whether to parse environment variables. z2. Try `pip install -U 'jsonargparse[signatures]'`.r7N)"_JSONARGPARSE_SIGNATURES_AVAILABLEModuleNotFoundErrorr*r+ callback_keys _optimizers_lr_schedulers)r,r8r9r:r&r'r-r/r0r+Jsz LightningArgumentParser.__init__T.)lightning_class nested_key subclass_moderequiredr(cCst|rt|tst|}t|tr~t|ttttfr~t|trL|j ||rb|j ||d|dS|j ||dt|t ddSt d|ddS)aAdds arguments from a lightning class to a nested key of the parser. Args: lightning_class: A callable or any subclass of {Trainer, LightningModule, LightningDataModule, Callback}. nested_key: Name of the nested namespace to store arguments. subclass_mode: Whether allow any subclass of the given class. required: Whether the argument group is required. Returns: A list with the names of the class arguments added. F) fail_untypedrCT)rD instantiate sub_configszCannot add arguments from: zt. You should provide either a callable or a subclass of: Trainer, LightningModule, LightningDataModule, or Callback.N)callable isinstancetyper issubclassrrrrr=appendadd_subclass_argumentsadd_class_argumentsr)r,r@rArBrCr/r/r0add_lightning_class_argsfs&      z0LightningArgumentParser.add_lightning_class_argsr$ AUTOMATIC)optimizer_classrAlink_tor(cCst|tr"tdd|Ds0Jnt|ts0Jdddhd}t|tr^|j||fi|n|j||fddi|||f|j|<dS) acAdds arguments from an optimizer class to a nested key of the parser. Args: optimizer_class: Any subclass of :class:`torch.optim.Optimizer`. Use tuple to allow subclasses. nested_key: Name of the nested namespace to store arguments. link_to: Dot notation of a parser key to set arguments or AUTOMATIC. css|]}t|tVqdSr))rJr.0or/r/r0 z=LightningArgumentParser.add_optimizer_args..FparamsrErDskiprFTN)rHtupleallrJrrLrMr>)r,rPrArQr'r/r/r0add_optimizer_argss  z*LightningArgumentParser.add_optimizer_args lr_scheduler)lr_scheduler_classrArQr(cCst|tr"tdd|Ds0Jnt|ts0Jdddhd}t|tr^|j||fi|n|j||fddi|||f|j|<dS) aAdds arguments from a learning rate scheduler class to a nested key of the parser. Args: lr_scheduler_class: Any subclass of ``torch.optim.lr_scheduler.{_LRScheduler, ReduceLROnPlateau}``. Use tuple to allow subclasses. nested_key: Name of the nested namespace to store arguments. link_to: Dot notation of a parser key to set arguments or AUTOMATIC. css|]}t|tVqdSr))rJLRSchedulerTypeTuplerRr/r/r0rUrVz@LightningArgumentParser.add_lr_scheduler_args..Fr$rXrFTN)rHrZr[rJr_rLrMr?)r,r^rArQr'r/r/r0add_lr_scheduler_argss  z-LightningArgumentParser.add_lr_scheduler_args)FT)r1r2r3__doc__rr4boolr+rrrrrrr r rNrr r\r_LRSchedulerTyper`r5r/r/r-r0r6Gs\& 0 r6c@s>eZdZdZd eeeeeddddZe e eddd d Z dS) SaveConfigCallbacka!Saves a LightningCLI config to the log_dir when training starts. Args: parser: The parser object used to parse the configuration. config: The parsed configuration that will be saved. config_filename: Filename for the config file. overwrite: Whether to overwrite an existing config file. multifile: When input is multiple config files, saved config preserves this structure. Raises: RuntimeError: If the config file already exists in the directory to avoid overwriting a previous run config.yamlFN)parserconfigconfig_filename overwrite multifiler(cCs(||_||_||_||_||_d|_dS)NF)rfrgrhrirj already_saved)r,rfrgrhrirjr/r/r0r+s zSaveConfigCallback.__init__)trainer pl_modulestager(cCs|jr dS|j}|dusJtj||j}t|}|jsv|jrJ| |nd}|j |}|rvt |j jd|d|jr|j|dd|jj|j|d|j|jdd|_|j |j|_dS)NFz expected a to NOT exist. Aborting to avoid overwriting results of a previous run. You can delete the previous config file, set `LightningCLI(save_config_callback=None)` to disable config saving, or set `LightningCLI(save_config_overwrite=True)` to overwrite the config file.T)exist_ok)Z skip_nonerirj)rklog_dirospathjoinrhrriZis_global_zeroisfileZstrategy broadcast RuntimeErrorr.r1makedirsrfsavergrj)r,rlrmrnrpZ config_pathfsZ file_existsr/r/r0setups(  zSaveConfigCallback.setup)reFF) r1r2r3rar6r r4rbr+rrrzr/r/r/r0rds rdc@seZdZdZddededdddddddf eeee e de ffeeee e de ffeeeee e efeeee deffee e efeeefeee e efe e e e efffeeeeeeddddZedd d d Ze e efee e efe e effd d dZeed ddZee e efe e efddddZeddddZeddddZeddddZeddddZee e ee fdddZ eedd d!d"Z!ee eed#d$d%Z"eeddd&d'Z#eedd(d)d*Z$ddd+d,Z%ddd-d.Z&eed d/d0Z'e e efe(e)ed1d2d3Z*ee ed4d5d6Z+edEe e,ee-ed7d8d9Z.ee dd4d:d;Z/dFe0e eeed<d=d>Z1e dd4d?d@Z2e e e efd4dAdBZ3dddCdDZ4dS)G LightningCLIzIImplementation of a configurable command line tool for pytorch-lightning.NTF.) model_classdatamodule_classsave_config_callbacksave_config_kwargs trainer_classtrainer_defaultsseed_everything_default parser_kwargssubclass_mode_modelsubclass_mode_datar&runauto_configure_optimizersr'r(cKs||_|p i|_||_|pi|_||_|p,i|_| |_||||_|pLt |_ |dupZ| |_ ||_ |pjt |_|dupx| |_||j\}}|| ||||j| | r|jdnd|_||||jdur||jdS)a9 Receives as input pytorch-lightning classes (or callables which return pytorch-lightning classes), which are called / instantiated using a parsed configuration file and / or command line args. Parsing of configuration from environment variables can be enabled by setting ``parser_kwargs={"default_env": True}``. A full configuration yaml would be parsed from ``PL_CONFIG`` if set. Individual settings are so parsed from variables named for example ``PL_TRAINER__MAX_EPOCHS``. For more info, read :ref:`the CLI docs `. .. warning:: ``LightningCLI`` is in beta and subject to change. Args: model_class: An optional :class:`~pytorch_lightning.core.module.LightningModule` class to train on or a callable which returns a :class:`~pytorch_lightning.core.module.LightningModule` instance when called. If ``None``, you can pass a registered model with ``--model=MyModel``. datamodule_class: An optional :class:`~pytorch_lightning.core.datamodule.LightningDataModule` class or a callable which returns a :class:`~pytorch_lightning.core.datamodule.LightningDataModule` instance when called. If ``None``, you can pass a registered datamodule with ``--data=MyDataModule``. save_config_callback: A callback class to save the config. save_config_kwargs: Parameters that will be used to instantiate the save_config_callback. trainer_class: An optional subclass of the :class:`~pytorch_lightning.trainer.trainer.Trainer` class or a callable which returns a :class:`~pytorch_lightning.trainer.trainer.Trainer` instance when called. trainer_defaults: Set to override Trainer defaults or add persistent callbacks. The callbacks added through this argument will not be configurable from a configuration file and will always be present for this particular CLI. Alternatively, configurable callbacks can be added as explained in :ref:`the CLI docs `. seed_everything_default: Number for the :func:`~lightning_fabric.utilities.seed.seed_everything` seed value. Set to True to automatically choose a seed value. Setting it to False will avoid calling ``seed_everything``. parser_kwargs: Additional arguments to instantiate each ``LightningArgumentParser``. subclass_mode_model: Whether model can be any `subclass `_ of the given class. subclass_mode_data: Whether datamodule can be any `subclass `_ of the given class. args: Arguments to parse. If ``None`` the arguments are taken from ``sys.argv``. Command line style arguments can be given in a ``list``. Alternatively, structured config options can be given in a ``dict`` or ``jsonargparse.Namespace``. run: Whether subcommands should be added to run a :class:`~pytorch_lightning.trainer.trainer.Trainer` method. If set to ``False``, the trainer and model classes will be instantiated only. N subcommand)r~rrrrrr_handle_deprecated_paramsr|r _model_classrr}r_datamodule_classr_setup_parser_kwargs setup_parserparse_argumentsrfrgr _set_seedbefore_instantiate_classesinstantiate_classes_run_subcommand)r,r|r}r~rrrrrrrr&rrr' main_kwargssubparser_kwargsr/r/r0r+ s.;       zLightningCLI.__init__)r'r(cCs|gd@D]B}||}|dddd}||j|<td|d|dq|gd @D]:}||}|d d }||j|<td|d |dqd|rtd |dS)N)Zsave_config_filenameZsave_config_overwriteZsave_config_multifileZ save_config_filenamerhzLightningCLI's za init parameter is deprecated from v1.8 and will be removed in v2.0.0. Use `save_config_kwargs={'z': ...}` instead.)r8r9 env_parserr:zZ init parameter is deprecated from v1.9 and will be removed in v2.0. Use `parser_kwargs={'zUnexpected keyword parameters: )keyspopreplacerrr ValueError)r,r'namevaluekeyr/r/r0rhs(     z&LightningCLI._handle_deprecated_params)rr(cs@|fdd|D}fdd|D}||fS)Ncsi|]\}}|vr||qSr/r/rSkvZsubcommand_namesr/r0 rVz5LightningCLI._setup_parser_kwargs..csi|]\}}|vr||qSr/r/rrr/r0rrV) subcommandsritems)r,rrrr/rr0r~s z!LightningCLI._setup_parser_kwargscKs:|ddtjgtfi|}|jddtdd|S)z-Method that instantiates the argument parser.Z dump_headerzpytorch_lightning==z-cz--configz4Path to a configuration file in json or yaml format.)actionhelp) setdefaultpl __version__r6 add_argumentr)r,r'rfr/r/r0 init_parsers zLightningCLI.init_parser)add_subcommandsrrr(cCsB|jfi||_|r2i|_|j|jfi|n ||jdS)z.model)rBNdataF)rBrC) rNrrr set_defaultsrrr}rr)r,rfrr/r/r0add_core_arguments_to_parsers   z)LightningCLI.add_core_arguments_to_parsercCsT|||||||jrF|js6|tf|jsF|t | |dSr)) rradd_arguments_to_parserrr>r\rr?r`r_!link_optimizers_and_lr_schedulersrr/r/r0rs     zLightningCLI._add_argumentscCsdS)zImplement to add extra arguments to the parser or link arguments. Args: parser: The parser object to which arguments can be added Nr/rr/r/r0rsz$LightningCLI.add_arguments_to_parser)r(cCs$hdhdhdhdhddS)zDDefines the list of available subcommands and the arguments to skip.> datamoduleZval_dataloadersrZtrain_dataloaders>rrZ dataloaders)ZfitvalidatetestZpredictZtuner/r/r/r/r0rs zLightningCLI.subcommands)rfr'r(c Ksi|_|}t|jtr |jnt|j}|D]\}t||}t|}| |i}| d||j ||fi|} | |j|<|j || |dq2dS)z%Adds subcommands to the input parser.r8)rN) _subcommand_parsersrrHrrIrrgetattr_get_short_descriptiongetr_prepare_subcommand_parserZadd_subcommand) r,rfr'Zparser_subcommandsrrfnr8rZsubcommand_parserr/r/r0rs     zLightningCLI._add_subcommands)klassrr'r(cKsH|jfi|}||t||}|j|||d}||j|<|S)N)rY)rrsetrZadd_method_argumentsr)r,rrr'rfrYaddedr/r/r0rs   z'LightningCLI._prepare_subcommand_parsercCsdi|j|j}|D]F\}\}}|dkr.qt|trF|||qt|}|j|||dqdS)z`Creates argument links for optimizers and learning rate schedulers that specified a ``link_to``.rO)Z compute_fnN)r>r?rrHrZZlink_arguments_add_class_path_generator)rfZoptimizers_and_lr_schedulersr class_typerQadd_class_pathr/r/r0rs z.LightningCLI.link_optimizers_and_lr_schedulers)rfr&r(cCsb|dur6ttjdkr6tdtjddd|dt|ttfrR|||_n | ||_dS)z?Parses command line arguments and stores it in ``self.config``.NzLightningCLI's args parameter is intended to run from within Python like if it were from the command line. To prevent mistakes it is not allowed to provide both args and command line arguments, got: sys.argv[1:]=z, args=.) lensysargvrrHdictr parse_objectrg parse_args)r,rfr&r/r/r0rs zLightningCLI.parse_argumentscCsdS)zrVz4LightningCLI.instantiate_trainer..rl)r)_parserrr=rr_instantiate_trainer)r,r'Zextra_callbacksZtrainer_configr/rr0rsz LightningCLI.instantiate_trainer)rgrr(cCsd}||vr||dur"g||<nt||ts>||g||<|||||jvr|j|}||t|trv|n|g7<|jr|dds|j||j|jt |j|jfi|j }|| |nt d|j jd|d|j fi|S)NrZ fast_dev_runFzThe `z` class does not expose the `z(` argument so they will not be included.)rHlistextendrr~rrrrgr4rrKrrr3)r,rgrrrZconfig_callbackr/r/r0rs,      z!LightningCLI._instantiate_trainer)rr(cCs|dur|jS|j|Sr))rfrr,rr/r/r0r8szLightningCLI._parser)lightning_moduler$r]r(cCs4|dur |St|tr(|||jddS|g|gfS)a,Override to customize the :meth:`~pytorch_lightning.core.module.LightningModule.configure_optimizers` method. Args: lightning_module: A reference to the model. optimizer: The optimizer. lr_scheduler: The learning rate scheduler (if used). N)Z schedulerr%r$r])rHr#r%)rr$r]r/r/r0configure_optimizers>s   z!LightningCLI.configure_optimizersc Cs|js dS||}ttttdfftttttttdfftffttddd}|t|j }|t |j }t |dkrdSt |dkst |dkrt d|jjd ||d |j |dd}||j|d}t|tst||}|sdSd}|r<|j |dd} ||j|d}t| ts.get_automatic..)rrHrZanyrK)rrZ automaticrZ base_classrQr/rr0 get_automatic[s  zMLightningCLI._add_configure_optimizers_method_to_model..get_automaticrr`z{.add_configure_optimizers_method_to_model` expects at most one optimizer and one lr_scheduler to be 'AUTOMATIC', but found z. In this case the user is expected to link the argument groups and implement `configure_optimizers`, see https://pytorch-lightning.readthedocs.io/en/stable/common/lightning_cli.html#optimizers-and-learning-rate-schedulersrz..configure_optimizers` will be overridden by `z.configure_optimizers`.r)rrrr r rr4r rr>r_r?rrr.r1rrrHrZ_global_add_class_pathrrrinstantiate_class parametersrrrr) r,rrfrZ optimizersZ lr_schedulersrPZoptimizer_initZlr_scheduler_initr^r$r]rr/r/r0rSsN 4         z6LightningCLI._add_configure_optimizers_method_to_model)rgrrr(cCs|t|j|||S)zAUtility to get a config value which might be inside a subcommand.)rr4r)r,rgrrr/r/r0rszLightningCLI._getcCstt|d|d}t|r |t|j|}t|||}||}|fi|t|d|d}t|rp|dS)zRun the chosen subcommand.Zbefore_NZafter_)rrGrl_prepare_subcommand_kwargs)r,rZ before_fnrr fn_kwargsZafter_fnr/r/r0rs   zLightningCLI._run_subcommandcs@fddjD}j|d<jdur<j|d<|S)z@Prepares the keyword arguments to pass to the subcommand to run.cs$i|]\}}|jvr||qSr/)rrrr/r0rsz;LightningCLI._prepare_subcommand_kwargs..rNr)rrrr)r,rrr/rr0rs     z'LightningCLI._prepare_subcommand_kwargscCsH||jd}|durdS|dur.tdd}n t|dd}||jd<dS)zSets the seed.rFNT)workers)rrgr)r,Z config_seedr/r/r0rs  zLightningCLI._set_seed)N)N)5r1r2r3rardrr rr rrrrr4rrbrArgsTyper+rrr rr6rrrrrr staticmethodr rrrrrrrrr rrrrLRSchedulerTypeUnionrrr rrrrr/r/r/r0r{s  & ]0          > r{)rr(cCs|jd|jS)Nr)r2r1rr/r/r0_class_path_from_classsr)r init_argsr(cCs$t|tr|}t||pidS)N) class_pathr)rHr Zas_dictr)rrr/r/r0rs rcs ttttfdfdd }|S)N)rr(cs t|Sr))r)rrr/r0rsz1_add_class_path_generator..add_class_path)r rr4r)rrr/rr0rsr.)r&initr(cCsV|di}t|ts|f}|ddd\}}t||gd}t||}||i|S)zInstantiates a class with the given args and init. Args: args: Positional arguments required for instantiation. init: Dict of the form {"class_path":...,"init_args":...}. Returns: The instantiated class object. rrrr)fromlist)rrHrZrsplit __import__r)r&rr'Z class_module class_namemoduleZ args_classr/r/r0rs   r) componentr(c Csf|jdurdSzt|j}|jWSttjfy`}z td|d|WYd}~n d}~00dS)NzFailed parsing docstring for z: )radocstring_parserparseZshort_descriptionr ParseErrorr)r docstringexr/r/r0rs  r)N)Hrqr functoolsrrtypesrtypingrrrrr r r r r rZtorchZ lightning_utilities.core.importsrZ"lightning_utilities.core.rank_zerorZ torch.optimrZpytorch_lightningrZ#lightning_fabric.utilities.cloud_iorZ lightning_fabric.utilities.typesrrrrrrZ&pytorch_lightning.utilities.exceptionsrZ)pytorch_lightning.utilities.model_helpersrZ%pytorch_lightning.utilities.rank_zerorrr;rZ jsonargparserrrr r!r"objectlocalsZoptimr]r#r_rrcr4rZOptimizerCallableZLRSchedulerCallabler6rdr{rrrrrr/r/r/r0sZ 0            B6 "*