a dZN @sddlZddlmZddlmZmZmZmZmZm Z m Z m Z ddl Z ddl mZddl mZddlmZmZmZmZmZmZmZddlZddlmZmZddlmZdd lmZ dd l!m"Z"dd l#m$Z$dd l%m&Z&dd l'm(Z(m)Z)m*Z*ddl+m,Z,ddl-m.Z.ddl/m0Z0m1Z1m2Z2e ee3eedfedfZ4e2Z5e4ee e6ddfdddZ7e4e6dddZ8e ee&fde de9dddZ:e eefe e6e;fdddZd8ee eefe e$e9ee3efd#d&d'Z?eed(d)d*Z@e9e e3d+fe ed+fee3efe e e3d+fee3effd,d-d.ZAeBe9dd/d0ZCeeed1d2d3Zeeed1d4d5ZdS)9N)fields)AnyDict GeneratorIterableMappingOptionalTupleUnion)is_dataclass_instance)Tensor) BatchSampler DataLoaderDatasetIterableDataset RandomSamplerSamplerSequentialSampler)_reinstantiate_wrapped_cls_replace_value_in_saved_args)has_iterable_dataset)has_len)IndexBatchSamplerWrapper) RunningStage)CombinedLoader)CaptureIterableDatasetCaptureMapDatasetFastForwardSampler)_FaultTolerantMode)MisconfigurationException)rank_zero_deprecationrank_zero_warn WarningCacheBType)batchreturnccst|tr*|jdkrdVq|dVnxt|ttfrnt|tsnt|trT|}|D]}t|EdHqXn4t |rt |D]}tt ||j EdHq~ndVdS)Nr) isinstancer ndimsizerrstrvalues_extract_batch_sizer rgetattrname)r$samplefieldr1i/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/pytorch_lightning/utilities/data.pyr,1s    r,cCsvd}d}z>t|D]0}|dur$|}q||krtd|dqDqWnty`t|Yn0|durrt||S)zUnpack a batch to find a ``torch.Tensor``. Returns: ``len(tensor)`` when found, or ``1`` when it hits an empty or non iterable. zWe could not infer the batch_size from the batch. Either simplify its structure or provide the batch_size as `self.log(..., batch_size=batch_size)`.NzZTrying to infer the `batch_size` from an ambiguous collection. The batch size we found is zK. To avoid any miscalculations, use `self.log(..., batch_size=batch_size)`.)r, warning_cachewarnRecursionErrorr)r$ error_msg batch_sizebsr1r1r2extract_batch_sizeDs&   r9zpl.strategies.Strategy)zpl.LightningModulezpl.LightningDataModule) dataloaderstrategymodelr%c Cszt|}|jtj||jddd}|dkr@td|jjd|dkr|dkr|jrptd|jjdd}qt d |jjd nd }Wnt t fyd}Yn0|rt |rtd |S) z}Checks if a given Dataloader has ``__len__`` method implemented i.e. if it is a finite dataloader or infinite dataloader.)Zdevicesum)Z reduce_oprzTotal length of `zA` across ranks is zero. Please make sure this was your intention.zb` across ranks is zero, but local rank has zero length. Please be cautious of uneven batch length.F`zW` within local rank has zero length. Please make sure that it returns at least 1 batch.TzYour `IterableDataset` has `__len__` defined. In combination with multi-process data loading (when num_workers > 1), `__len__` could be inaccurate if each worker is not configured independently to avoid having duplicate data.) lenreducetorchZtensorZ root_devicer! __class____name__Z2allow_zero_length_dataloader_with_multiple_devicesr TypeErrorNotImplementedErrornew_has_iterable_dataset)r:r;r<Z local_length total_lengthrr1r1r2has_len_all_ranksbs0  rH)r:r%cCst|rt|StdS)zsReturn the length of the given DataLoader. If ``__len__`` method is not implemented, return float('inf'). inf) new_has_lenr?float)r:r1r1r2get_lensrL)r:samplermoder%cCs*t|||\}}t|g|Ri|}|SN)$_get_dataloader_init_args_and_kwargsr)r:rMrNdl_args dl_kwargsr1r1r2_update_dataloadersrSF)r:rMrNdisallow_batch_samplerr%c s6t|tstd|dt|d}|rB|j}|j|j|j}n(ddt| Dd}|j d<dt t |jj}tdd |D}|r|r|d dt tjj Dn |t tjj|d d|sfd d | Ddfdd Dd}d|} t| trHdd<dd<nt||||fdd |D} | rt| } |jj} ddd | D} td| d| d| d| d |sttB|}|rt|}|jj} td| d|d| dt j!r.t"||\}|fS)NzThe dataloader z0 needs to subclass `torch.utils.data.DataLoader`__pl_saved_argscSs i|]\}}|ds||qS)_) startswith.0kvr1r1r2 z8_get_dataloader_init_args_and_kwargs..multiprocessing_contextr1css|]}|j|juVqdSrO)kind VAR_KEYWORDrYpr1r1r2 r]z7_get_dataloader_init_args_and_kwargs..cSs"i|]\}}|j|jur||qSr1)defaultemptyrXr1r1r2r\r]selfcs*h|]"\}}|vr|j|ur|qSr1)rd)rYr.rb)attrsr1r2 r]z7_get_dataloader_init_args_and_kwargs..datasetcsi|]\}}|vr||qSr1r1rX) non_defaultsr1r2r\r] batch_samplerrMcsDh|]<}|j|j|jfvr|j|jur|jvr|jvr|jqSr1)r_POSITIONAL_ONLYPOSITIONAL_OR_KEYWORDrdrer.ra) arg_namesrRr1r2rhs    z, css|]}d|dVqdS)z`self.r>Nr1)rYZarg_namer1r1r2rcr]z,Trying to inject custom `Sampler` into the `z` instance. This would fail as some of the `__init__` arguments are not available as instance attributes. The missing attributes are z. If you instantiate your `zZ` inside a `*_dataloader` hook of your module, we will do this for you. Otherwise, define z inside your `__init__`.z&Trying to inject parameters into the `z{` instance. This would fail as it doesn't expose all its attributes in the `__init__` signature. The missing arguments are z. HINT: If you wrote the `zA` class, add the `__init__` arguments or allow passing `**kwargs`)#r'r ValueErrorhasattrrU__pl_saved_kwargs__pl_saved_arg_namesZ __datasetvarsitemsr^dictinspect signature__init__ parametersanyr+updatepopaddgetr'_dataloader_init_kwargs_resolve_samplersortedrBrCjoinrsetkeysrdetect_current_mode is_automatic7_apply_fault_tolerant_automatic_capture_dataset_wrapper)r:rMrNrT was_wrappedrQZoriginal_datasetparamsZhas_variadic_kwargsriZ required_argsZsorted_required_argsZdataloader_cls_nameZmissing_args_messageZmissing_kwargsZsorted_missing_kwargsr1)rnrgrRrjr2rPs           rPc Cst}t|d}|tjk}|dur|rXt|turL|j|krL|j|jksTt dnft|tusj|rt|}t |dr|j }|j } |j } |j} |rtdd|| | | \} }} | std|jdtd ||| | | \} }} | std |jd t|g|Ri| }nrz |||j|r"dn|jd }WnPty~} z6d dl}|dt| }|s`t d| WYd} ~ n d} ~ 00|rt|}|jrt|}}|jdddd|dddS|jrt|}}|j|jd|dddS)aSThis function is used to handle the sampler, batch_sampler arguments associated within a DataLoader for its re-instantiation. If the dataloader is being used for prediction, the sampler will be wrapped into an `IndexBatchSamplerWrapper`, so Lightning can keep track of its indices. If fault tolerant training is enabled, the sampler will be wrapped into a `FastForwardSampler`. If there are multiple devices in IPU mode, it is necessary to disallow BatchSampler that isn't instantiated automatically, since `poptorch.DataLoader` will try to increase the batch_size rkNzdIt is not possible to have a batch sampler in your dataloader, when running on multiple IPU devices.rU drop_lastFzlTrying to inject `drop_last=False` into batch sampler since you are predicting, however it seems the class `z` does not support it. Your predictions might be incomplete. To mitigate this, expose `drop_last` in the `__init__` method of your custom class.rMzYTrying to inject a modified sampler into the batch sampler; however, it seems the class `z` does not have an argument called `sampler.` To mitigate this, expose an argument `sampler` in the `__init__` method of your custom class.)r7rrz:.*__init__\(\) (got multiple values)|(missing \d required)zWe tried to re-instantiate your custom batch sampler and failed. To mitigate this, either follow the API of `BatchSampler` or instantiate your custom batch sampler inside `*_dataloader` hooks of your module.r&)Zdataloader_batch_size)rMshufflerkr7r)rMrrk)rrr-rZ PREDICTINGtyper rMr7rrprUrqZ__pl_saved_default_kwargsrrrr! __qualname__rDrrrematchr*rrrsetup)r:rMrNrTZfault_tolerant_moderkZ is_predictingZbatch_sampler_clsargskwargsZdefault_kwargsrnsuccesserrZfast_forward_samplerr1r1r2rs            r)rir%cCs:t|trt|dSt|tdkr.t|dStddS)NrirIzKThis shouldn't happen, please open an issue on Lightning Github repository.)r'rrrLrKr RuntimeErrorrr1r1r2_wrap_with_capture_datasetss    r.)rrnrQrRr%cCs~d|vrt|d|d<n\d|vr\|d}t||}|d||f||dd}n|rfd}nd}td|||fS)Nrir&z To avoid this, either pass `DataLoader(dataset=your_dataset)` or the positional dataset argument `DataLoader(your_dataset, ...)`.zT To avoid this, define `self.dataset = dataset` inside your DataLoader's `__init__`.zYou enabled automatic Fault Tolerant mode, but we were not able to replace your dataset with Fault Tolerant wrapper, because you have a custom DataLoader.)rindexr)rrnrQrRZ dataset_idxriZ avoid_messager1r1r2r|s   $rcCsvt|dr:d|jvr|jdSd|jvr:|j|jdSt|jtrJdSt|dsXdS|j}t|t rldSt|t S)NrqrFrM) rprqrrrUrr'rirrMrr)r:rMr1r1r2_is_dataloader_shuffleds       r)rrr%cOstdt|i|S)Nz`pytorch_lightning.utilities.data.has_iterable_dataset` has been deprecated in v1.8.0 and will be removed in v2.0.0. Please use `lightning_fabric.utilities.data.has_iterable_dataset` instead.)r rFrrr1r1r2rsrcOstdt|i|S)Nz`pytorch_lightning.utilities.data.has_len` has been deprecated in v1.8.0 and will be removed in v2.0.0. Please use `lightning_fabric.utilities.data.has_len` instead.)r rJrr1r1r2rsr)N)NF)NF)DrvZ dataclassesrtypingrrrrrrr r rAZ#lightning_utilities.core.apply_funcr r Ztorch.utils.datar rrrrrrZpytorch_lightningplZlightning_fabric.utilities.datarrrrFrrJZ'pytorch_lightning.overrides.distributedrZ pytorch_lightning.trainer.statesrZ$pytorch_lightning.trainer.supportersrZ(pytorch_lightning.utilities.auto_restartrrrZ!pytorch_lightning.utilities.enumsrZ&pytorch_lightning.utilities.exceptionsrZ%pytorch_lightning.utilities.rank_zeror r!r"r*r#r3intr,r9boolrHrKrLrSrPrrrobjectrr1r1r1r2sr (  $          -    g  l