a  d@sLddlZddlZddlmZddlmZGdddeZGdddeZdS)N)Samplerc@s*eZdZdZd ddZddZddZdS) OrderedDistributedSampleraSampler that restricts data loading to a subset of the dataset. It is especially useful in conjunction with :class:`torch.nn.parallel.DistributedDataParallel`. In such case, each process can pass a DistributedSampler instance as a DataLoader sampler, and load a subset of the original dataset that is exclusive to it. .. note:: Dataset is assumed to be of constant size. Arguments: dataset: Dataset used for sampling. num_replicas (optional): Number of processes participating in distributed training. rank (optional): Rank of the current process within num_replicas. NcCs|dur tstdt}|dur@ts8tdt}||_||_||_tt t |jd|j|_ |j |j|_ dS)N,Requires distributed package to be availableg?)dist is_available RuntimeErrorget_world_sizeget_rankdataset num_replicasrankintmathceillen num_samples total_size)selfr r r rf/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/timm/data/distributed_sampler.py__init__s z"OrderedDistributedSampler.__init__cCslttt|j}||d|jt|7}t||jks>J||j|j|j}t||jksdJt|SN) listrangerr rr r riter)rindicesrrr__iter__%s z"OrderedDistributedSampler.__iter__cCs|jSr)rrrrr__len__2sz!OrderedDistributedSampler.__len__)NN)__name__ __module__ __qualname____doc__rrrrrrrrs  rc@s2eZdZdZdddZd d Zd d Zd dZdS)RepeatAugSampleraSampler that restricts data loading to a subset of the dataset for distributed, with repeated augmentation. It ensures that different each augmented version of a sample will be visible to a different process (GPU). Heavily based on torch.utils.data.DistributedSampler This sampler was taken from https://github.com/facebookresearch/deit/blob/0c4b8f60/samplers.py Used in Copyright (c) 2015-present, Facebook, Inc. NTrcCs|dur tstdt}|dur@ts8tdt}||_||_||_||_||_ d|_ t t t|j||j|_|j|j|_|p|}|rt t t|j||||_nt t t|j||_dS)Nrr)rrrrr r r r shuffle num_repeatsepochr rrrrrfloornum_selected_samples)rr r r r&r'Zselected_roundZselected_ratiorrrrAs,   zRepeatAugSampler.__init__cs"t}|jjr0tjtj|d}ntjdtjd}t j t rj st j tj}|tfddt|D}ntj|tj dd}|}jt|}|dkr||d|7}t|jksJ|jjj}t|jksJt|djS)N) generatorr)startendcsg|]}t|jqSr)r r').0irrr rz-RepeatAugSampler.__iter__..)ZrepeatsZdim)torch GeneratorZ manual_seedr(r&Zrandpermrr Zarange isinstancer'float is_integerrrZtensorrZrepeat_interleaver tolistrr r rrr*)rgrZ repeat_sizeZ padding_sizerrrres" "zRepeatAugSampler.__iter__cCs|jSr)r*rrrrrszRepeatAugSampler.__len__cCs ||_dSr)r()rr(rrr set_epochszRepeatAugSampler.set_epoch)NNTr$r%r)rr r!r"rrrr9rrrrr#6s  $r#) rr2Ztorch.utils.datarZtorch.distributedZ distributedrrr#rrrrs   /