a d+@sUdZddlZddlmZmZmZmZmZmZddlZddlm Z ddl ZgdZ ee e d<ee Zee efZeegeeee ffZGdddZGd d d Zdd d d ZeeedddZeedddZdS)zManipulation of micro-batches.N)AnyCallableListUnioncastSequence)Tensor)NoChunkBatchcheckscattergather__all__c@s*eZdZdZedddZeddZdS)r a Wrapper for a Tensor in :meth:`Pipe.forward` indicating that the tensor should not be chunked on the batch dimension and instead be replicated as-is across all micro-batches. This is useful for tensors which might not have any 'batch' semantics for the model. )inpcCs"t|std|||_dS)Nz+NoChunk only supported for tensors, found: )torch is_tensor TypeError_tensor)selfrrs/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/torch/distributed/pipeline/sync/microbatch.py__init__s zNoChunk.__init__cCs|jSN)rrrrrtensor#szNoChunk.tensorN)__name__ __module__ __qualname____doc__rrpropertyrrrrrr sr c@seZdZdZeeeefddddZe edddZ e d d Z d d Z d dZ eddddZedddZddZedddZedddZejeeddddZejeeddddZeeefddd dZeddd!d"Zeddd#d$ZdS)%r zC An abstraction representing a microbatch in the pipeline. N)valuesreturncCs@||_t||_|js3z!Batch.__init__..zNo tensors found in batch: )_valuesrratomicanyr)rr rrrr-s  zBatch.__init__r!cCs|jstdtt|jS)z Retrieves the underlying tensor.znot atomic batch)r(AttributeErrorrrr'rrrrr6sz Batch.tensorcCs|jS)z-Retreives the underlying values for the batch)r'rrrrr =sz Batch.valuescCs<|jr dSt|jD]\}}t|r|SqtddS)z< Retrieves the index of first tensor found. rzNo tensor found!N)r( enumerater'rrr)rir$rrrfind_tensor_idxBs   zBatch.find_tensor_idxcCs2|jr|jjS|jD]}t|r|jSqdS)z; Retrieves the device for this microbatch. N)r(r'devicerr)rr$rrr get_deviceNs   zBatch.get_device)functionr!cCs&|jrt||jSt||jSdS)zbCalls a function on the microbatch. It also wraps the output with :class:`Batch`. N)r(r r')rr1rrrcallYsz Batch.callcCsd|jd|jdS)Nz Batch[atomic=z]()r(r'rrrr__repr__bszBatch.__repr__ccs |jr|jVn |jEdHdSrr4rrrr__iter__es zBatch.__iter__cCs|jr dSt|jS)N)r(lenr'rrrr__len__ksz Batch.__len__)indexcCs&|js|j|S|dkr td|jS)Nr atomic batch allows index 0 onlyr(r' IndexError)rr:rrr __getitem__ns  zBatch.__getitem__)r:r$r!cCsdSrrrr:r$rrr __setitem__xszBatch.__setitem__cCsdSrrr?rrrr@|s)r:r!cCs(t|tr|||n |||dSr) isinstanceint_setitem_by_index_setitem_by_slicer?rrrr@s cCsP|js6|}|jd||f|j|dd|_dS|dkrFtd||_dS)Nr7rr;r<)rr:r$r-rrrrCs(zBatch._setitem_by_indexcCs`|j|jur$|jur$dus.ntd|js>||_dSt|dkrRtd|d|_dS)Nzonly slice [:] supportedr7z5atomic batch cannot be replaced with multiple tensorsr)startstopstepNotImplementedErrorr(r'r8r=r?rrrrDs& zBatch._setitem_by_slice)rrrrrrrrrrrr r.r0Functionr2strr5r6rBr9r>typingoverloadr@sliceTensorsrCrDrrrrr (s(       r r*csBtdd|Ds td|tfdd|Dr>tddS)z Checks whether the input contains at least one tensor and each tensor is on the same device as the first partition. Raises: ValueError: input does not contain at least one tensor css|]}t|VqdSrr"r#inputrrrr%r&zcheck..z inputs do not have any tensors: c3s"|]}t|o|jkVqdSr)rrr/rO first_devicerrr%r&z>All inputs should be on the same device as the first partitionN)r)r ValueError)rRinputsrrQrr s r )chunksr!cGst|dkr2t|dtr2dd|d|DSddt|D}d}|D]}t|r||}|dkr|t|krtd|dt|t|}t|D]\}}|| |qqLt|D].}t|t r|| |j q|| |qqL|d |}d d|DS) z7Splits an input mini-batch into multiple micro-batches.r7rcSsg|] }t|qSrr r#xrrr r&zscatter..cSsg|]}gqSrr)r#_rrrrYr&z6Found different number of chunks produced for inputs:  and NcSsg|] }t|qSrrVrWrrrrYr&) r8rArchunkrangerr RuntimeErrorr,appendr r)rUrTZbatchesZ num_chunksrPtensorsr-rrrrr s$     r )outputsc Cs|djr(tdd|D}t|}ng}tt|dD]}t|d|}g}|D]>}|t||krtd|dt|||||qXt |d|r|t|q<||q.z2Types for microbatch outputs do not match, found: r\) r(tuplercatr^r8typerr`r)rbraoutputZ output_bufr- output_typeZcurrent_outputsbatchrrrr s    r )rrKrrrrrrrrZtorch.cuda.commrrJ__annotations__rNZTensorOrTensorsrIr r r rBr r rrrrs   w#