a d @s2ddlZddlZddlmZddlmZmZmZmZm Z m Z m Z m Z ddl Z ddlmZddlmZddlmZddlmZdd Zd d Zee e jee e e eefed d dZe e je jedddZe e jedddZe je jedddZdee eddddZ!e je j"j#ddddZ$dS) N) OrderedDict)AnyCallablecastDictListSetTupleUnion) _BatchNorm)_is_namedtuple)PackedSequence) no_dispatchcCstdd|DS)Ncss|]}t|tVqdSN) isinstancer ).0modrf/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/torch/distributed/fsdp/_utils.py z&_contains_batchnorm..)anymodules)modulerrr_contains_batchnormsrcCs&|D]}t|trddi|_qdS)NZmixed_precision)rrr Z_wrap_overrides)rrrrr#_override_batchnorm_mixed_precisions  r)fn containerreturncs2ttjttttttft dfdd |S)zFRecursively apply to all tensor in different kinds of container types.)xrcst|r|St|drVt|}t|D] }|j}t||t||q0|St |t r| }| D]\}}|||<qp|St |t r|j|St |tr‡fdd| DSt|rfdd|D}t||St |tttfrt|fdd|DS|SdS)NZ__dataclass_fields__csi|]\}}||qSrr)rkeyvalueapplyrr 3rz4_apply_to_tensors..apply..c3s|]}|VqdSrrrelr"rrr5rz3_apply_to_tensors..apply..c3s|]}|VqdSrrr%r"rrr8r)torchZ is_tensorhasattr dataclassesreplacefieldsnamesetattrgetattrrr __class__itemsr datadictr typelisttupleset)rdcfr,Zodr r!resr#rrrr#s0        z _apply_to_tensors..apply) r r'Tensorrrr rrr r)rrrr:r_apply_to_tensorssr<)tensorsizercCsP||k}|sJ|}t|dkd|||| S)z Allocate storage for ``tensor`` with the given size. Returns: bool: ``True`` if this method allocated storage and ``False`` if the storage was already allocated. rz8Tensor storage should have been resized to be 0 but got )_typed_storage_sizeZnumelp_assert_resize_)r=r>Zalready_allocatedZtensor_storage_sizerrr_alloc_storage?s  rC)r=rcCsZ|dk}|sTt|dkd|d|d|j|d| S)z Frees the underlying storage of ``tensor``. Returns: bool: ``True`` if the method freed the storage and ``False`` if the storage was already freed. rzVFreeing a tensor's storage is unsafe when it is not the sole occupant storage offset: z storage size: z tensor shape: )r?r@rAZstorage_offsetshaperB)r=Z already_freedrrr _free_storageSs   rE)ryrcCs||kS)z2Returns if ``x`` and ``y`` share the same storage.)r?Z _data_ptr)rrFrrr _same_storageisrGT)condsraise_assertion_errorrcCs$|s t|t|r t|dS)zThis is used as an alternate to ``assert`` when in the backward context to print the error message ``s`` since otherwise, it is swallowed.N)print traceback print_stackAssertionError)rHrIrJrrrrAos rA)r=streamrcCs>t$|ttjj|Wdn1s00YdSr)rZ record_streamrr'Z_CStream)r=rOrrr_no_dispatch_record_streamysrQ)T)%r)rL collectionsrtypingrrrrrrr r r'Ztorch.nn.modules.batchnormr Z torch.nn.parallel.scatter_gatherr Ztorch.nn.utils.rnnr Ztorch.utils._mode_utilsrrrr;r<Zno_gradSizeboolrCrErGstrrAZcudarPrQrrrrs* (     &