a d@sUdZddlmZmZddlZddlmZddlmZgdZee e d<eeeefd d d Z Gd d d ej j ZeeedddZGdddej j ZdS)z0Arbitrary dependency between two autograd lanes.)ListTupleN)Tensor) get_phony)forkForkjoinJoin__all__)inputreturncCs4tr|jrt|\}}nt|jdd}||fS)z7Branches out from an autograd lane of the given tensor.F requires_grad)torchis_grad_enabledrrapplyrdevicer phonyrs/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/torch/distributed/pipeline/sync/dependency.pyrsrc@sBeZdZedeeeefdddZedeeedddZdS)r)ctxr r cCst|jdd}||fS)NFr)rrdetachrr rrrrforwardsz Fork.forward)r grad_input grad_gradr cCs|SNr)rrrrrrbackward"sz Fork.backwardN)__name__ __module__ __qualname__ staticmethodrrrrrrrrrsr)r rr cCs$tr |js|jr t||}|S)zMerges two autograd lanes.)rrrr rrrrrr 's r c@sBeZdZedeeedddZedeeedfdddZdS)r )rr rr cCs|Sr)rrrrrr0sz Join.forwardN)rrr cCs|dfSrr)rrrrrr4sz Join.backward)r r!r"r#rrrrrrrrr /sr )__doc__typingrrrrrrr str__annotations__rZautogradZFunctionrr r rrrrs