a d) @sddlZddlZddlmZmZmZmZmZddlm Z ddl m Z m Z ej eeefeedfddd Zeejjeejjd d d Zeejjeejjd d dZejjdddZdejjeeeeeeeed ddZejjdddZdS)N)TypeDictAnyTupleCallable)merge)isinstance_strinit_generator.)x tome_inforeturnc Cs|d\}}||}ttt||jd}|d}||dkrtt||}tt||}t|jd|d} |ddurt|j|d<n&|dj|jkrt|j|dd|d<|jdd dkrd n|d } tj||||d |d | | |dd\} } ntj tj } } |dr0| | fn tj tj f\} }|drR| | fn tj tj f\}}|drt| | fn tj tj f\}}| |||||fS)Nsizerargsmax_downsampleratio generator)fallbackrFuse_randsxsy)Zno_randr merge_attnmerge_crossattn merge_mlp) intmathceilsqrtshaper ZdevicerZ bipartite_soft_matching_random2dZ do_nothing)r r Z original_hZ original_wZoriginal_tokensZ downsamplerwhrrmum_au_am_cu_cm_mu_mr+U/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/tomesd/patch.py compute_merge s*   """r-) block_classr csGfddd}|S)z Make a patched class on the fly so we don't have to import any specific modules. This patch applies ToMe to the forward function of the block. cs*eZdZZdejejejdddZdS)z"make_tome_block..ToMeBlockN)r contextr c Sst||j\}}}}}}||j||||jr4|ndd|}||j||||d|}||||||}|S)N)r/) r- _tome_infoattn1norm1disable_self_attnattn2norm2ffnorm3) selfr r/r%r'r)r&r(r*r+r+r,_forward:s * z+make_tome_block..ToMeBlock._forward)N)__name__ __module__ __qualname___parenttorchTensorr9r+r.r+r, ToMeBlock6srAr+r.rAr+r@r,make_tome_block0srCcsGfddd}|S)zw Make a patched class for a diffusers model. This patch applies ToMe to the forward function of the block. cs"eZdZZdejdddZdS)z,make_diffusers_tome_block..ToMeBlockN)r cSs|t||j\}} } } } } |jr,|||}n0|jrR|j||||jd\}}}}}n ||}||}|durp|ni}|j|f|jr|nd|d|}|jr|d|}| ||}|j dur |jr| ||n| |}| |}|j |f||d|}| ||}| |}|jrD|d|dddf|dddf}| |}| |}|jrl|d|}| ||}|S)N)Z hidden_dtype)encoder_hidden_statesattention_maskr) r-r0use_ada_layer_normr2use_ada_layer_norm_zeroZdtyper1Zonly_cross_attentionZ unsqueezer4r5r7r6)r8Z hidden_statesrErDZencoder_attention_maskZtimestepZcross_attention_kwargsZ class_labelsr%r'r)r&r(r*Znorm_hidden_statesZgate_msaZ shift_mlpZ scale_mlpZgate_mlpZ attn_outputZ ff_outputr+r+r,forwardTsV        (  z4make_diffusers_tome_block..ToMeBlock.forward)NNNNNN)r:r;r<r=r>r?rHr+r@r+r,rAPs rAr+rBr+r@r,make_diffusers_tome_blockKsOrI)modelcCs"dd}|jd||dS)z\ Adds a forward pre hook to get the image size. This hook can be removed with remove_patch. cSs&|djd|djdf|jd<dS)Nrrr)rr0)modulerr+r+r,hooks"zhook_tome_model..hookhooksN)r0appendZregister_forward_pre_hook)rJrMr+r+r,hook_tome_modelsrP?rTF) rJrrrrrrrrc  Cst|t|dpt|d} | sHt|dr6t|jds>td|jj} nt|drX|jn|} dg|||||d|||d d | _t| | D]`\} } t| d r| rt nt } | | j | _ | j| _t| d s| sd | _ t| d s| rd | _d | _q|S)af Patches a stable diffusion model with ToMe. Apply this to the highest level stable diffusion object (i.e., it should have a .model.diffusion_model). Important Args: - model: A top level Stable Diffusion module to patch in place. Should have a ".model.diffusion_model" - ratio: The ratio of tokens to merge. I.e., 0.4 would reduce the total number of tokens by 40%. The maximum value for this is 1-(1/(sx*sy)). By default, the max is 0.75 (I recommend <= 0.5 though). Higher values result in more speed-up, but with more visual quality loss. Args to tinker with if you want: - max_downsample [1, 2, 4, or 8]: Apply ToMe to layers with at most this amount of downsampling. E.g., 1 only applies to layers with no downsampling (4/15) while 8 applies to all layers (15/15). I recommend a value of 1 or 2. - sx, sy: The stride for computing dst sets (see paper). A higher stride means you can merge more tokens, but the default of (2, 2) works well in most cases. Doesn't have to divide image size. - use_rand: Whether or not to allow random perturbations when computing dst sets (see paper). Usually you'd want to leave this on, but if you're having weird artifacts try turning this off. - merge_attn: Whether or not to merge tokens for attention (recommended). - merge_crossattn: Whether or not to merge tokens for cross attention (not recommended). - merge_mlp: Whether or not to merge tokens for the mlp layers (very not recommended). ZDiffusionPipelineZ ModelMixinrJdiffusion_modelzPProvided model was not a Stable Diffusion / Latent Diffusion model, as expected.unetN) rrrrrrrrr)rrNrZBasicTransformerBlockr3FrG) remove_patchr hasattrrJ RuntimeErrorrRrSr0rP named_modulesrIrC __class__r3rFrG)rJrrrrrrrrZ is_diffusersrR_rLZmake_tome_block_fnr+r+r, apply_patchs>!    rZcCsnt|dr|jn|}|D]L\}}t|drT|jdD] }|q8|jd|jjdkr|j|_q|S)zI Removes a patch from a ToMe Diffusion module if it was already patched. rSr0rNrA) rUrSrWr0removeclearrXr:r=)rJrYrLrMr+r+r,rT s    rT)rQrrrTTFF)r>rtypingrrrrrrutilsr r r?strr-nnModulerCrIrPfloatrboolrZrTr+r+r+r,s4 $&  [ T