a dl@s6UddlZddlZddlZddlmZddlmZmZmZmZm Z m Z m Z ej rXddl ZddlmZddlmZddlmZmZmZe dZe eeegefZeeefZd Zd gZeeed <d Zeee d ddZ!eeedddZ"eeedddZ#ddZ$eeeeeeeeeefeefZ%GdddZ&e&eee%dddZ'e(de'Z)e&eeeddd Z*e(d!eeeed"fe&d#d$d%Z+eeeeed&d'd(Z,ej-eeeed)d*d+Z.ej-eeeeed,d-d+Z.ed.d/d+Z.eeed0d1d2Z/ed.d3d4Z0d5d6Z1d7d8d9d:Z2dS);N) OrderedDict)TupleListDictUnionCallableOptionalTypeVar) EinopsError) get_backend)ParsedExpression _ellipsis AnonymousAxisTensor)minmaxsummeanprod_ellipsis_not_in_parenthesisi)groupreturncCst|dkrdS|ddkS)Nr Frr)len)rrV/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/einops/einops.pyis_ellipsis_not_in_parenthesiss r)sequencercCsd}|D] }||9}q|S)zU minimalistic product that works both with numbers and symbols. Supports empty lists r r)rresultelementrrr_products r!)reduction_type reduced_axescCsbt|}t|r|||St|dkr*|S|tvs6J|dkrP||sPtd||||SdS)Nrrz5reduce_mean is not available for non-floating tensors)tuplecallabler _reductionsZ is_float_typeNotImplementedErrorreduce)tensorr"r#backendrrr _reduce_axes%s    r+c stttksJttttddddD]}|d|dkrB|d}dddd|9<d|dtdd|ddDqBfdd}|}ttddddD]}||durq||ddur0q||d||dkr|d}tfddtD} tfd dDdddd|9<} g| D]6} | | krn$| | kr| n| dq|}q|fS) Nr css|]}|dVqdSr Nr.0axisrrr Bz+_optimize_transformation..csTi}ttD]>}|vr&d||<qtdd|D}t|||<q|S)Ncss|]}|duVqdSNrr/xrrrr1Kr2zB_optimize_transformation..build_mapping..)rangerrvalueslistindex)Z init_to_finalr0Zafter_reduction)axes_reordering init_shapesr#rr build_mappingEs z/_optimize_transformation..build_mappingc3s|]}|vVqdSr4rr5)r#rrr1Yr2c3s"|]}|kr|n|dVqdSr-rr.) removed_axisrrr1[r2)rr$sortedr7rappend) r<r#r; final_shapesiZremoved_lengthr=Zinit_axis_to_final_axisZ init_axisZremoved_axis_after_reductionZold_reorderingr0r)r;r<r#r>r_optimize_transformation5sB  0      rCc @s`eZdZdZdeeeeeeeefeeeeeeefeeeeedddZ dS)TransformRecipezi Recipe describes actual computation pathway. Recipe can be applied to a tensor or variable. Nelementary_axes_lengthsinput_composite_axesreduced_elementary_axesaxes_permutation added_axesoutput_composite_axesellipsis_position_in_lhscCs:||_||_||_||_||_||_|dur0|nd|_dS)N')rFrGrKrIrJrHrL)selfrFrGrHrIrJrKrLrrr__init__wszTransformRecipe.__init__)N) __name__ __module__ __qualname____doc__rintrrrrOrrrrrDos  rD)rNshapercs~t|j|jdkrJt|t|jdkrvtdt|jdt|n,t|t|jkrvtdt|jt|g}t|jD]Z\}\}}|}|t|t|j}||jkrt|dkrt|dksJ|\}|||d}|D]} | durtdqt|} | |<q||jkr,||} n||} d} |D]} | | 9} qShape mismatch, can't divide axis of length {} in chunks of {}csg|] }|qSrr)r/elementary_axis axes_lengthsrr r2z4_reconstruct_from_shape_uncached..csi|]\}}||qSrr)r/posZpos_in_elementaryrWrr sz4_reconstruct_from_shape_uncached..)r9rFrLrrGr format enumerater! isinstancerTrJrKrextendr@rHrIitems)rNrUZellipsis_shapeZ input_axisZ known_axesZ unknown_axesZbefore_ellipsisZafter_ellipsisZ unknown_axisdZtotal_dim_sizelengthZ known_productr0Zinferred_lengthr<rAZ output_axisgroupinglengthsr#r;rJrrWr _reconstruct_from_shape_uncachedsd       "&    rei)reciper)r"rc Cs~t|}t|||\}}}}}|||}t||||d}|||}t|dkrr|j|t|t||d}|||S)N)r"r#r*r)Zn_axesZpos2len)r _reconstruct_from_shaperUreshaper+Z transposerZadd_axes) rfr)r"r*r<r#r;rJrArrr _apply_recipes    ri.)pattern operationrXrc s|d\ttjs8jr8td|jrRjrRtd||dkrtjj}j svj r~tdt |dkrtd|n|dkrt jj}t |dkrtd |t d d jDhjd d |D}t |dkrltd|nT|t vs,t |r\t jj}t |dkrltd||ntd|t tjD]2}|D]&}t|tr|j|<nt|<qqxg}jD]:}|vrt|tr|j|<nt|<||qddtDfddD}tt|}|D]@\} } t| sHtd| | vr`td| | | <q*g} jD]}fdd |D} fdd |D} t | dkrtd| t | t | t |ksJ| fdd| Dfdd| DfqvitjjD]}|jvrt |<qg}jD]4}|tkrZ|tn|fdd|Dq@ttjj}fdd|D}fddt|D}tjvrd n jt}tt | |||||d!S)"z Perform initial parsing of pattern and provided supplementary info axes_lengths is a tuple of tuples (axis_name, axis_length) z->z?Ellipsis found in left side, but not right side of a pattern {}z;Ellipsis is parenthesis in the left side is not allowed: {} rearrangezQNon-unitary anonymous axes are not supported in rearrange (exception is length 1)rzBIdentifiers only on one side of expression (should be on both): {}repeatz5Unexpected identifiers on the left side of repeat: {}cSsh|]}t|ts|qSr)r^r)r/axrrr r2z1_prepare_transformation_recipe..css|]\}}|VqdSr4r)r/ro_rrrr1 r2z1_prepare_transformation_recipe..z(Specify sizes for new axes in repeat: {}z9Unexpected identifiers on the right side of reduce {}: {}z'Unknown reduction {}. Expect one of {}.cSsi|]\}}||qSrr)r/positionnamerrrr[+r2z2_prepare_transformation_recipe..csg|]\}}|jvr|qSr identifiers)r/r0rr)rghtrrrY,s  z2_prepare_transformation_recipe..zInvalid name for an axisz Axis {} is not used in transformcsh|]}|tkr|qSr_unknown_axis_lengthr.axis_name2known_lengthrrrp:r2csh|]}|tkr|qSrrwr.ryrrrp;r2r zCould not infer sizes for {}csg|] }|qSrrr.axis_name2positionrrrY@r2csg|] }|qSrrr.r{rrrYAr2csg|] }|qSrrr.r{rrrYNr2csg|]}|jvr|qSrrtr.)axis_position_after_reductionleftrrrYQscs$i|]\}}|jvr||qSrrt)r/rB axis_name)r|r~rrr[Ss  NrE)!splitr has_ellipsisr r\Zhas_ellipsis_parenthesizedsetsymmetric_differenceruZhas_non_unitary_anonymous_axesr differencer&r%r compositionr^rvaluerxr@r]r`r9r?Zcheck_axis_name itertoolschainrrr:rDr8)rkrlrXrZaxes_without_sizeZcomposite_axisrZrepeat_axes_namesr#rV axis_lengthZinput_axes_known_unknownZknownunknownZresult_axes_groupingZordered_axis_rightrIrJZ ellipsis_leftr)rzr|r}r~rvr_prepare_transformation_recipes                          r)r)rk reductionrXrc Ksz.tt|}t|||d}t|||dWSty}zdd||}t|tsn|dt | |7}n|d7}|d|7}t|d|WYd}~n d}~00dS) a einops.reduce provides combination of reordering and reduction using reader-friendly notation. Examples for reduce operation: ```python >>> x = np.random.randn(100, 32, 64) # perform max-reduction on the first axis >>> y = reduce(x, 't b c -> b c', 'max') # same as previous, but with clearer axes meaning >>> y = reduce(x, 'time batch channel -> batch channel', 'max') >>> x = np.random.randn(10, 20, 30, 40) # 2d max-pooling with kernel size = 2 * 2 for image processing >>> y1 = reduce(x, 'b c (h1 h2) (w1 w2) -> b c h1 w1', 'max', h2=2, w2=2) # if one wants to go back to the original height and width, depth-to-space trick can be applied >>> y2 = rearrange(y1, 'b (c h2 w2) h1 w1 -> b c (h1 h2) (w1 w2)', h2=2, w2=2) >>> assert parse_shape(x, 'b _ h w') == parse_shape(y2, 'b _ h w') # Adaptive 2d max-pooling to 3 * 4 grid >>> reduce(x, 'b c (h1 h2) (w1 w2) -> b c h1 w1', 'max', h1=3, w1=4).shape (10, 20, 3, 4) # Global average pooling >>> reduce(x, 'b c h w -> b c', 'mean').shape (10, 20) # Subtracting mean over batch for each channel >>> y = x - reduce(x, 'b c h w -> () c () ()', 'mean') # Subtracting per-image mean for each channel >>> y = x - reduce(x, 'b c h w -> b c () ()', 'mean') ``` Parameters: tensor: tensor: tensor of any supported library (e.g. numpy.ndarray, tensorflow, pytorch, mxnet.ndarray). list of tensors is also accepted, those should be of the same type and shape pattern: string, reduction pattern reduction: one of available reductions ('min', 'max', 'sum', 'mean', 'prod'), case-sensitive alternatively, a callable f(tensor, reduced_axes) -> tensor can be provided. This allows using various reductions, examples: np.max, tf.reduce_logsumexp, torch.var, etc. axes_lengths: any additional specifications for dimensions Returns: tensor of the same type as input rW)r"z2 Error while processing {}-reduction pattern "{}".z Input tensor shape: {}. z Input is list. zAdditional info: {}.z {}N) r$r?r`rrir r\r^r9r rU)r)rkrrXZhashable_axes_lengthsrfemessagerrrr(cs4  r()r)rk axes_lengthrcKsdSr4r)r)rkrrrrrmsrm)r)rkrXrcKsdSr4rr)rkrXrrrrms)rkcKsFt|tr0t|dkrtdt|d|}t||fddi|S)a( einops.rearrange is a reader-friendly smart element reordering for multidimensional tensors. This operation includes functionality of transpose (axes permutation), reshape (view), squeeze, unsqueeze, stack, concatenate and other operations. Examples for rearrange operation: ```python # suppose we have a set of 32 images in "h w c" format (height-width-channel) >>> images = [np.random.randn(30, 40, 3) for _ in range(32)] # stack along first (batch) axis, output is a single array >>> rearrange(images, 'b h w c -> b h w c').shape (32, 30, 40, 3) # concatenate images along height (vertical axis), 960 = 32 * 30 >>> rearrange(images, 'b h w c -> (b h) w c').shape (960, 40, 3) # concatenated images along horizontal axis, 1280 = 32 * 40 >>> rearrange(images, 'b h w c -> h (b w) c').shape (30, 1280, 3) # reordered axes to "b c h w" format for deep learning >>> rearrange(images, 'b h w c -> b c h w').shape (32, 3, 30, 40) # flattened each image into a vector, 3600 = 30 * 40 * 3 >>> rearrange(images, 'b h w c -> b (c h w)').shape (32, 3600) # split each image into 4 smaller (top-left, top-right, bottom-left, bottom-right), 128 = 32 * 2 * 2 >>> rearrange(images, 'b (h1 h) (w1 w) c -> (b h1 w1) h w c', h1=2, w1=2).shape (128, 15, 20, 3) # space-to-depth operation >>> rearrange(images, 'b (h h1) (w w1) c -> b h w (c h1 w1)', h1=2, w1=2).shape (32, 15, 20, 12) ``` When composing axes, C-order enumeration used (consecutive elements have different last axis) Find more examples in einops tutorial. Parameters: tensor: tensor of any supported library (e.g. numpy.ndarray, tensorflow, pytorch, mxnet.ndarray). list of tensors is also accepted, those should be of the same type and shape pattern: string, rearrangement pattern axes_lengths: any additional specifications for dimensions Returns: tensor of the same type as input. If possible, a view to the original tensor is returned. rz+Rearrange can't be applied to an empty listrrm)r^r9r TypeErrorr Zstack_on_zeroth_dimensionr(rrrrrms 7  )r)rkrcKst||fddi|S)a einops.repeat allows reordering elements and repeating them in arbitrary combinations. This operation includes functionality of repeat, tile, broadcast functions. Examples for repeat operation: ```python # a grayscale image (of shape height x width) >>> image = np.random.randn(30, 40) # change it to RGB format by repeating in each channel >>> repeat(image, 'h w -> h w c', c=3).shape (30, 40, 3) # repeat image 2 times along height (vertical axis) >>> repeat(image, 'h w -> (repeat h) w', repeat=2).shape (60, 40) # repeat image 2 time along height and 3 times along width >>> repeat(image, 'h w -> h (repeat w)', repeat=3).shape (30, 120) # convert each pixel to a small square 2x2. Upsample image by 2x >>> repeat(image, 'h w -> (h h2) (w w2)', h2=2, w2=2).shape (60, 80) # pixelate image first by downsampling by 2x, then upsampling >>> downsampled = reduce(image, '(h h2) (w w2) -> h w', 'mean', h2=2, w2=2) >>> repeat(downsampled, 'h w -> (h h2) (w w2)', h2=2, w2=2).shape (30, 40) ``` When composing axes, C-order enumeration used (consecutive elements have different last axis) Find more examples in einops tutorial. Parameters: tensor: tensor of any supported library (e.g. numpy.ndarray, tensorflow, pytorch, mxnet.ndarray). list of tensors is also accepted, those should be of the same type and shape pattern: string, rearrangement pattern axes_lengths: any additional specifications for dimensions Returns: Tensor of the same type as input. If possible, a view to the original tensor is returned. rrn)r(rrrrrns/rnc Cs t|dd}t||}|r4tdj||dt|t|jkr|jrvt|t|jdkrtdj||dntdj||d|jr|j t }|jd|d gt|t|jd|j|dd}n|j}i}t ||D]\\}}|d kr|||<q|S) a Parse a tensor shape to dictionary mapping axes names to their lengths. ```python # Use underscore to skip the dimension in parsing. >>> x = np.zeros([2, 3, 5, 7]) >>> parse_shape(x, 'batch _ h w') {'batch': 2, 'h': 5, 'w': 7} # `parse_shape` output can be used to specify axes_lengths for other operations: >>> y = np.zeros([700]) >>> rearrange(y, '(b c h w) -> b c h w', **parse_shape(x, 'b _ h w')).shape (2, 10, 5, 7) ``` For symbolic frameworks may return symbols, not integers. Parameters: x: tensor of any of supported frameworks pattern: str, space separated names for axes, underscore means skip axis Returns: dict, maps axes names to their lengths T)Zallow_underscorez8Can't parse shape with composite axes: {pattern} {shape})rkrUr zCCan't parse shape with this number of dimensions: {pattern} {shape}zHCan't parse shape with different number of dimensions: {pattern} {shape}Nrq) r r rUZhas_composed_axes RuntimeErrorr\rrrr:rzip) r6rkexprUZ ellipsis_idxrrrrrrr parse_shapes8     rc Cs\t|}||}g}t|D]8\}}dgt|}|||<|||d||q|S)aa For an n-dimensional tensor, returns tensors to enumerate each axis. ```python x = np.zeros([2, 3, 4]) # or any other tensor i, j, k = _enumerate_directions(x) result = i + 2*j + 3*k ``` `result[i, j, k] = i + 2j + 3k`, and also has the same shape as result Works very similarly to numpy.ogrid (open indexing grid) r r)r rUr]rr@rhZarange)r6r*rUrZaxis_idrrrr_enumerate_directionsRs  rz numpy.ndarray)rcCst||S)z Convert a tensor of an imperative framework (i.e. numpy/cupy/torch/gluon/etc.) to `numpy.ndarray` Parameters: tensor: tensor of any of known imperative framework Returns: `numpy.ndarray`, converted to numpy )r Zto_numpy)r)rrrasnumpyhs r)3 functoolsrtyping collectionsrrrrrrrr TYPE_CHECKINGnumpynpr Z _backendsr Zparsingr rrrrTZReductionCallablestrZ Reductionr&r__annotations__rxboolrr!r+rCZ CookedReciperDre lru_cachergrirr(overloadrmrnrrrrrrrsN  $   7*'L  pC>26