a d@sdZddlZddlZddlZddlZddlZddlmZddlmZddl m Z ddl m Z ddl m Z ejZd d d Zd d ZdS)aPSimple Python Parser Parse Python code into a list of logical lines, represented by LogicalLine objects. This uses Python's tokenizer to generate the tokens. As such, YAPF must be run with the appropriate Python version---Python >=3.7 for Python 3.7 code, Python >=3.8 for Python 3.8 code, etc. This parser uses Python's native "tokenizer" module to generate a list of tokens for the source code. It then uses Python's native "ast" module to assign subtypes, calculate split penalties, etc. A "logical line" produced by Python's "tokenizer" module ends with a tokenize.NEWLINE, rather than a tokenize.NL, making it easy to separate them out. Comments all end with a tokentizer.NL, so we need to make sure we don't errantly pick up non-comment tokens when parsing comment blocks. ParseCode(): parse the code producing a list of logical lines. N)StringIO) TokenInfo)split_penalty_visitor) format_token) logical_line cCsv|tjs|tj7}z.t||}t|t|j}t |}Wnt yXYn0t |}t |||S)acParse a string of Python code into logical lines. This provides an alternative entry point to YAPF. Arguments: unformatted_source: (unicode) The code to format. filename: (unicode) The name of the file being reformatted. Returns: A list of LogicalLines. Raises: An exception is raised if there's an error during AST parsing. )endswithoslinesepastparsefix_missing_locationsrreadlinetokenizegenerate_tokens Exception_CreateLogicalLinesrZ SplitPenaltyvisit)Zunformatted_sourcefilenameZast_treertokens logical_linesr_/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/yapf/pyparser/pyparser.py ParseCode2s      rc CsJg}d}|D]}t|}|r|jdr|jd|jdkrttd|jd|jddf|jd|jddf|jd}|jd|_|jd|_d|_ | t |d|jd|_|jd|_|j |_ | t |tj|j|}q gg}}d}|D]}|jtjkrq|jtjkr<| t||g}n|jtjkrT|d7}nn|jtjkrl|d8}nV|jtjkr|nF|r|jtjks|djtjkr| t||g}| |q|D]z}|j} | r| gng} |jddD]J}| |_|| _|} |r | |n|r|| d_ | !|_ qq|S) zSeparate tokens into logical lines. Arguments: tokens: (list of tokenizer.TokenInfo) Tokens generated by tokenizer. Returns: A list of LogicalLines. N\r)typestringstartendline CONTINUATION)"rr!rstriprrr"r linenocolumnvalueappendrZ FormatTokenrtokentok_namerr ENDMARKERNEWLINErZ LogicalLineINDENTDEDENTNLCOMMENTfirstZ OpensScoperZprevious_tokenZ next_tokenZ ClosesScopeZmatching_bracketpop) rZformatted_tokensZprev_toktokZctokrZcur_logical_linedepthr!previousZ bracket_stackrrrrTst            r)r)__doc__r codecsr r)riorrZ yapf.pyparserrZ yapf.yapflibrrN_TOKENSr"rrrrrrs      "