a d@sXdZddlZddlmZeeedddZeeddd Zd eeeed d d ZdS)aV A collection of utilities for comparing `examples/complete_*_example.py` scripts with the capabilities inside of each `examples/by_feature` example. `compare_against_test` is the main function that should be used when testing, while the others are used to either get the code that matters, or to preprocess them (such as stripping comments) N)List)linesnamecCs|dkr |dkr td|dgd}}|D]d}|sTd||vrTd}||q.|r.|dkrpd|vrp|S|dkrd |vr|S||q.d S) a. Extracts a function from `lines` of segmented source code with the name `name`. Args: lines (`List[str]`): Source code of a script seperated by line. name (`str`): The name of the function to extract. Should be either `training_function` or `main` training_functionmainz Incorrect function name passed: z-, choose either 'main' or 'training_function'Fzdef Tzdef mainz if __name__N) ValueErrorappend)rrZ good_linesZ found_startliner g/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/accelerate/test_utils/examples.pyget_function_contents_by_names   r rcCsdd|DS)z Filters `lines` and removes any entries that start with a comment ('#') or is just a newline (' ') Args: lines (`List[str]`): Source code of a script seperated by line. cSs&g|]}|ds|dkr|qS)# )lstrip startswith.0r r r r <zclean_lines..r r r r r clean_lines4sr) base_filenamefeature_filename parser_onlysecondary_filenamecst|d}|}Wdn1s(0Yttjtjddd}|}Wdn1sl0Yt|d}|}Wdn1s0Y|durt|d}|}Wdn1s0Y|r,tt|d} tt|d} tt|d} |durntt|dnBtt|d} tt|d} tt|d} |durntt|dd} g} g}t| }t t | dD]X}||vrt |}|| vr| | krd |vr| || |nt |}qgg}t| D]J\}}||vr|| vr| | krd |vr || |qfd d | D}|durfd d |Dfd d |D}|S)a Tests whether the additional code inside of `feature_filename` was implemented in `base_filename`. This should be used when testing to see if `complete_*_.py` examples have all of the implementations from each of the `examples/by_feature/*` scripts. It utilizes `nlp_example.py` to extract out all of the repeated training code, so that only the new additional code is examined and checked. If something *other* than `nlp_example.py` should be used, such as `cv_example.py` for the `complete_cv_example.py` script, it should be passed in for the `secondary_filename` parameter. Args: base_filename (`str` or `os.PathLike`): The filepath of a single "complete" example script to test, such as `examples/complete_cv_example.py` feature_filename (`str` or `os.PathLike`): The filepath of a single feature example script. The contents of this script are checked to see if they exist in `base_filename` parser_only (`bool`): Whether to compare only the `main()` sections in both files, or to compare the contents of `training_loop()` secondary_filename (`str`, *optional*): A potential secondary filepath that should be included in the check. This function extracts the base functionalities off of "examples/nlp_example.py", so if `base_filename` is a script other than `complete_nlp_example.py`, the template script should be included here. Such as `examples/cv_example.py` rNZexamplesznlp_example.pyrrzMtrain_dataloader, eval_dataloader = get_dataloaders(accelerator, batch_size) ZTESTING_MOCKED_DATALOADERScsg|]}|vr|qSr r r)new_full_example_partsr r rrz(compare_against_test..csg|]}|vr|qSr r r)secondary_file_funcr r rrcsg|]}|vr|qSr r r) diff_from_twor r rr)open readlinesospathabspathjoinrr iterrangelennextrr enumerate)rrrrfZbase_file_contentsZfull_file_contentsZfeature_file_contentsZsecondary_file_contentsZbase_file_funcZfull_file_funcZfeature_file_funcZ_dl_lineZnew_feature_codeZ passed_idxsitir _Zdiff_from_exampler )rrrr compare_against_test?s^ && & &           r/)N) __doc__r"typingrstrr rboolr/r r r r s