a  de@sHddlmZddlmZmZmZmZmZmZddl Z ddl m Z m Z ddl m Z e dZedZeeeeee feee feee feee feee feee ffdd d Zeeeed d d ZeeedddZeeedddZeeeeeeeedfe ffdddZeeeeeeeeeeedfe ffeeeeedfe ffeee feee ffdddZeeeeedfe ffeeeeedfe ffeee fdddZeee feee feee fddd Zeee feee feee feee feee feee feee d! d"d#Zeeeeeeedfe ffeeeeedfe ffeee feee feeeeeeee eee feee feee feee ffd$ d%d&Zd4eeeefeeeeeefeee feee feee feee feee feee feeeeeeeee eeee feee feee feee feee feee feee fd'd(d)Zeee feee feee feee feee feee feee d* d+d,Z d5eeeefeeeeefeeeeeeee ee e ffd1 d2d3Z!dS)6 defaultdict)DictListOptionalSequenceTupleUnionN)Tensortensor)_validate_inputsgؗҜ?@[\]^_`{|}~) n_char_order n_word_orderreturncCs|ddt|D}ddt|D}ddt|D}ddt|D}ddt|D}ddt|D}||||||fS)aiPrepare dictionaries with default zero values for total ref, hypothesis and matching chraracter and word n-grams. Args: n_char_order: A character n-gram order. n_word_order: A word n-gram order. Return: Dictionaries with default zero values for total reference, hypothesis and matching character and word n-grams. cSsi|]}|dtdqSr .0nrj/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/torchmetrics/functional/text/chrf.py @z*_prepare_n_grams_dicts..cSsi|]}|dtdqSrrrrrrrArcSsi|]}|dtdqSrrrrrrrBrcSsi|]}|dtdqSrrrrrrrCrcSsi|]}|dtdqSrrrrrrrDrcSsi|]}|dtdqSrrrrrrrEr)range)r rtotal_preds_char_n_gramstotal_preds_word_n_gramstotal_target_char_n_gramstotal_target_word_n_gramstotal_matching_char_n_gramstotal_matching_word_n_gramsrrr_prepare_n_grams_dicts0sr")sentence whitespacercCs |r t|St|ddS)aSplit sentence into individual characters. Args: sentence: An input sentence to split. whitespace: An indication whether to keep whitespaces during character n-gram extraction. Return: A list of separated characters.  )liststripreplace)r#r$rrr_get_charactersQs r*)wordrcCsXt|dkr|gS|dtvr2|dd|dgS|dtvrR|d|ddgS|gS)aySeparates out punctuations from beginning and end of words for chrF. Adapted from https://github.com/m-popovic/chrF and https://github.com/mjpost/sacrebleu/blob/master/sacrebleu/metrics/chrf.py. Args: word: An input word to be separated from a punctuation if present. Return: A list of a single word or a separated word and punctuation. rNr)len _PUNCTUATIONS)r+rrr_separate_word_and_punctiationas   r/)r#rcCstdd|DgS)zSeparates out punctuations from beginning and end of words for chrF for all words in the sentence. Args: sentence: An input sentence to split Return: An aggregated list of separated words and punctuations. css|]}t|VqdSN)r/)rr+rrr rz-_get_words_and_punctiation..)sumr(split)r#rrr_get_words_and_punctiationxs r4.)char_or_word_list n_gram_orderrcsftdd}td|dD]FfddttdDD]}||td7<qBq|S)zCalculate n-gram counts. Args: char_or_word_list: A list of characters of words n_gram_order: The largest number of n-gram. Return: A dictionary of dictionaries with a counts of given n-grams. cSs tddS)NcSstdSNrrrrrrrz1_ngram_counts....rrrrrr8rz_ngram_counts..rc3s"|]}t||VqdSr0)tuple)rir5rrrr1rz _ngram_counts..)rrr-r )r5r6ZngramsZngramrr;r _ngram_countss (r<)r#r r lowercaser$rc sttttttttttdftfftttttdftfffdfdd }tttttdftfftttfddd}|||||\}}||} ||} ||| | fS)aGet n-grams and total n-grams. Args: sentence: An input sentence n_char_order: A character n-gram order. n_word_order: A word n-gram order. lowercase: An indication whether to enable case-insensitivity. whitespace: An indication whether to keep whitespaces during character n-gram extraction. Return: char_n_grams_counts: A dictionary of dictionaries with sentence character n-grams. word_n_grams_counts: A dictionary of dictionaries with sentence word n-grams. total_char_n_grams: A dictionary containing a total number of sentence character n-grams. total_word_n_grams: A dictionary containing a total number of sentence word n-grams. .)r#r rr=rcs2|r |}tt||}tt||}||fS)z@Get a dictionary of dictionaries with a counts of given n-grams.)lowerr<r*r4)r#r rr=char_n_grams_countsword_n_grams_countsr$rr_char_and_word_ngrams_countss zJ_get_n_grams_counts_and_total_ngrams.._char_and_word_ngrams_counts)n_grams_countsrcSs2tdd}|D]}tt||||<q|S)z.Get total sum of n-grams over n-grams w.r.t n.cSstdSr7rrrrrr8rzQ_get_n_grams_counts_and_total_ngrams.._get_total_ngrams..)rr r2values)rC total_n_gramsrrrr_get_total_ngramss z?_get_n_grams_counts_and_total_ngrams.._get_total_ngrams)strintboolrrr ) r#r rr=r$rBrFr?r@Ztotal_char_n_gramsZtotal_word_n_gramsrrAr$_get_n_grams_counts_and_total_ngramss: 0rJ)hyp_n_grams_countsref_n_grams_countsrcs@tdd}D]*ttfddD|<q|S)zGet a number of n-gram matches between reference and hypothesis n-grams. Args: hyp_n_grams_counts: n-grams counts for hypothesis ref_n_grams_counts: n-grams counts for reference Return: matching_n_grams cSstdSr7rrrrrr8rz$_get_ngram_matches..c3s*|]"}t||VqdSr0)torchmin)rZn_gramrKrrLrrr1sz%_get_ngram_matches..)rr r2)rKrLmatching_n_gramsrrOr_get_ngram_matchess  rQ)rEn_gramsrcCs"|D]}||||7<q|S)aBAggregate total n-grams to keep corpus-level statistics. Args: total_n_grams: A dictionary containing a total corpus-level number of n-grams. n_grams: A dictionary containing a sentence-level number of n-grams. Return: A dictionary containing a total corpus-level number of n-grams. r)rErRrrrr_sum_over_dictss rS) matching_char_n_gramsmatching_word_n_gramshyp_char_n_gramshyp_word_n_gramsref_char_n_gramsref_word_n_gramsn_orderbetarc Csrtttftttftttfttttfddd}|||||} |||||} t| t| t|S)aCalculate sentence-level chrF/chrF++ score. For given hypothesis and reference statistics (either sentence-level or corpus-level) the chrF/chrF++ score is returned. Args: matching_char_n_grams: A total number of matching character n-grams between the best matching reference and hypothesis. matching_word_n_grams: A total number of matching word n-grams between the best matching reference and hypothesis. hyp_char_n_grams: A total number of hypothesis character n-grams. hyp_word_n_grams: A total number of hypothesis word n-grams. ref_char_n_grams: A total number of reference character n-grams. ref_word_n_grams: A total number of reference word n-grams. n_order: A sum of character and word n-gram order. beta: A parameter determining an importance of recall w.r.t. precision. If `beta=1`, their importance is equal. Return: A chrF/chrF++ score. This function is universal both for sentence-level and corpus-level calucation. )rP ref_n_grams hyp_n_gramsr[rcsZfddDfddDfddDfddD}|S)zGet n-gram level f-score.cs2i|]*}||dkr&||ntdqSrrrr)r]rPrrrszA_calculate_fscore.._get_n_gram_fscore..cs2i|]*}||dkr&||ntdqSr^rr)rPr\rrrscs.i|]&}|td||tqS))rMmax_EPS_SMOOTHINGr)r[ precisionrecallrrrscs2i|]*}|dd|||qS)rr_rr)r[ denominatorrbrcrrr sr)rPr\r]r[f_scorer)r[rdr]rPrbrcr\r_get_n_gram_fscores  z-_calculate_fscore.._get_n_gram_fscore)rrHr floatr2rDr ) rTrUrVrWrXrYrZr[rfZchar_n_gram_f_scoreZword_n_gram_f_scorerrr_calculate_fscores !  rh) targetspred_char_n_grams_countspred_word_n_grams_countspred_char_n_gramspred_word_n_gramsr rrZr[r=r$rc  Cstd} tdd} tdd} tdd}tdd}|D]b}t|||| | \}}}}t||}t||}t||||||||}|| kr<|} |} |} |}|}q<| | | ||fS)aSCalculate the best sentence-level chrF/chrF++ score. For a given pre-processed hypothesis, all references are evaluated and score and statistics for the best matching reference is returned. Args: targets: An iterable of references. pred_char_n_grams_counts: A dictionary of dictionaries with hypothesis character n-grams. pred_word_n_grams_counts: A dictionary of dictionaries with hypothesis word n-grams. pred_char_n_grams: A total number of hypothesis character n-grams. pred_word_n_grams: A total number of hypothesis word n-grams. n_char_order: A character n-gram order. n_word_order: A word n-gram order. n_order: A sum of character and word n-gram order. beta: A parameter determining an importance of recall w.r.t. precision. If `beta=1`, their importance is equal. lowercase: An indication whether to enable case-insensitivity. whitespace: An indication whether to keep whitespaces during character n-gram extraction. Return: Return chrF/chrF++ score and statistics for the best matching hypothesis and reference. f_score: A sentence-level chrF/chrF++ score. matching_char_n_grams: A total number of matching character n-grams between the best matching reference and hypothesis. matching_word_n_grams: A total number of matching word n-grams between the best matching reference and hypothesis. target_char_n_grams: A total number of reference character n-grams. target_word_n_grams: A total number of reference word n-grams. rcSstdSr7rrrrrr8Xrz6_calculate_sentence_level_chrf_score..cSstdSr7rrrrrr8YrcSstdSr7rrrrrr8ZrcSstdSr7rrrrrr8[r)r rrJrQrh)rirjrkrlrmr rrZr[r=r$Z best_f_scoreZbest_matching_char_n_gramsZbest_matching_word_n_gramsZbest_target_char_n_gramsZbest_target_word_n_gramstargetZtarget_char_n_grams_countsZtarget_word_n_grams_countstarget_char_n_gramstarget_word_n_gramsrTrUrerrr$_calculate_sentence_level_chrf_score,sH+       rq)predsrnrrrrr r!r rrZr[r=r$sentence_chrf_scorerc Cst||\}}t||D]\}}t||| | | \}}}}t||}t||}t||||||| | | | | \}}}}}|dur||dt||}t||}t||}t||}q|||||||fS)aUpdate function for chrf score. Args: preds: An iterable of hypothesis corpus. target: An iterable of iterables of reference corpus. total_preds_char_n_grams: A dictionary containing a total number of hypothesis character n-grams. total_preds_word_n_grams: A dictionary containing a total number of hypothesis word n-grams. total_target_char_n_grams: A dictionary containing a total number of reference character n-grams. total_target_word_n_grams: A dictionary containing a total number of reference word n-grams. total_matching_char_n_grams: A dictionary containing a total number of matching character n-grams between references and hypotheses. total_matching_word_n_grams: A dictionary containing a total number of total matching word n-grams between references and hypotheses. n_char_order: A character n-gram order. n_word_order: A word n-gram order. n_order: Sum of character and word n-gram order. beta: A parameter determining an importance of recall w.r.t. precision. If `beta=1`, their importance is equal. lowercase: An indication whether to enable case-insensitivity. whitespace: An indication whether to keep whitespaces during character n-gram extraction. sentence_chrf_score: A list of sentence-level chrF/chrF++ scores. Return: total_target_char_n_grams: number of reference character n-grams. total_target_word_n_grams: number of reference word n-grams. total_preds_char_n_grams: number of hypothesis character n-grams. total_preds_word_n_grams: number of hypothesis word n-grams. total_matching_char_n_grams: number of matching character n-grams between references and hypotheses. total_matching_word_n_grams: number of total matching word n-grams between references and hypotheses. sentence_chrf_score: A list of sentence-level chrF/chrF++ scores. Raises: ValueError: If length of ``preds`` and ``target`` differs. Nr)r ziprJrSrqappendZ unsqueeze)rrrnrrrrr r!r rrZr[r=r$rsZ target_corpuspredrirjrkrlrmZsentence_level_f_scorerTrUrorprrr_chrf_score_updatesV<      rw) rrrrr r!rZr[rc Cst||||||||S)akCompute chrF/chrF++ score based on pre-computed target, prediction and matching character and word n-grams. Args: total_preds_char_n_grams: number of hypothesis character n-grams. total_preds_word_n_grams: number of hypothesis word n-grams. total_target_char_n_grams: number of reference character n-grams. total_target_word_n_grams: number of reference word n-grams. total_matching_char_n_grams: number of matching character n-grams between references and hypotheses. total_matching_word_n_grams: number of total matching word n-grams between references and hypotheses. n_order: A sum of character and word n-gram order. beta: A parameter determining an importance of recall w.r.t. precision. If `beta=1`, their importance is equal. Return: A corpus-level chrF/chrF++ score. )rh)rrrrr r!rZr[rrr_chrf_score_computesrxr_@F) rrrnr rr[r=r$return_sentence_level_scorercCst|tr|dkrtdt|tr,|dkr4td|dkrDtdt||}t||\} } } } } }|rngnd}t||| | | | | ||||||||\} } } } } }}t| | | | | |||}|r|t|fS|S)u_Calculate `chrF score`_ of machine translated text with one or more references. This implementation supports both chrF score computation introduced in [1] and chrF++ score introduced in `chrF++ score`_. This implementation follows the implmenetaions from https://github.com/m-popovic/chrF and https://github.com/mjpost/sacrebleu/blob/master/sacrebleu/metrics/chrf.py. Args: preds: An iterable of hypothesis corpus. target: An iterable of iterables of reference corpus. n_char_order: A character n-gram order. If `n_char_order=6`, the metrics refers to the official chrF/chrF++. n_word_order: A word n-gram order. If `n_word_order=2`, the metric refers to the official chrF++. If `n_word_order=0`, the metric is equivalent to the original chrF. beta: A parameter determining an importance of recall w.r.t. precision. If `beta=1`, their importance is equal. lowercase: An indication whether to enable case-insesitivity. whitespace: An indication whether to keep whitespaces during character n-gram extraction. return_sentence_level_score: An indication whether a sentence-level chrF/chrF++ score to be returned. Return: A corpus-level chrF/chrF++ score. (Optionally) A list of sentence-level chrF/chrF++ scores if `return_sentence_level_score=True`. Raises: ValueError: If ``n_char_order`` is not an integer greater than or equal to 1. ValueError: If ``n_word_order`` is not an integer greater than or equal to 0. ValueError: If ``beta`` is smaller than 0. Example: >>> from torchmetrics.functional.text import chrf_score >>> preds = ['the cat is on the mat'] >>> target = [['there is a cat on the mat', 'a cat is on the mat']] >>> chrf_score(preds, target) tensor(0.8640) References: [1] chrF: character n-gram F-score for automatic MT evaluation by Maja Popović `chrF score`_ [2] chrF++: words helping character n-grams by Maja Popović `chrF++ score`_ rzMExpected argument `n_char_order` to be an integer greater than or equal to 1.rzMExpected argument `n_word_order` to be an integer greater than or equal to 0.z.Expected argument `beta` to be greater than 0.N) isinstancerH ValueErrorrgr"rwrxrMcat)rrrnr rr[r=r$r{rZrrrrr r!rsZ chrf_f_scorerrr chrf_scoresl7   r)N)ryr_rzFFF)" collectionsrtypingrrrrrr rMr r Z#torchmetrics.functional.text.helperr rasetr.rHr"rGrIr*r/r4r<rJrQrSrgrhrqrwrxrrrrr#s   > ! .    4 *       :  0 e               p       *