a  d&@s8ddlZddlZddlmZddlmZmZmZmZm Z m Z ddl m Z m Z ddlmZeeefZe eeefZeee eeee eeeeffffZe eeefZe eeee eeffeeee eeeefffZdgdgdd d d d d ZeedddZeeedddZeee dddZeee dddZeeege feeee dddZeee eeefeeeeeeeeeeffffdddZeeefeeeeeeeeeefffe e e e fdd d!Ze e e eee fd"d#d$Zeeeee fdd%d&Z dS)'N)Counter)AnyCallableDictListTupleUnion)Tensortensor)rank_zero_warnzThis is a test text)Z answer_starttextzThis is a test context.1zIs this a test?z train test)answerscontextidZquestiontitle)sreturncCsTttddd}ttddd}ttddd}ttddd }|||||S) zALower text and remove punctuation, articles and extra whitespace.)r rcSstdd|S)Nz\b(a|an|the)\b )resubr rk/var/www/html/stable-diffusion-webui/venv/lib/python3.9/site-packages/torchmetrics/functional/text/squad.pyremove_articles,sz(_normalize_text..remove_articlescSsd|S)Nr)joinsplitrrrrwhite_space_fix/sz(_normalize_text..white_space_fixcs"ttjdfdd|DS)Nc3s|]}|vr|VqdSNr).0chexcluderr 4z7_normalize_text..remove_punc..)setstring punctuationrrrr#r remove_punc2s z$_normalize_text..remove_punccSs|Sr )lowerrrrrr+6sz_normalize_text..lower)str)rrrr*r+rrr_normalize_text)s r-cCs|sgSt|S)z&Split a sentence into separate tokens.)r-r)rrrr _get_tokens<sr.)predicted_answer target_answerrcCst|}t|}t|t|@}tt|}t|dksHt|dkrXtt||kS|dkrhtdSd|tt|}d|tt|}d||||S)z#Compute F1 Score for two sentences.rg?)r.rr sumvalueslenint)r/r0Z target_tokensZpredicted_tokenscommonZnum_sameZ precisionZrecallrrr_compute_f1_scoreAsr8) prediction ground_truthrcCsttt|t|kS)z&Compute Exact Match for two sentences.)r r6r-)r9r:rrr_compute_exact_match_scoreQsr;) metric_fnr9 ground_truthsrcstfdd|DS)zJCalculate maximum score for a predicted answer with all reference answers.c3s|]}|VqdSr r)r!truthr<r9rrr%Zr&z1_metric_max_over_ground_truths..)max)r<r9r=rr?r_metric_max_over_ground_truthsVsrA)predstargetsrc st|tr|g}t|tr |g}|D]$}|}d|vs@d|vr$tdq$|D]L}|}d|vsjd|vrxtdt|d}d|vrNtdtqNdd |D}d d d d fdd|Digig}||fS)zOCheck for types and convert the input to necessary format to compute the input.prediction_textrzExpected keys in a single prediction are 'prediction_text' and 'id'.Please make sure that 'prediction_text' maps to the answer string and 'id' maps to the key string.rzExpected keys in a single target are 'answers' and 'id'.Please make sure that 'answers' maps to a `SQuAD` format dictionary and 'id' maps to the key string. SQuAD Format: r zzExpected keys in a 'answers' are 'text'.Please make sure that 'answer' maps to a `SQuAD` format dictionary. SQuAD Format: cSsi|]}|d|dqS)rrDr)r!r9rrr r&z&_squad_input_check..cSs dd|ddD|ddS)NcSsg|] }d|iqSrr)r!txtrrr r&z8_squad_input_check....rr r)rrr)Ztgtrrrr&z$_squad_input_check.. paragraphsqascsg|] }|qSrr)r!targetZ _fn_answerrrrGr&z&_squad_input_check..) isinstancerkeysKeyError SQuAD_FORMAT) rBrCpredZ pred_keysrK target_keysr preds_dictZ targets_dictrrLr_squad_input_check]s:   rT)rBrKrc Cstd}td}td}|D]}|dD]z}|dD]l}|d7}|d|vrbtd|ddq4d d |d D}||d} |tt| |7}|tt| |7}q4q(q|||fS) auCompute F1 Score and Exact Match for a collection of predictions and references. Args: preds: A dictionary mapping an `id` to the predicted `answer`. target: A list of dictionary mapping `paragraphs` to list of dictionary mapping `qas` to a list of dictionary containing `id` and list of all possible `answers`. Return: Tuple containing F1 score, Exact match score and total number of examples. Example: >>> from torchmetrics.functional.text.squad import _squad_update >>> preds = [{"prediction_text": "1976", "id": "56e10a3be3433e1400422b22"}] >>> target = [{"answers": {"answer_start": [97], "text": ["1976"]}, "id": "56e10a3be3433e1400422b22"}] >>> preds_dict = {pred["id"]: pred["prediction_text"] for pred in preds} >>> targets_dict = [ ... dict(paragraphs=[dict(qas=[dict(answers=[ ... {"text": txt} for txt in tgt["answers"]["text"]], id=tgt["id"]) for tgt in target ... ])]) ... ] >>> _squad_update(preds_dict, targets_dict) (tensor(1.), tensor(1.), tensor(1)) r1rrIrJr rzUnanswered question z will receive score 0.cSsg|] }|dqSrr)r!xrrrrGr&z!_squad_update..r)r r rAr;r8) rBrKf1 exact_matchtotalarticleZ paragraphZqar=rQrrr _squad_updates    rZ)rVrWrXrcCs"d||}d||}||dS)zAggregate the F1 Score and Exact match for the batch. Return: Dictionary containing the F1 score, Exact match score for the batch. gY@)rWrVr)rVrWrXrrr_squad_computes  r[cCs*t||\}}t||\}}}t|||S)aCalculate `SQuAD Metric`_ . Args: preds: A Dictionary or List of Dictionary-s that map `id` and `prediction_text` to the respective values. Example prediction: .. code-block:: python {"prediction_text": "TorchMetrics is awesome", "id": "123"} target: A Dictionary or List of Dictionary-s that contain the `answers` and `id` in the SQuAD Format. Example target: .. code-block:: python { 'answers': [{'answer_start': [1], 'text': ['This is a test answer']}], 'id': '1', } Reference SQuAD Format: .. code-block:: python { 'answers': {'answer_start': [1], 'text': ['This is a test text']}, 'context': 'This is a test context.', 'id': '1', 'question': 'Is this a test?', 'title': 'train test' } Return: Dictionary containing the F1 score, Exact match score for the batch. Example: >>> from torchmetrics.functional.text.squad import squad >>> preds = [{"prediction_text": "1976", "id": "56e10a3be3433e1400422b22"}] >>> target = [{"answers": {"answer_start": [97], "text": ["1976"]},"id": "56e10a3be3433e1400422b22"}] >>> squad(preds, target) {'exact_match': tensor(100.), 'f1': tensor(100.)} Raises: KeyError: If the required keys are missing in either predictions or targets. References: [1] SQuAD: 100,000+ Questions for Machine Comprehension of Text by Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, Percy Liang `SQuAD Metric`_ . )rTrZr[)rBrKrS target_dictrVrWrXrrrsquads7r])!rr( collectionsrtypingrrrrrrZtorchr r Ztorchmetrics.utilitiesr r,ZSINGLE_PRED_TYPEZ PREDS_TYPEr6ZSINGLE_TARGET_TYPEZ TARGETS_TYPEZUPDATE_METHOD_SINGLE_PRED_TYPErPr-r.r8r;rArTrZr[r]rrrrs@    ,:   6 , & /