jdZddlZddlmZddlmZmZmZmZm Z ddl m Z ddl m Z ddlmZGdd eZGd d eZy) zEvaluators for parsing strings.N)eq)AnyCallableOptionalUnioncast)parse_json_markdown)override)StringEvaluatorc eZdZdZdeddffd ZedefdZedefdZ ede fdZ e d d e d e e d e e dedef d ZxZS)JsonValidityEvaluatoraEvaluate whether the prediction is valid JSON. This evaluator checks if the prediction is a valid JSON string. It does not require any input or reference. Attributes: requires_input (bool): Whether this evaluator requires an input string. Always False. requires_reference (bool): Whether this evaluator requires a reference string. Always False. evaluation_name (str): The name of the evaluation metric. Always "json". Examples: >>> evaluator = JsonValidityEvaluator() >>> prediction = '{"name": "John", "age": 30, "city": "New York"}' >>> evaluator.evaluate(prediction) {'score': 1} >>> prediction = '{"name": "John", "age": 30, "city": "New York",}' >>> evaluator.evaluate(prediction) {'score': 0, 'reasoning': 'Expecting property name enclosed in double quotes'} kwargsreturnNc "t|yN)super__init__)selfr __class__s k/var/www/html/Fitness-lenito-AI-main/venv/lib/python3.12/site-packages/langchain/evaluation/parsing/base.pyrzJsonValidityEvaluator.__init__&s cyNFrs rrequires_inputz$JsonValidityEvaluator.requires_input)rcyrrrs rrequires_referencez(JsonValidityEvaluator.requires_reference-rrcy)N json_validityrrs revaluation_namez%JsonValidityEvaluator.evaluation_name1r predictioninput referencec t|tjddiS#t$r}dt |dcYd}~Sd}~wwxYw)aLEvaluate the prediction string. Args: prediction (str): The prediction string to evaluate. input (str, optional): Not used in this evaluator. Defaults to None. reference (str, optional): Not used in this evaluator. Defaults to None. Returns: dict: A dictionary containing the evaluation score. The score is 1 if the prediction is valid JSON, and 0 otherwise. If the prediction is not valid JSON, the dictionary also contains a "reasoning" field with the error message. )parserr)score reasoningNr))r jsonloads Exceptionstr)rr$r%r&res r_evaluate_stringsz'JsonValidityEvaluator._evaluate_strings5sB, 5  4:: >| 5SV4 4 5s! A =AANN)__name__ __module__ __qualname____doc__rrpropertyboolrrr/r"r rdictr1 __classcell__rs@rr r s0D $#' }C=     rr c eZdZdZddeededdffd Zede fdZ ede fdZ ede fd Z d edeeedee ee ffd Ze dd e d ee dee dedef dZxZS)JsonEqualityEvaluatoraEvaluate whether the prediction is equal to the reference after parsing both as JSON. This evaluator checks if the prediction, after parsing as JSON, is equal to the reference, which is also parsed as JSON. It does not require an input string. Attributes: requires_input (bool): Whether this evaluator requires an input string. Always False. requires_reference (bool): Whether this evaluator requires a reference string. Always True. evaluation_name (str): The name of the evaluation metric. Always "parsed_equality". Examples: >>> evaluator = JsonEqualityEvaluator() >>> evaluator.evaluate_strings('{"a": 1}', reference='{"a": 1}') {'score': True} >>> evaluator.evaluate_strings('{"a": 1}', reference='{"a": 2}') {'score': False} >>> evaluator = JsonEqualityEvaluator(operator=lambda x, y: x['a'] == y['a']) >>> evaluator.evaluate_strings('{"a": 1}', reference='{"a": 1}') {'score': True} >>> evaluator.evaluate_strings('{"a": 1}', reference='{"a": 2}') {'score': False} Noperatorrrc @t||xst|_yr)rrrr>)rr>rrs rrzJsonEqualityEvaluator.__init__qs  B rcyrrrs rrz$JsonEqualityEvaluator.requires_inputurrcy)NTrrs rrz(JsonEqualityEvaluator.requires_referenceysrcy)N json_equalityrrs rr"z%JsonEqualityEvaluator.evaluation_name}r#rstringc<t|tr t|S|Sr) isinstancer/r )rrDs r _parse_jsonz!JsonEqualityEvaluator._parse_jsons fc "&v. . rr$r%r&c |j|}|jtt|}t|tr0t|tsddiSt |d}t |d}d|j ||iS)aOEvaluate the prediction string. Args: prediction (str): The prediction string to evaluate. input (str, optional): Not used in this evaluator. reference (str): The reference string to compare against. Returns: dict: A dictionary containing the evaluation score. r)rct|Srr/xs rz9JsonEqualityEvaluator._evaluate_strings..s#a&r)keyct|SrrJrKs rrMz9JsonEqualityEvaluator._evaluate_strings..sAr)rGrr/rFlistsortedr>)rr$r%r&rparsedlabels rr1z'JsonEqualityEvaluator._evaluate_stringssy$!!*-  c9!56 eT "fd+|#F(89F5&67Evu566rrr2)r3r4r5r6rrrrr7r8rrr/r"rr9rPfloatintrGr r1r:r;s@rr=r=Rs<'(!3'c'd'D tT4c36 7 $#' 77}7C= 7  7  77rr=)r6r,r>rtypingrrrrrlangchain_core.utils.jsonr typing_extensionsr langchain.evaluation.schemar r r=rrrrZs:% 779&7BOBJP7OP7r