Ah%SrSSKJr SSKJrJrJr SSKJr SSK J r SSK J r SSK Jr SSKJr SS KJrJr SS KJrJr SS KJr SS KJrJr SS KJr SSKJr SSK J!r!J"r"J#r# SSK$J%r% SSK&J'r'J(r(J)r) SSK*J+r+J,r, SSK-J.r.J/r/ S\0S\1\24Sjr30\'Rh\#_\'Rj\"_\'Rl\!_\'Rn\_\'Rp\,_\'Rr\_\'Rt\+_\'Rv\ _\'Rx\_\'Rz\_\'R|\/_\'R~\._\'R\_\'R\_\'R\_\'R\_\'R\_\'R\\'R\%\'R\0ErH\2\'\\I\(\I\ \I\)44\JS'SS.S\'S\\S\S\\ \)44SjjrKSSS.S\\'S\\S \\2S\S\1\\ \)44 S!jjrLg)"z Loading datasets and evaluators.)Sequence)AnyOptionalUnion)BaseLanguageModel)Chain)TrajectoryEvalChain)PairwiseStringEvalChain)LabeledPairwiseStringEvalChain)CriteriaEvalChainLabeledCriteriaEvalChain)EmbeddingDistanceEvalChain"PairwiseEmbeddingDistanceEvalChain)ExactMatchStringEvaluator)JsonEqualityEvaluatorJsonValidityEvaluator)JsonEditDistanceEvaluator)JsonSchemaEvaluator)ContextQAEvalChainCotQAEvalChain QAEvalChain)RegexMatchStringEvaluator) EvaluatorType LLMEvalChainStringEvaluator)LabeledScoreStringEvalChainScoreStringEvalChain)PairwiseStringDistanceEvalChainStringDistanceEvalChainurireturncSSKJn U"SU35n[US5$![anSn[U5UeSnAff=f)aLoad a dataset from the `LangChainDatasets on HuggingFace `_. Args: uri: The uri of the dataset to load. Returns: A list of dictionaries, each representing a row in the dataset. **Prerequisites** .. code-block:: shell pip install datasets Examples -------- .. code-block:: python from langchain.evaluation import load_dataset ds = load_dataset("llm-math") r) load_datasetzXload_dataset requires the `datasets` package. Please install with `pip install datasets`NzLangChainDatasets/train)datasetsr# ImportErrorlist)r r#emsgdatasets T/var/www/html/shao/venv/lib/python3.13/site-packages/langchain/evaluation/loading.pyr#r#(sY,&)/u56G  !! & : #A% &s! >9>_EVALUATOR_MAPN)llm evaluatorr-kwargsc U[;a.SUS[[R553n[U5e[Un[ U[ 5(a-SSKJn U=(d U"SSSS 9nUR"S S U0UD6$U"S 0UD6$![a* SSK Jn N@![anSn[U5UeSnAff=ff=f![anS US 3n[U5UeSnAff=f)aLoad the requested evaluation chain specified by a string. Parameters ---------- evaluator : EvaluatorType The type of evaluator to load. llm : BaseLanguageModel, optional The language model to use for evaluation, by default None **kwargs : Any Additional keyword arguments to pass to the evaluator. Returns ------- Chain The loaded evaluation chain. Examples -------- >>> from langchain.evaluation import load_evaluator, EvaluatorType >>> evaluator = load_evaluator(EvaluatorType.QA) zUnknown evaluator type: z Valid types are: r) ChatOpenAIzCould not import langchain_openai or fallback onto langchain_community. Please install langchain_openai or specify a language model explicitly. It's recommended to install langchain_openai AND specify a language model explicitly.Nzgpt-4*)modelseed temperaturezEvaluation with the z requires a language model to function. Failed to create the default 'gpt-4' model. Please manually provide an evaluation LLM or check your openai credentials.r-) r,r'keys ValueError issubclassrlangchain_openair1r&&langchain_community.chat_models.openai Exceptionfrom_llm)r.r-r/r) evaluator_clsr1r(s r+load_evaluatorr?fs6&&yk!$~':':'<"=!> @ o"9-M-.. ) 27 J'JC%%8#888  "6 ""5 2 2#2?&c*12 2  )&}o655 S/q ( )sNB C CBC B<)B77B<<CC C$ CC$)r-config evaluatorsr@c /nUH?nU(aURU05O0nUR[U4SU00UEUED65 MA U$)a Load evaluators specified by a list of evaluator types. Parameters ---------- evaluators : Sequence[EvaluatorType] The list of evaluator types to load. llm : BaseLanguageModel, optional The language model to use for evaluation, if none is provided, a default ChatOpenAI gpt-4 model will be used. config : dict, optional A dictionary mapping evaluator types to additional keyword arguments, by default None **kwargs : Any Additional keyword arguments to pass to all evaluators. Returns ------- List[Chain] The loaded evaluators. Examples -------- >>> from langchain.evaluation import load_evaluators, EvaluatorType >>> evaluators = [EvaluatorType.QA, EvaluatorType.CRITERIA] >>> loaded_evaluators = load_evaluators(evaluators, criteria="helpfulness") r-)getappendr?)rAr-r@r/loadedr._kwargss r+load_evaluatorsrGsUBF /5&**Y+2 nYQCQ;Pf;P;PQR  M)M__doc__collections.abcrtypingrrrlangchain_core.language_modelsrlangchain.chains.baser1langchain.evaluation.agents.trajectory_eval_chainr langchain.evaluation.comparisonr *langchain.evaluation.comparison.eval_chainr (langchain.evaluation.criteria.eval_chainr r ,langchain.evaluation.embedding_distance.baserr%langchain.evaluation.exact_match.baser!langchain.evaluation.parsing.baserr*langchain.evaluation.parsing.json_distancer(langchain.evaluation.parsing.json_schemarlangchain.evaluation.qarrr%langchain.evaluation.regex_match.baserlangchain.evaluation.schemarrr'langchain.evaluation.scoring.eval_chainrr)langchain.evaluation.string_distance.baserrstrr'dictr#QACOT_QA CONTEXT_QAPAIRWISE_STRING SCORE_STRINGLABELED_PAIRWISE_STRINGLABELED_SCORE_STRINGAGENT_TRAJECTORYCRITERIALABELED_CRITERIASTRING_DISTANCEPAIRWISE_STRING_DISTANCEEMBEDDING_DISTANCEPAIRWISE_EMBEDDING_DISTANCE JSON_VALIDITY JSON_EQUALITYJSON_EDIT_DISTANCEJSON_SCHEMA_VALIDATION REGEX_MATCH EXACT_MATCHr,type__annotations__r?rGr6rHr+rts&$''<'QCULQHSSKTT "c "d4j "Lk.0!!#:   4  ))+I &&(C""$7-""$<!!#:**,K$$&@--/Q!6 !6!"$$&?#$((*=88) $| d5k4+@ @AC<(,@#@# # $@# @#  5/ !" @#L(,! %'% # $% TN %  %  %& '( %rH