tj (UdZddlmZddlmZddlmZddlmZddl m Z ddl m Z ddl mZdd lmZmZdd lmZmZdd lmZdd lmZmZdd lmZddlmZddlmZm Z m!Z!ddl"m#Z#ddl$m%Z%m&Z&m'Z'ddl(m)Z)m*Z*ddl+m,Z,m-Z-de.de/e0fdZ1ie%j2e!e%j3e e%j4ee%j5e e%j6e*e%j7ee%j8e)e%j9e e%j:ee%j;ee%j<e-e%j=e,e%j>ee%j?ee%j@ee%jAee%jBee%jCee%jDe#e%jEeiZFe0e%eGe&eGezeGe'zfeHd<ddde%dedzdedee'zfdZIddddee%dedzd e0dzdede/ee'zf d!ZJdS)"z Loading datasets and evaluators.)Sequence)Any)BaseLanguageModel)Chain)TrajectoryEvalChain)PairwiseStringEvalChain)LabeledPairwiseStringEvalChain)CriteriaEvalChainLabeledCriteriaEvalChain)EmbeddingDistanceEvalChain"PairwiseEmbeddingDistanceEvalChain)ExactMatchStringEvaluator)JsonEqualityEvaluatorJsonValidityEvaluator)JsonEditDistanceEvaluator)JsonSchemaEvaluator)ContextQAEvalChainCotQAEvalChain QAEvalChain)RegexMatchStringEvaluator) EvaluatorType LLMEvalChainStringEvaluator)LabeledScoreStringEvalChainScoreStringEvalChain)PairwiseStringDistanceEvalChainStringDistanceEvalChainurireturnc ddlm}n$#t$r}d}t||d}~wwxYw|d|}t|dS)aLoad a dataset from the [LangChainDatasets on HuggingFace](https://huggingface.co/LangChainDatasets). Args: uri: The uri of the dataset to load. Returns: A list of dictionaries, each representing a row in the dataset. **Prerequisites** ```bash pip install datasets ``` Examples: -------- ```python from langchain_classic.evaluation import load_dataset ds = load_dataset("llm-math") ``` r) load_datasetzXload_dataset requires the `datasets` package. Please install with `pip install datasets`NzLangChainDatasets/train)datasetsr! ImportErrorlist)rr!emsgdatasets b/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/langchain_classic/evaluation/loading.pyr!r!4s.&))))))) &&& : #A% &l55566G  ! !!s *%*_EVALUATOR_MAPN)llm evaluatorr+kwargsc |tvr;d|dtt}t|t|}t |t r ddlm}n<#t$r/ ddl m}n$#t$r}d}t||d}~wwxYwYnwxYw|p |ddd }n(#t$r}d |d }t||d}~wwxYw|j d d |i|S|d i|S)aFLoad the requested evaluation chain specified by a string. Parameters ---------- evaluator : EvaluatorType The type of evaluator to load. llm : BaseLanguageModel, optional The language model to use for evaluation, by default None **kwargs : Any Additional keyword arguments to pass to the evaluator. Returns: ------- Chain The loaded evaluation chain. Examples: -------- >>> from langchain_classic.evaluation import load_evaluator, EvaluatorType >>> evaluator = load_evaluator(EvaluatorType.QA) zUnknown evaluator type: z Valid types are: r) ChatOpenAIzCould not import langchain_openai or fallback onto langchain_community. Please install langchain_openai or specify a language model explicitly. It's recommended to install langchain_openai AND specify a language model explicitly.Nzgpt-4*)modelseed temperaturezEvaluation with the z requires a language model to function. Failed to create the default 'gpt-4' model. Please manually provide an evaluation LLM or check your openai credentials.r+) r*r%keys ValueError issubclassrlangchain_openair/r$&langchain_community.chat_models.openai Exceptionfrom_llm)r,r+r-r' evaluator_clsr/r&s r)load_evaluatorr=ss6&& @y @ @"&~':':'<'<"="= @ @ oo"9-M-..9 ) 27777777 2 2 2 2#222?&c**12 2J'JJJCC ) ) )5}555 S//q ( )&}%88#8888 = " "6 " ""sZ)A0/B=0 B);BB) B# BB##B)&B=(B))B== C"CC")r+config evaluatorsr>c g}|D]B}|r||ini}|t|fd|ii||C|S)aoLoad evaluators specified by a list of evaluator types. Parameters ---------- evaluators : Sequence[EvaluatorType] The list of evaluator types to load. llm : BaseLanguageModel, optional The language model to use for evaluation, if none is provided, a default ChatOpenAI gpt-4 model will be used. config : dict, optional A dictionary mapping evaluator types to additional keyword arguments, by default None **kwargs : Any Additional keyword arguments to pass to all evaluators. Returns: ------- List[Chain] The loaded evaluators. Examples: -------- >>> from langchain_classic.evaluation import load_evaluators, EvaluatorType >>> evaluators = [EvaluatorType.QA, EvaluatorType.CRITERIA] >>> loaded_evaluators = load_evaluators(evaluators, criteria="helpfulness") r+)getappendr=)r?r+r>r-loadedr,_kwargss r)load_evaluatorsrEsrBFSS /5=&**Y+++2 nYQQCQ;Pf;P;PQQRRRR M)K__doc__collections.abcrtypingrlangchain_core.language_modelsrlangchain_classic.chains.baser9langchain_classic.evaluation.agents.trajectory_eval_chainr'langchain_classic.evaluation.comparisonr2langchain_classic.evaluation.comparison.eval_chainr 0langchain_classic.evaluation.criteria.eval_chainr r 4langchain_classic.evaluation.embedding_distance.baser r -langchain_classic.evaluation.exact_match.baser)langchain_classic.evaluation.parsing.baserr2langchain_classic.evaluation.parsing.json_distancer0langchain_classic.evaluation.parsing.json_schemarlangchain_classic.evaluation.qarrr-langchain_classic.evaluation.regex_match.baser#langchain_classic.evaluation.schemarrr/langchain_classic.evaluation.scoring.eval_chainrr1langchain_classic.evaluation.string_distance.baserrstrr%dictr!QACOT_QA CONTEXT_QAPAIRWISE_STRING SCORE_STRINGLABELED_PAIRWISE_STRINGLABELED_SCORE_STRINGAGENT_TRAJECTORYCRITERIALABELED_CRITERIASTRING_DISTANCEPAIRWISE_STRING_DISTANCEEMBEDDING_DISTANCEPAIRWISE_EMBEDDING_DISTANCE JSON_VALIDITY JSON_EQUALITYJSON_EDIT_DISTANCEJSON_SCHEMA_VALIDATION REGEX_MATCH EXACT_MATCHr*type__annotations__r=rEr4rFr)rrs&&&$$$$$$<<<<<<//////LKKKKKTSSSSSYXXXXXPPPPPP TSSSSS  !"c!"d4j!"!"!"!"Nk.0!#:   4  )+I &(C"$7-"$<!#:*,K$&@-/Q!6 !6!"$&?#$(*=88)e$tO'<<><%)@#@#@#@# T !@# @#  _ @#@#@#@#L%) %%%'% T !% 4K %  %  %/ !" %%%%%%rF