~tjUdZddlmZddlZddlZddlZddlmZeedrej ej fnej fZ de d<dd Z ddZddZdS)aStatic extraction of feedback keys from evaluator functions. An evaluator's feedback key is the ``key`` value it returns (e.g. ``{"key": "correctness", "score": 1}``). The runner uses these keys to render per-evaluator progress and to synthesize error feedback when an evaluator raises. They are inferred by parsing the function's source into an AST and reading the literal ``key`` values it produces -- never by executing the function. ) annotationsN)CallableStrtuple_AST_STR_TYPESnodeast.exprreturnstrcRt|tjr|jn|jS)z.Get string value from ast.Str or ast.Constant.) isinstanceastConstantvalues)rs b/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/langsmith/evaluation/_key_extraction.py_get_str_valuers!#D#,77 C4::TVCfuncr list[str]ctj|}d}dfd}tj|} t j|}|jd}t|tjtj fsgSi}g}t j |D]7}t|tj rt|j tj rig} |j jD] } | | !t|jdtjr| ||jdj<t|tjrw|j p||j } |j } ||j |} || || || 9|r|n|jgS#t*$rgcYSwxYw)NcJt|tjryg}d}t|j|jD]S\}}t|t r9t|}|dkr$t|t rt|}T|r|gn|St|tjrut|j tj rV|j j dkrF|j D]>}|j dkr1t|jt rt|jgcS?gS)Nkeydict)r rDictzipkeysvaluesrrCallrNameidkeywordsargr)rr key_valuerrkey_strkeywords rextract_dict_keysz@_extract_code_evaluator_feedback_keys..extract_dict_keys s" dCH % % ;DI!$)T[99 : : Uc>22:,S11G%''Jun,M,M'$25$9$9 "+5I;; 5 tSX & & ;49ch// ; &&= ; ;;%''Jw}n,U,U'*7=99:::: rc$t|tjrut|jtjrV|jjdkrF|jD]>}|jdkr1t|jtrt|jgcS?gS)NEvaluationResultr) r rrrr r!r"r#rrr)rr&s rextract_evaluation_result_keyzL_extract_code_evaluator_feedback_keys..extract_evaluation_result_key4s tSX & & ;49ch// ; 222= ; ;;%''Jw}n,U,U'*7=99:::: rct|tjrt|jtjr|jjdkr|jD]}|jdkrt|jtjr"| |jjgcSt|jtj r3g}|jj D] }|  |!|cSnt|tj rt|j|jD]\}}t|t"rxt%|dkrdt|tj rIg}|j D]:}t|tj rt|j|jD]d\}}t|t"rJt%|dkr7t|t"r"|t%|et|tjrt|jtjrd|jjdvrV|jD]N}|jdkrAt|jt"r'|t%|jO<|cSgS)NEvaluationResultsresultsr)r)r)r rrrr r!r"r#rgetListeltsextendrrrrrrappend) r variablesr&reltrrelt_key elt_valuer*s rextract_evaluation_results_keyszN_extract_code_evaluator_feedback_keys..extract_evaluation_results_keys?s tSX & &' $49ch//' $ 333= $ $;)++!'-::$(}}W]-=rBBBBB#GM38<<$!#*=#5LLC KK(E(Ec(J(JKKKK#  $ch ' ' $!$)T[99 $ $ Uc>22$~c7J7Ji7W7W!%22$!#(:SSC)#sx88S:=ch :S:S!S!S$6GY(27N(K(K%S,:7,C,Cu,L,L+5i+P+P)S,0KKy8Q8Q,R,R,R !S!+3 9 9 S$.sx$B$B S%(HK3O$O$O/2|!S!SG'.{e';'; (/ ~A&A&';)- N7=4Q4Q(R(R(R#  rr)inspect getsourcetextwrapdedentrparsebodyr FunctionDefAsyncFunctionDefwalkAssignrr/r0r1targetsr r!Returnname SyntaxError)r python_coder'r7tree function_defr3rr list_keysr4 dict_keyseval_result_keyeval_results_keysr*s @r%_extract_code_evaluator_feedback_keysrMs$#D))K(   )))))V/+..K y%%y| ,#:N(OPP I H\** / /D$ ++ /dj#(33B "I#zMM!(()F)Fs)K)KLLLL!$,q/38<<B8A $,q/"45D#*-- /$*2H--dj99 "?"? "K"K$C$CJ %%! I&&& O,,, -...4tt,"3!44  sAG?EG G+*G+cF t|S#t$rgcYSwxYw)a'Best-effort static extraction of an evaluator's feedback keys. Reads the function's source and returns the literal ``key`` values it emits. Degrades to ``[]`` when keys can't be statically inferred (dynamic keys, source unavailable, etc.) so evaluator construction never fails. )rM Exception)rs r_safe_extract_feedback_keysrPs94T:::  s   )rr r r )rrr r)__doc__ __future__rrr8r:typingrhasattrrrr__annotations__rrMrPrrrWs#"""""  'wsE22GSWcl DDDD ooood      r