ޠj=kSSKJr SSKJrJr SSKJr SSKrSSKJ r SSK J r J r J r SSKJrJrJrJrJrJrJr SS KJrJrJr SS KJr S S KJrJrJrJ r J!r!J"r" SS K#J$r$J%r% SSK&J'r'J(r( SSK)J*r*J+r+ SSK,J-r-J.r. SSK/J0r0 SSK1J2r2 SSK3J4r4 SSK5J6r6 SSK7J8r8 SSK9J:r: SS/r;"SS\$5r<"SS\%5r="SS5r>"SS5r?"SS 5r@"S!S"5rAg)#) annotations)IterableOptional)LiteralN)_legacy_response)eval_list_paramseval_create_paramseval_update_params)BodyOmitQueryHeadersNotGivenomit not_given) path_templatemaybe_transformasync_maybe_transform)cached_property)Runs AsyncRunsRunsWithRawResponseAsyncRunsWithRawResponseRunsWithStreamingResponseAsyncRunsWithStreamingResponse)SyncAPIResourceAsyncAPIResource)to_streamed_response_wrapper"async_to_streamed_response_wrapper)SyncCursorPageAsyncCursorPage)AsyncPaginatormake_request_options)EvalListResponse)EvalCreateResponse)EvalDeleteResponse)EvalUpdateResponse)EvalRetrieveResponse)MetadataEvals AsyncEvalsc j\rSrSrSr\SSj5r\SSj5r\SSj5r\ \ SSS\ S.SSjjr SSS\ S .SS jjr \ \ SSS\ S.SS jjr \ \ \ \ SSS\ S .SS jjrSSS\ S .SSjjrSrg)r,%,Manage and run evals in the OpenAI platform.c,[UR5$r0)r_clientselfs V/var/www/html/gaurav/venv/lib/python3.13/site-packages/openai/resources/evals/evals.pyruns Evals.runs(sDLL!!c[U5$z This property can be used as a prefix for any HTTP method call to return the raw response object instead of the parsed content. For more information, see https://www.github.com/openai/openai-python#accessing-raw-response-data-eg-headers )EvalsWithRawResponser4s r6with_raw_responseEvals.with_raw_response-s$D))r9c[U5$z An alternative to `.with_raw_response` that doesn't eagerly read the response body. For more information, see https://www.github.com/openai/openai-python#with_streaming_response )EvalsWithStreamingResponser4s r6with_streaming_responseEvals.with_streaming_response7s*$//r9Nmetadataname extra_headers extra_query extra_bodytimeoutc URS[UUUUS.[R5[ UUUUSS0S9[ S9$) Create the structure of an evaluation that can be used to test a model's performance. An evaluation is a set of testing criteria and the config for a data source, which dictates the schema of the data used in the evaluation. After creating an evaluation, you can run it on different models and model parameters. We support several types of graders and datasources. For more information, see the [Evals guide](https://platform.openai.com/docs/guides/evals). Args: data_source_config: The configuration for the data source used for the evaluation runs. Dictates the schema of the data used in the evaluation. testing_criteria: A list of graders for all eval runs in this group. Graders can reference variables in the data source using double curly braces notation, like `{{item.variable_name}}`. To reference the model's output, use the `sample` namespace (ie, `{{sample.output_text}}`). metadata: Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters. name: The name of the evaluation. extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds /evalsdata_source_configtesting_criteriarErF bearer_authTrGrHrIrJsecuritybodyoptionscast_to)_postrr EvalCreateParamsr%r' r5rOrPrErFrGrHrIrJs r6create Evals.create@se`zz  *<(8 (  #33)+'%'. '%  r9rGrHrIrJc U(d[SU<35eUR[SUS9[UUUUSS0S9[S9$) Get an evaluation by ID. Args: extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds 6Expected a non-empty value for `eval_id` but received /evals/{eval_id}eval_idrQTrRrVrW ValueError_getrr%r*r5rcrGrHrIrJs r6retrieveEvals.retrieves^.UV]U`ab byy ,g >(+'%'. )  r9c U(d[SU<35eUR[SUS9[UUS.[R 5[ UUUUSS0S9[S9$)  Update certain properties of an evaluation. Args: metadata: Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters. name: Rename the evaluation. extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds r`rarbrErFrQTrRrT)rfrXrrr EvalUpdateParamsr%r)r5rcrErFrGrHrIrJs r6update Evals.updatesDUV]U`ab bzz ,g > ( #33 )+'%'. '!  r9afterlimitorderorder_byrGrHrIrJcURS[[[UUUU[ UUUUS.[ R 5SS0S9[S9$a List evaluations for a project. Args: after: Identifier for the last eval from the previous pagination request. limit: Number of evals to retrieve. order: Sort order for evals by timestamp. Use `asc` for ascending order or `desc` for descending order. order_by: Evals can be ordered by creation time or last updated time. Use `created_at` for creation time or `updated_at` for last updated time. extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds rM)rsrtrurvrQT)rGrHrIrJqueryrS)pagerVmodel) _get_api_listr"r&r%rr EvalListParams r5rsrtrurvrGrHrIrJs r6list Evals.listspH!!  01(+'%%!&!&!&$,  %33(. #'"  r9c U(d[SU<35eUR[SUS9[UUUUSS0S9[S9$) Delete an evaluation. Args: extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds r`rarbrQTrRrdrf_deleterr%r(rhs r6delete Evals.deletes^.UV]U`ab b|| ,g >(+'%'. '  r9)returnr)rr<)rrArOz#eval_create_params.DataSourceConfigrPz-Iterable[eval_create_params.TestingCriterion]rEOptional[Metadata] | OmitrF str | OmitrGHeaders | NonerH Query | NonerI Body | NonerJ(float | httpx2.Timeout | None | NotGivenrr' rcstrrGrrHrrIrrJrrr*rcrrErrFrrGrrHrrIrrJrrr))rsrrt int | OmitruLiteral['asc', 'desc'] | Omitrv*Literal['created_at', 'updated_at'] | OmitrGrrHrrIrrJrrz SyncCursorPage[EvalListResponse] rcrrGrrHrrIrrJrrr(__name__ __module__ __qualname____firstlineno____doc__rr7r=rBrrr[rirprr__static_attributes__rr9r6r,r,%sG6""**00/3)-$("&(+'%'. )    AA A A c # U(d[SU<35eUR[SUS9[UUS.[R 5IShvN [ UUUUSS0S9[S 9IShvN $N!N7f) rlr`rarbrmNrQTrRrT)rfrXrrr rnr%r)ros r6rpAsyncEvals.updatesDUV]U`ab bZZ ,g >, ( #33 )+'%'. '!     s$A A3 A/ A3*A1+A31A3rrcURS[[[UUUU[ UUUUS.[ R 5SS0S9[S9$rx)r|r#r&r%rr r}r~s r6rAsyncEvals.listspH!!  !12(+'%%!&!&!&$,  %33(. #'"  r9c # U(d[SU<35eUR[SUS9[UUUUSS0S9[S9IShvN $N7f) rr`rarbrQTrRrdNrrhs r6rAsyncEvals.delete7sk.UV]U`ab b\\ ,g >(+'%'. '"    rr)rr)rr)rrrrr)rsrrtrrurrvrrGrrHrrIrrJrrzCAsyncPaginator[EvalListResponse, AsyncCursorPage[EvalListResponse]]rrrr9r6r-r-AsH6''//55/3)-$("&> LL  )@@ NN  '>> LL  %<< JJ  '>> LL  r9c@[URR5$r2)rrr7r4s r6r7EvalsWithRawResponse.runsqs#4;;#3#344r9rr[rrrirpNrr,rNone)rrrrrrrrr7rrr9r6r<r<]s &55r9r<c2\rSrSrSSjr\SSj5rSrg)riwcXl[R"UR5Ul[R"UR5Ul[R"UR 5Ul[R"UR 5Ul[R"UR5Ulgr)rrasync_to_raw_response_wrapperr[rirprrrs r6r"AsyncEvalsWithRawResponse.__init__xs &DD LL  )FF NN  'DD LL  %BB JJ  'DD LL  r9c@[URR5$r2)rrr7r4s r6r7AsyncEvalsWithRawResponse.runss( (8(899r9rNrr-rr)rrrrr9r6rrws &::r9rc2\rSrSrSSjr\SSj5rSrg)rAicXl[UR5Ul[UR5Ul[UR5Ul[UR 5Ul[UR 5Ulgr)rr r[rirprrrs r6r#EvalsWithStreamingResponse.__init__sn 2 LL  5 NN  3 LL  1 JJ  3 LL  r9c@[URR5$r2)rrr7r4s r6r7EvalsWithStreamingResponse.runss))9)9::r9rNr)rrrrr9r6rArAs &;;r9rAc2\rSrSrSSjr\SSj5rSrg)ricXl[UR5Ul[UR5Ul[UR5Ul[UR 5Ul[UR 5Ulgr)rr!r[rirprrrs r6r(AsyncEvalsWithStreamingResponse.__init__sn 8 LL  ; NN  9 LL  7 JJ  9 LL  r9c@[URR5$r2)rrr7r4s r6r7$AsyncEvalsWithStreamingResponse.runss.dkk.>.>??r9rNr)rrrrr9r6rrs &@@r9r)B __future__rtypingrrtyping_extensionsrhttpx2rtypesr r r _typesr r rrrrr_utilsrrr_compatr runs.runsrrrrrr _resourcerr _responser r! paginationr"r# _base_clientr$r%types.eval_list_responser&types.eval_create_responser'types.eval_delete_responser(types.eval_update_responser)types.eval_retrieve_responser*types.shared_params.metadatar+__all__r,r-r<rrArrr9r6rs#%% MMKKKKK&;Y9@8<<<@4 L !Y OY xY !Y x554::4;;4@@r9