ޠj,;SrSSKJr SSKrSSKrSSKJrJrJrJ r J r J r J r SSK Jr SSK Jr SSK Jr SSK Jr \(a SS KJr SS KJr "S S 5r\"5r"S S5r"SS5r\"5rS/rg)a<Make approximate assertions as "expectations" on test results. This module is designed to be used within test cases decorated with the `@pytest.mark.decorator` decorator It allows you to log scores about a test case and optionally make assertions that log as "expectation" feedback to LangSmith. Example: ```python import pytest from langsmith import expect @pytest.mark.langsmith def test_output_semantically_close(): response = oai_client.chat.completions.create( model="gpt-3.5-turbo", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Say hello!"}, ], ) response_txt = response.choices[0].message.content # Intended usage expect.embedding_distance( prediction=response_txt, reference="Hello!", ).to_be_less_than(0.9) # Score the test case matcher = expect.edit_distance( prediction=response_txt, reference="Hello!", ) # Apply an assertion and log 'expectation' feedback to LangSmith matcher.to_be_less_than(1) # You can also directly make assertions on values directly expect.value(response_txt).to_contain("Hello!") # Or using a custom check expect.value(response_txt).against(lambda x: "Hello" in x) # You can even use this for basic metric logging within tests expect.score(0.8) expect.score(0.7, key="similarity").to_be_greater_than(0.7) ``` ) annotationsN) TYPE_CHECKINGAnyCallableLiteralOptionalUnionoverloadclient) run_helpers) run_trees)utils)EditDistanceConfig)EmbeddingConfigc,\rSrSrSrSSjrSSjrSrg) _NULL_SENTRYLzA sentinel singleton class used to distinguish omitted keyword arguments from those passed in with the value None (which may have different behavior). cg)NFselfs K/var/www/html/gaurav/venv/lib/python3.13/site-packages/langsmith/_expect.py__bool___NULL_SENTRY.__bool__Qscg)N NOT_GIVENrrs r__repr___NULL_SENTRY.__repr__TsrrN)returnzLiteral[False])r!str)__name__ __module__ __qualname____firstlineno____doc__rr__static_attributes__rrrrrLsrrc\rSrSrSrSSSjjrSSSjjrSSjrSSjrSSjr SS jr SSS jjr SS jr SS jr SS jrSSjrSrg)_Matcher[z4A class for making assertions on expectation values.NcXlX lX0lU=(d [R"SS9Ul[ R"5UlUR(aURRUl gUUl g)N max_workers) _clientkeyvaluels_utilsContextThreadPoolExecutor _executorrhget_current_run_tree_rttrace_id_run_id)rr r1r2r5run_ids r__init___Matcher.__init__^sX  "Wh&H&HUV&W**,,0HHtxx(( & rc [R"5(dUR(d[R"5UlUR R URRURSUUUR(aURROSUR(aURROSS9 gg)N expectation)r;r1scorecomment session_id start_time) r3test_tracking_is_disabledr0rtget_cached_clientr5submitcreate_feedbackr:r8rBrC)rr@messages r_submit_feedback_Matcher._submit_feedbackms1133<<!335 NN ! ! ,,||!26((488..26((488.. " 4rcU(dU5eURSSURSU3S9 g![a#nURS[U55 USeSnAff=f)Nz Success: .)rIr)rJr1AssertionErrorrepr)r conditionrI method_namees r_assert_Matcher._assert{sd  %g %9  ! !!y !K=-Q ! R   ! !!T!W -  s.1 AAAc ~URURU:SURSUSUR3S5 g)zAssert that the expectation value is less than the given value. Args: value: The value to compare against. Raises: AssertionError: If the expectation value is not less than the given value. Expected z to be less than , but got to_be_less_thanNrTr2r1rr2s rrY_Matcher.to_be_less_thans> JJ z!25'DJJ< P  rc ~URURU:SURSUSUR3S5 g)zAssert that the expectation value is greater than the given value. Args: value: The value to compare against. Raises: AssertionError: If the expectation value is not greater than the given value. rWz to be greater than rXto_be_greater_thanNrZr[s rr^_Matcher.to_be_greater_thans> JJ z!5eWJtzzl S  rc URXRs=:=(a U:Os SURSUSUSUR3S5 g)aAssert that the expectation value is between the given min and max values. Args: min_value: The minimum value (exclusive). max_value: The maximum value (exclusive). Raises: AssertionError: If the expectation value is not between the min and max. rWz to be between z and rX to_be_betweenNrZ)r min_value max_values rra_Matcher.to_be_betweensP  . .Y .z 5 L | %   rc UR[URU5[X5:HSURSUSUR3S5 g)a3Assert that the expectation value is approximately equal to the given value. Args: value: The value to compare against. precision: The number of decimal places to round to for comparison. Raises: AssertionError: If the rounded expectation value does not equal the rounded given value. rWz to be approximately rXto_be_approximatelyN)rTroundr2r1)rr2 precisions rrf_Matcher.to_be_approximatelysJ $**i (E%,C Cz!6ugZ | T ! rc ~URURU:HSURSUSUR3S5 g)zAssert that the expectation value equals the given value. Args: value: The value to compare against. Raises: AssertionError: If the expectation value does not exactly equal the given value. rWz to be equal to rXto_equalNrZr[s rrk_Matcher.to_equals> JJ% z!1% 4::, O  rcvURURSLSURSUR3S5 g)zrAssert that the expectation value is `None`. Raises: AssertionError: If the expectation value is not `None`. NrWz to be None, but got to_be_nonerZrs rrn_Matcher.to_be_nones8 JJ$ z!6tzzl C  rcdURXR;SURSUS3S5 g)zAssert that the expectation value contains the given value. Args: value: The value to check for containment. Raises: AssertionError: If the expectation value does not contain the given value. rWz to contain z, but it does not to_containNrZr[s rrq_Matcher.to_contains4 ZZ zeW4E F  rc[R"U5nURU"UR5SUSUR3S5 g)zAssert the expectation value against a custom function. Args: func: A custom function that takes the expectation value as input. Raises: AssertionError: If the custom function returns False. z Assertion z failed for againstN)inspect signaturerTr2r1)rfuncfunc_signatures rrt_Matcher.againstsD!**40  ( TXXJ ?  r)r0r5r8r:r1r2)NN) r Optional[ls_client.Client]r1r"r2rr5z,Optional[ls_utils.ContextThreadPoolExecutor]r; Optional[str]N)r@intrIr{r!None)rQboolrIr"rRr"r!r~)r2floatr!r~)rbrrcrr!r~))r2rrhr}r!r~)r!r~)r2rr!r~)rwrr!r~)r#r$r%r&r'r<rJrTrYr^rarfrkrnrqrtr(rrrr*r*[s}>CG $ A* A A A @ A  A     " "     rr*c\rSrSrSrSS.SSjjrSS.SSjjrSS.SSjjrSS jrS SSS .SS jjr \ SS j5r \ SSj5r \ S4SSjjr SSjr Srg)_Expectiz1A class for setting expectations on test results.Nr cXl[R"SS9Ul[R "URR SS9 g)Nr-r.T)wait)r0r3r4executoratexitregistershutdownrr s rr<_Expect.__init__s1  ::qI  ..T:rconfigc (SSKJn U=(d 0nURS5(aSOSnU"US9nURXS9nXVRS.nUR S UUS US UR3S .5 [ URS XpRS 9$)aCompute the embedding distance between the prediction and reference. This logs the embedding distance to LangSmith and returns a `_Matcher` instance for making assertions on the distance value. By default, this uses the OpenAI API for computing embeddings. Args: prediction: The predicted string to compare. reference: The reference string to compare against. config: Optional configuration for the embedding distance evaluator. Supported options: - `encoder`: A custom encoder function to encode the list of input strings to embeddings. Defaults to the OpenAI API. - `metric`: The distance metric to use for comparison. Supported values: `'cosine'`, `'euclidean'`, `'manhattan'`, `'chebyshev'`, `'hamming'`. Returns: A `_Matcher` instance for the embedding distance value. Example: ```python expect.embedding_distance( prediction="hello", reference="hi", ).to_be_less_than(1.0) ``` r)EmbeddingDistanceencodercustomopenair prediction reference)rmetricembedding_distanceUsing z , Metric: r@ source_inforAr5) 'langsmith._internal._embedding_distancergetevaluatedistancerJr*r0r) rrrrr encoder_func evaluatorr@src_infos rr_Expect.embedding_distance sT N2#)::i#8#8xh %V4 ""j"N+7I7IJ  '#L>I)>JJ0$7  / ""j"N$=  '#F8= D   LL  mm   rc@[URSXRS9$)zCreate a `_Matcher` instance for making assertions on the given value. Args: value: The value to make assertions on. Returns: A `_Matcher` instance for the given value. Example: ```python expect.value(10).to_be_less_than(20) ``` r2r)r*r0rr[s rr2 _Expect.values gu NNrr@)r1 source_run_idrAcrURUUSS0UUS.5 [URX!URS9$)aNLog a numeric score to LangSmith. Args: score: The score value to log. key: The key to use for logging the score. Defaults to `'score'`. Example: ```python expect.score(0.8) # doctest: +ELLIPSIS expect.score(0.8, key="similarity").to_be_greater_than(0.7) ``` methodz expect.score)r@rrrAr)rJr*r0r)rr@r1rrAs rr@ _Expect.scoresD,   (.9!."    cDMMJJrcgr|rr[s r__call___Expect.__call__s36rcgr|rrs rrrsCFrcL[US9nU[LaURU5$U$)Nr )rrr2)rr2r expecteds rrrs* &)  !>>%( (rcJ[R"5nU(a UROSn[R"5(d_UR (d[ R"5UlURR"UR R4XAS.UD6 gg)N)r;r1) r6r7r9r3rDr0rErFrrGrH)rr1results current_runr;s rrJ_Expect._submit_feedbacksy--/ )4%%$1133<<!335 MM  ,, 5; HO 4r)r0r)r rz)rr"rr"rzOptional[EmbeddingConfig]r!r*)rr"rr"rzOptional[EditDistanceConfig]r!r*)r2rr!r*) r@zUnion[float, int, bool]r1r"rzOptional[ls_client.ID_TYPE]rAr{r!r*)r zls_client.Clientr!r)r rzr2z Optional[Any]r!zUnion[_Expect, _Matcher])r1r"rdict)r#r$r%r&r'r<rrr2r@r rrrJr(rrrrrs;?C;-1 ; ; ; * ;  ; D04 8 8 8 - 8  8 tO(59!% K&K K 3 K  K KF66 FF )-1 +   " rrexpect) r' __future__rrrutypingrrrrrr r langsmithr ls_clientr r6rrErr3rrrrrrr*rr__all__rrrrsy0d# *'%'EG   N c c LKK\  *r