~tj< UdZddlmZddlZddlZddlZddlmZddl Z ddl Z ddl Z ddl Z ddl Z ddlZ ddlZddlZddlZddlZddlZddlZddlZddlZddlZddlZddlZddlZddlZddlmZmZmZm Z m!Z!ddl m"Z"m#Z#ddl$m%Z%ddl&m'Z'ddl(m)Z)m*Z*dd lm+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3m4Z4dd l5m6Z7ddl8Z9ddl:Z;ddlZ>dd lddDZdEZ ddFlmZn#e$rGdGdHZYnwxYw ddIlmZn#e$rGdJdKZYnwxYwe+rddlZddLlmZddlLmOZOddMlmZddNlmZddOlmZddPlmZddQlmZddRlmZddSlmZddTlmZddUlmZ ddVlmZddWlmZn#e$re-Ze-ZYnwxYwddXlmZddYlmZmZmZddZlmZmZmZmZmZmZmZmZejeZejd[Zd\Zd]Zd^ed_<eZd`e%eIjvZdaZeZdbedc<e"ddeddgZddhdidjddqZddtZdd|Z dddZddZe2ejefZe/dZe j"ddeddZddZddZddddZddZddZ dddZddZdddZddZejGddZejGddddZddddZe j"ddedZGdde@jZGdde1ZddZddZGddZ dddÄZ dddĄZGdńdejZddǜdd˄Zdd΄Z ddЄZ dd҄Z ddԄZ dd՜ddׄZ ddلZddZddZdS( aClient for interacting with the LangSmith API. Use the client to customize API keys / workspace connections, SSL certs, etc. for tracing. Also used to create, read, update, and delete LangSmith resources such as runs (~trace spans), datasets, examples (~records), feedback (~metrics), projects (tracer sessions/groups), etc. For detailed API documentation, visit the [LangSmith docs](https://docs.langchain.com/langsmith/home). ) annotationsN) AsyncIterableIterableIteratorMappingSequence) lru_cachepartial) signature)Path)Full PriorityQueue) TYPE_CHECKING AnnotatedAnyCallableLiteralOptional TypedDictUnioncastget_args)parse)Field)adapters) multipart) TypeGuard deprecatedoverload)PoolKey)Retry)env)schemas)utils)_aiter)_orjson _profiles_v2_migration_utils)_check_backend_version)TracingQueueItem)tracing_control_thread_func)r)suppress_deprecation_warning) warn_beta)CompressedTraces)_AUTO_SCALE_UP_NTHREADS_LIMIT_BLOCKSIZE_BYTES _BOUNDARY_SIZE_LIMIT_BYTES_TRACING_QUEUE_MAX_SIZE)HUBREPO_HANDLE_PATTERNbuild_commit_urlplatform_hub_pathvalidate_parent_commit) MultipartPartMultipartPartsAndContext join_multipart_parts_and_context)SerializedFeedbackOperationSerializedRunOperation#combine_serialized_queue_operations$compress_multipart_parts_and_contextserialize_feedback_dictserialize_run_dictc&kk\>* * * * + Nrisessionrequests.Sessiondict[str, Any]cd|ji}tjfd|jD}|r||d<|jr |j|d<|jdur |j|d<|jr |j|d<|j dp|j d }|r.d ttj jjvrd nd }|||<|S) aTranslate a `requests.Session`'s transport config into `httpx` kwargs. The generated OpenAPI client is `httpx`-based, so a `requests.Session` cannot be shared with it directly. Instead we carry over the settings that have a direct `httpx` equivalent, so that a caller-supplied session's TLS, proxy, cookie and header configuration also applies to the v2 endpoints. Not carried over (no `httpx` equivalent): custom mounted `HTTPAdapter`s, `session.auth`, and `session.hooks`. Connection pool bounds and retries are left at the generated client's own defaults rather than being derived from the session's adapter. trust_envcRi|]#\}}|||k ||$SNget).0keyvaluerequests_defaultss re z._httpx_kwargs_from_session..sH C  !2!6!6s!;!;u!D!D U!D!D!DriheaderscookiesTverifycerthttpshttpproxyproxies)rwrequestsr$default_headersritemsrrrrr{r _httpxClient__init__ parameters)rskwargsrr proxy_kwargrs @re_httpx_kwargs_from_sessionrs)*7+<=F!6688!///11G $#y,$Oy~T!!">x|& v O   ( ( GGO,?,?,G,GE $ )FM$:;;FFF G  ${ Mri) OPENAI_API_KEYANTHROPIC_API_KEY otel_enabled tracing_modeOptional[TracingMode]rOptional[bool] TracingModec $d}d}d}tj|}|^|}|tvr?tjd|ddt t|S|8tjdtd |rtj |rd Sd Sd S||}|tvr?tjd|ddt ttj |}tj |}|s|r#tjd|d|d|d|d d |Stj |rd Stj drd Sd S)a*Resolve the effective tracing mode from the constructor arg and env vars. Priority: explicit ``tracing_mode`` argument > deprecated ``otel_enabled`` argument > ``LANGSMITH_TRACING_MODE`` env var > legacy ``OTEL_ENABLED`` / ``OTEL_ONLY`` env vars > default ``"langsmith"``. TRACING_MODE OTEL_ENABLED OTEL_ONLYNzInvalid tracing_mode=z. Must be one of: , zThe 'otel_enabled' parameter is deprecated and will be removed in the next minor version. Use 'tracing_mode' instead, e.g. Client(tracing_mode="hybrid") or Client(tracing_mode="otel").r stacklevelrSrTrRzInvalid LANGSMITH_TRACING_MODE=zBoth LANGSMITH_z and the legacy LANGSMITH_z / LANGSMITH_z env vars are set. LANGSMITH_z takes precedence.) ls_utils get_env_varlowerrULangSmithUserErrorjoinsortedwarningswarn FutureWarningis_env_var_truish)rrmode_envvar_nameotel_enabled_envvar_nameotel_only_envvar_nameenv_mode legacy_otel legacy_onlys re_resolve_tracing_modersA&-'#$455H#))++ 3 3 3-M MM#'99V4H-I-I#J#JMM   L        )*?@@ v8{>>## / / /-M(MM#'99V4H-I-I#J#JMM 01IJJ 01FGG  +  MB"2BB5BB2BB.BBB     !"788v!.11x ;ricz ddlm}ddlm}ddlm}ddlm}||||fS#t$rtdwxYw)z,Dynamically import OTEL modules when needed.rtrace)set_span_in_context)get_otlp_tracer_provider OTELExporterzKTo use OTEL tracing, you must install it with `pip install langsmith[otel]`) opentelemetryropentelemetry.tracer%langsmith._internal.otel._otel_clientr'langsmith._internal.otel._otel_exporterr ImportError) otel_tracerrrs re _import_otelr.s  555555;;;;;;       IHHHHH.0H,VV     Y    s :)ZoneInfoceZdZdZdS)rzIntroduced in python 3.9.N__name__ __module__ __qualname____doc__rhrirerrDs''''rir)TracerProviderceZdZdZdS)rzUsed for optional OTEL tracing.NrrhrirerrLs----rir)Runnable)AsyncAnnotationQueuesResource)AsyncDatasetsResource)AsyncOnlineEvaluatorsResource)AsyncPublicResource)AsyncRunsResource)AsyncSandboxesResource)AsyncThreadsResource)AsyncTracesResource)Runrr evaluator) AEVALUATOR_T ATARGET_TAsyncExperimentResults)COMPARATIVE_EVALUATOR_TDATA_T EVALUATOR_T EXPERIMENT_TSUMMARY_EVALUATOR_TTARGET_TComparativeExperimentResultsExperimentResultszurllib3.connectionpool x-api-keyrhztuple[dict, ...] EMPTY_SEQ key_blocksizez:How are people using my agent? What are they asking about?zset[str]_fallback_dirs_createdr[maxsizeboolcd} ddlm}tj|}|tjdko"|tjdkp"|tjdk}n?#t t tf$r%}t d|Yd}~nd}~wwxYw|S) zCheck if the installed `langchain_core.load.load` supports the `allowed_objects` parameter. Returns `True` if `langchain-core >= 0.3.81` and `< 1.0`, or `>= 1.2.5`. Fr) __version__z0.3.81z1.0.0z1.2.5zbFailed to determine langchain-core version for allowed_objects support, defaulting to disabled: %sN) langchain_corer packagingversionrr ValueError TypeErrorradebug)allowed_objects_supportedr lc_versionexcs re&_lc_load_allowed_objects_arg_supportedrs !& ......&,,[99  )+11(;; ; >Y.44W===%>I-33G<<< "! Y /     2           %$sBBC*C  Cr2depth max_depth max_widthmanifest dict | listrintrrc&dkrtddkrtdkrdSt|tr#t|hdkr|ddkrdSdzkrdSt|trBt fd t j|DSt|ttfr%t fd |d DSdS) zidlctypersecretTc3DK|]}t|dzVdSr[rN_manifest_has_secretsr|xrrrs re z(_manifest_has_secrets..U   "i9         ric3DK|]}t|dzVdSrrrs rer z(_manifest_has_secrets..r riN) r isinstancedictsetany itertoolsislicevaluestuplelist)rrrrs ```rerrss1}}67771}}6777 u8T"" MM111 1 1 V  ( (t i  u Hd # #      %hoo&7&7CC       Hudm , ,      jyj)       uriprompt_identifierdangerously_pull_public_promptchtj|\}}}|dkr|stddSdS)N-zPulling a public prompt by owner/name is disabled by default because prompts may contain untrusted serialized LangChain objects. If you trust this prompt, set `dangerously_pull_public_prompt=True` to acknowledge the risk.)rparse_prompt_identifierr)rrowner_s re_validate_public_prompt_pullrsK23DEEKE1a ||:| $   |||ri prompt_objectr include_model bool | Nonesecretsdict[str, str] | Nonesecrets_from_envc  ddlm}ddlm}ddlm}ddlm}ddlm }ddl m } m } n#t$rtdwxYw dd lm} n%#t$rt jd } YnwxYwi} t%r |rd nd | d <| 5 ||jf||d| } n>#t($r1}t+|jr|s|st-d||d}~wwxYw dddn #1swxYwYt/| |s%t/| | rt/| j|rt/| |r| n-t/| | rt/| j|r| jnd}|t3jd|ji|_|j|j|j|jd|r!t/| | rt/| j|rtA| j!dkrt/| j"|st/| j"| rt/| j"j#|rytI| | j| j"j#z}tA|j!dkr<|j!d}| | j|j%dii| j"j&|j"} n)|} n&t/| j"|r| j| j"z} n | S)atProcess a prompt manifest into a LangChain prompt object. This is the common logic shared between `Client.pull_prompt()` and `AsyncClient.pull_prompt()`. Args: prompt_object: The prompt commit object containing the manifest. include_model: Whether to include model information. secrets: Map of secrets to use when loading. secrets_from_env: Whether to load secrets from environment variables. Returns: The processed prompt object. Raises: ImportError: If `langchain-core` is not installed. ValueError: If secrets are required but not provided. r)BaseLanguageModel)load)BaseOutputParser)BasePromptTemplateStructuredPromptRunnableBindingRunnableSequencezuThe client.pull_prompt function requires the langchain-core package to run. Install with `pip install langchain-core`)suppress_langchain_beta_warningc3KdVdSryrhrhrirer-zA_process_prompt_manifest..suppress_langchain_beta_warnings EEEEEriallcoreallowed_objects) secrets_mapr"aFailed to load prompt. The prompt manifest contains secrets (like API keys or access tokens) but no secrets were provided. This is due to a security patch in langsmith 0.5.1 that disabled reading secrets from environment variables by default. To resolve this: - Recommended: Pass secrets directly via `secrets={'KEY_NAME': 'value'}` - If this is a trusted prompt: Set `secrets_from_env=True` to read secrets from environment variables Underlying error: Nz-Prompt object is not a valid prompt template.) lc_hub_owner lc_hub_repolc_hub_commit_hashrr[rh)'#langchain_core.language_models.baser$langchain_core.load.loadr%langchain_core.output_parsersr&langchain_core.promptsr'!langchain_core.prompts.structuredr)langchain_core.runnables.baser+r,rlangchain_core._apir- contextlibcontextmanagerrr Exceptionrrr firstrLangSmithErrormetadataupdaterrepo commit_hashrpstepslastboundrbindr)rrr r"r$r%r&r'r)r+r,r- load_kwargsprompteprompt_templateseq rebound_llms re_process_prompt_manifestrQs2  IIIIII111111BBBBBB======FFFFFFSSSSSSSSS     I    GGGGGGG   "   # "    K-//L2?)KV %& ( ( * * T&#!1 FF    %m&<== (    !.+,..   G#  8 6-.. f. / /  v|%7 8 8 &"455 FFf&677v|-?@@    ")?   # +')O $ '' - 3,1&3&?       v/ 0 0 v|%5 6 6   ! # #Jv{DT,U,U # fk? 3 3  K 09 9  ' 8I)IJJC39~~""!il ))L$K$>>'=&+*<'=>>H   %6 7 7 '-|fk'AFF  MsQ&)AAA0/A0C5B*)C5* C%4,C  C%%C55C9<C9r6dataset url_or_tokenUnion[str, uuid.UUID] num_partskindtuple[str, str]c t|tjstj|r|t|fSn#t$rYnwxYwt jt|}|jd}t||kr|| }t|dntj d|d||j dkrd}n |j dkrd }||fS) z*Parse a public dataset URL or share token.rmz token parts)varzInvalid public z URL: zsmith.langchain.comhttps://api.smith.langchain.comzbeta.smith.langchain.comz$https://beta.api.smith.langchain.com)r uuidUUIDrWr urllib_parseurlparsepathsplitrp_as_uuidrrnetloc)rSrkrUrV parsed_url path_parts token_uuids re_parse_token_or_urlrfcs lDI . . .$)L2I2I .C --- - .      &s<'8'899J&&s++J :)## + /////)*VD*V*V *V*VWWW1113  8 8 88 J s>A AAurlc tj|jdd}|dkp|dS#t $rYdSwxYw)zCheck if the URL is langchain hosted. Args: url: The URL to check. Returns: `True` if the URL is langchain hosted, `False` otherwise. :rz langchain.comz.langchain.comF)r]urlsplitrbr`ror@)rgrbs re_is_langchain_hostedrksl&s++288==a@(MFOO-D-DV-L-LL/*. &'  " 2 2\ 2 22ricbtd|dS)zIClose the session. Args: session: The session to close. zClosing Client.sessionN)rarclose)rss re close_sessionrs(  LL)*** MMOOOOOriname Optional[str]cdD]D}tj|d|}||dkr|cSEdS)N) LANGSMITH LANGCHAINr)osenvironr{strip)r namespacer~s re_get_langsmith_env_var_uncachedrsY/  )44d4455  "!4!4LLL 4rirRrapi_keyc|sIt|r<|dkr8tjr'tjdtjdSdSdSdSdS)apVerify API key is provided if url not localhost. Args: api_url: The API URL. api_key: The API key. tracing_mode: Resolved tracing mode; when ``"otel"`` the warning is suppressed because no LangSmith REST calls are made. Raises: LangSmithUserError: If the API key is not provided when using the hosted service. rSz8API key must be provided when using hosted LangSmith APIN)rkrtracing_is_enabledrrLangSmithMissingAPIKeyWarning)rkrrs re_validate_api_key_if_hostedrs"    ) ) &&+--' MJ6         &&&&riscoreUnion[float, int, bool, None]cPt|trt|dS|S)zFormat a feedback score by truncating numerical values to 4 decimal places. Args: score: The score to format, can be a number or any other type Returns: The formatted score )r floatround)rs re_format_feedback_scorers)%UA Lriinfols_schemas.LangSmithInfocd}t|jtjkrt d|t jd|tjddS)zuRaise on SmithDB-only deployments, warn elsewhere. Call only when run-level feedback has no ``session_id``. zJhttps://docs.langchain.com/langsmith/smithdb-sdk-migration#feedback-createztsession_id must be provided when creating feedback for a run: this deployment cannot locate the run without it. See zlCreating feedback for a run without session_id is deprecated and will stop working in a future release. See rrN) rGinstance_flagsrF SMITHDB_ONLYrrrrLangSmithWarning)rdocss re_check_feedback_session_idrs XD,--1JJJ MFJ M M    M 926 9 9! ritracing_sampling_rateOptional[float] float | Nonec|tjd}|sdSnt|}t|}|dks|dkrtjd||S)zTGet the tracing sampling rate. Returns: The tracing sampling rate. NTRACING_SAMPLING_RATErr[zELANGSMITH_TRACING_SAMPLING_RATE must be between 0 and 1 if set. Got: )rrrWrr)rsampling_rate_str sampling_rates re_get_tracing_sampling_raters$$01HII  4  566+,,MqMA--) %" % %    ri_write_api_urlsOptional[dict[str, str]]dict[str, str]c|pi}i}|D]\}}|}|stjd|ddd}|dd}t |||||<|S)Nz&LangSmith runs API URL cannot be empty"'rm)rrrrrnr)rprocessed_write_api_urlsrgr processed_urlprocessed_api_keys re_get_write_api_urlsrs%+O!'--//DD W  X-.VWW W%++--33C88>>sCCJJ3OO #MMOO11#66<  s0AB  B ) rrrrrrrrrrrXrYF) rrr __attrs__requests_adaptersDEFAULT_POOLSIZEDEFAULT_RETRIESDEFAULT_POOLBLOCKrr __classcell__rs@rerrjsI!2 B->/@/P,> R R R R R R RrircFeZdZUdZded<ded<ded<ded <ded <d S) ListThreadsItemz-Item returned by :meth:`Client.list_threads`.rW thread_idzlist[ls_schemas.Run]runsrrdrmin_start_timemax_start_timeN)rrrr__annotations__rhrirerrsN77NNNJJJ!!!!!!!!!!rir attachmentct|trdSt|ttfr/t |dkrt|dt SdS)aReturn True if an attachment is a filesystem path rather than inline bytes. Serialization opens any attachment data that isn't ``bytes`` as a file path (see ``serialized_run_operation_to_multipart_parts_and_context``). Inline data is ``bytes`` or a 2-element ``(content_type, bytes)``; anything else is treated as a filesystem reference (fail closed). Unlike the old ``(tuple, Path)`` guard, this also catches JSON/dict-shaped input (``list``/``str``). Fr6r[T)r bytesrrrp)rs re!_attachment_references_filesystemrs]*e$$u*udm,,4ZA1E1EjmU3333 4ri attachmentsOptional[dict]dangerously_allow_filesystemc||s|sdS|D] }t|rtd!dS)zRaise unless every attachment is inline data, or filesystem access is opted in. Centralizes the guard that previously lived (incorrectly) in three places. See GHSA-f4xh-w4cj-qxq8. NzAttachments must be inline bytes or a (content_type, bytes) pair. To load an attachment from a filesystem path, set dangerously_allow_filesystem=True.)rrr)rrrs re_reject_filesystem_attachmentsrsi$;!((** ,Z 8 8 5  riceZdZUdZgdZded<ded<ded<ded<d ed <d ed <d ed<ded<ded<ded<d ed< d+dddddddddddddddddddddddddddd,dEZd-dGZd.dIZd/dKZ e d0dMZ e d1dOZ e d2dQZ e d3dSZe d4dUZe d5dWZe d6dYZe d7d[Zd8d_Zedd`d9dgZd:dhZd;dkZd:dlZe d:dmZe d:dnZddtZe d?duZejd@dvZe d?dwZ e jd@dxZ e dd߄Zdddddddd\dZ?dddddddd]dZ@ dZddd^dZAddddddddd_dZBddd`dZCdddddddddddddddddddddadZDddddddd̜dbdZEddddddd̜dcd ZF ddded ZGd+dfd ZHdgdZIeJd dhddddidZKeJddddddddddjd#ZLeJd$ddddddddddddddddd%dkd-ZMeJd.dddd/ddd0dld3ZNddddddddddddddddd4dmd=ZOeJd>ddd?dndAZPddd?dndBZQeJdCddDdodFZReJdGdpdHZSeJdIdqdJZTdrdKZUeJdL d+dsdOZVeJdP d+dtdQZW d+ddRdudVZX d+ddRdudWZYdvdXZZdwdYZ[dddZdxd]Z\ddddd^dydbZ]dddddddddcdzdkZ^ddddddld{dmZ_d|doZ`d}dqZaebjcdrddddsd~dvZdddddsd~dwZeddxddyZfdddrddzZg dddZhebjcd?ddd?ddZide(j)j*dddddddZjdddddZkebjcddddddZl d+ddRddZm d+ddRddZndddddddddZoebjcddddddZpdddddZqdddddddZrdddddddZsdddddZtddZuebjcd dddZvebjcd dddZw dddZx dJddZydddddZzdddddZ{dddddZ|ddĄZ} dddȄZ~dddddɄZdddʜdd̈́ZebjcdddddddΜddӄZdԄZdՄZebjcd ddd݄Zddޜdd߄Z dd/dddddddZddddddddddZdddddddZddZddddZddddddZddddddZdddddZddZddddZeJd dddddd ddZ dddddZeJddddddd ddZ ddddddde(jjddddddddddddddd/Zddddd0dd1Zdd2Zddddd3dd7Zdd8Z d+ddddd9dd;Zdddd<dd@ZdddAddFZddGddIZdJZedJdddd/dKddMZedJddPZedJdddQddVZedJddWZedJddXZddYdd]Zdddd/d^dd`ZdddaddbZddcZddddddddgZdddddhddnZddpZdddddqdÐdrZdĐdsZdddtdŐdvZdƐdwZdǐdzZddd{dȐdZdddddddɐdZdʐdZdːdZ d̐d͐dZdΐdZdϐdZdАdZdѐdZdҐdZdҐdZdd/dde(jjddddӐdZdԐdZddddddՐdZddddd֐dZdddddddאdZdؐdZ dhdِdZ dhdڐdZdddddېdZdd/dddܐdZdddddddݐdZddddddddddސdńZddƜdߐdʄZdddddd˜ddτZddƜddфZdddddd˜dd҄Z͐ddӄZΐddԄZϐddՄZАddքZѐdd/ddddלdd؄ZҐdd/ddddלddلZӐdd܄ZԐdd݄ZՐddބZ֐dd߄ZאddZؐddZِddZڐd+dfdZېd+dfdZe dddZe dddZ dddZ dddZ ddd ZeJd  ddd Zeeddddd ddZedddddddddZeddddddd"Zddd#dd&Zdddd'dd)Zdd*ZdS(rz.Client for interacting with the LangSmith API.)5 __weakref__rk_api_key_oauth_access_token _workspace_id_headers_custom_headers retry_config timeout_ms_timeoutrs_get_data_type_cached_web_url _tenant_idtracing_sample_rate_filtered_post_uuids tracing_queue _anonymizer _hide_inputs _hide_outputs_hide_metadata_omit_traced_runtime_info_process_buffered_run_ops_run_ops_buffer_size_run_ops_buffer_timeout_ms_run_ops_buffer_last_flush_time_infor _settings_manual_cleanup_atexit_handler _pyo3_clientcompressed_traces_data_available_event_futures_run_ops_buffer_run_ops_buffer_lock otel_exporter _otel_trace_set_span_in_context_max_batch_size_bytes_use_daemon_threads_tracing_error_callback_multipart_disabled_cache_failed_traces_dir_failed_traces_max_bytes _tracing_mode _profile_auth_profile_auth_headers_langsmith_api_langsmith_api_sync_session_providedrrrrrr ztuple[float, float]r rrzOptional[_profiles.ProfileAuth]r1r2z Optional[LangsmithOpenAPIClient]r3z$Optional[SyncLangsmithOpenAPIClient]r4r5NTF)rr r web_urlrsauto_batch_tracing anonymizer hide_inputs hide_outputs hide_metadataomit_traced_runtime_infoprocess_buffered_run_opsrun_ops_buffer_sizerun_ops_buffer_timeout_msrapi_urlsotel_tracer_providerrrr workspace_idmax_batch_size_bytesrtracing_error_callbackdisable_prompt_cachecacherkrr Optional[Retry]r %Optional[Union[int, tuple[int, int]]]r6rsOptional[requests.Session]r7r8 Optional[Callable[[dict], dict]]r9-Optional[Union[Callable[[dict], dict], bool]]r:r;r<r=4Optional[Callable[[Sequence[dict]], Sequence[dict]]]r> Optional[int]r?rr/Optional[Union[dict, ls_schemas.LangSmithInfo]]r@rrAOptional[TracerProvider]rrrrrrBrCrrD%Optional[Callable[[Exception], None]]rErF"Optional[Union[bool, PromptCache]]rXrYc|r|rtjdtjdstjdr(tjdrtjdt ||}||_t d}t d}t d }tj} ||n|p| j }!||n|}"|d uo|d u}#|#o| j }$|"|"n |$rd n| j }%||n|p| j }&|$r| j nd |_d |_i|_t#||_t'|_t+||_tj|&|_|pi|_|jrSt5t7|j|_ d |_d |_i|_|j|j |_ ntj|!|_ |$rOtj| t< |_|j|_|jj |_tj |%|_ tC|j |j p|jp|j|jj"rd nd | |j |j i|_|p tG|_$tK|tLr||fn|pd|_'|j'ddz |j'ddz f|_(||_)d |_*|r|ntWj,}'|'|_-|d u|_.|tK|t^j0r|nt_j0d7i||_1tej3|th|j-tkj6th|'|_7tqj9|j7d |_:d |_;d |_<g|_=t}j?|_@d |_A||_Bd|_CtjDddk|_E|dvrS t\}(})}*}+|(jG},|,Ht}-|tK|,|(jJr=t|-dr-tKt|(jM|-jN|(jOstt|,}n|*}|(jQ||+||_A|(|_R|)|_Snr#t$retjVd|ddd |_Ad|_tC|j |j p|jp|j|jj"rd nd d YnwxYw|rtjDd}.|.tM|.nt}/t|/|_Yt}jZttej\|f|jE]nd |_Yt|j$ttdz }0|j-ad!|0|j-ad"|0tkjbd#|jc|_d||_e| | ntjDd$dk|_f| | ntjDd%dk|_g| | ntjDd&dk|_h| |_i| |_j||_k|pd'|_ltjm|_n| |td(| |td)d |_ptjDd*d }1 dd lq}1n?#t$r2}2tsd+t|2Yd }2~2nd }2~2wwxYw|1rrd,}3d-}4d}5d}6 |1u|j |j |3|4|5|6|_pn?#t$r2}2tsd.t|2Yd }2~2nd }2~2wwxYwd |_wd|_x||_y||rtjVd/td|AtjVd0td|durd |_{n/|d1ur t|_{n||_{n|s t|_{nd |_{tjDd2|_}tjDd3}7 |7rtM|7nd}8|8dkrtM|8d4zd4znd5|_~n9#ttf$r%tsd6|7d5|_~YnwxYwd |_d |_d S)8u!Initialize a `Client` instance. Args: api_url: URL for the LangSmith API. Defaults to the `LANGCHAIN_ENDPOINT` environment variable or `https://api.smith.langchain.com` if not set. api_key: API key for the LangSmith API. Defaults to the `LANGCHAIN_API_KEY` environment variable. retry_config: Retry configuration for the `HTTPAdapter`. timeout_ms: Timeout for the `HTTPAdapter`. Can also be a 2-tuple of `(connect timeout, read timeout)` to set them separately. web_url: URL for the LangSmith web app. Default is auto-inferred from the `ENDPOINT`. session: The session to use for requests. If `None`, a new session will be created. v2 endpoints are served by an `httpx`-based client, which cannot share the session itself. Its headers, cookies, `verify`, `cert`, proxies and `trust_env` settings are translated onto that client; custom mounted `HTTPAdapter`s, `session.auth` and `session.hooks` are not. auto_batch_tracing: Whether to automatically batch tracing. anonymizer: A function applied for masking serialized run inputs, outputs, metadata and error, before sending to the API. hide_inputs: Whether to hide run inputs when tracing with this client. If `True`, hides the entire inputs. If a function, applied to all run inputs when creating runs. hide_outputs: Whether to hide run outputs when tracing with this client. If `True`, hides the entire outputs. If a function, applied to all run outputs when creating runs. hide_metadata: Whether to hide run metadata when tracing with this client. If `True`, hides the entire metadata. If a function, applied to all run metadata when creating runs. omit_traced_runtime_info: Whether to omit runtime information from traced runs. If `True`, runtime information (SDK version, platform, Python version, etc.) will not be stored in the `extra.runtime` field of runs. Defaults to `False`. process_buffered_run_ops: A function applied to buffered run operations that allows for modification of the raw run dicts before they are converted to multipart and compressed. Useful specifically for high throughput tracing where you need to apply a rate-limited API or other costly process to the runs before they are sent to the API. Note that the buffer will only flush automatically when `run_ops_buffer_size` is reached or a new run is added to the buffer after `run_ops_buffer_timeout_ms` has elapsed - it will not flush outside of these conditions unless you manually call `client.flush()`, so be sure to do this before your code exits. run_ops_buffer_size: Maximum number of run operations to collect in the buffer before applying `process_buffered_run_ops` and sending to the API. Required when `process_buffered_run_ops` is provided. run_ops_buffer_timeout_ms: Maximum time in milliseconds to wait before flushing the run ops buffer when new runs are added. Defaults to `5000`. Only used when `process_buffered_run_ops` is provided. info: The information about the LangSmith API. If not provided, it will be fetched from the API. api_urls: A dictionary of write API URLs and their corresponding API keys. Useful for multi-tenant setups. Data is only read from the first URL in the dictionary. However, ONLY Runs are written (`POST` and `PATCH`) to all URLs in the dictionary. Feedback, sessions, datasets, examples, annotation queues and evaluation results are only written to the first. otel_tracer_provider: Optional tracer provider for OpenTelemetry integration. If not provided, a LangSmith-specific tracer provider will be used. tracing_mode: Where to send traces. One of: - ``"langsmith"`` (default) — LangSmith only. - ``"otel"`` — OpenTelemetry export only. - ``"hybrid"`` — both OTel and LangSmith. Falls back to the ``LANGSMITH_TRACING_MODE`` env var, then to the legacy ``OTEL_ENABLED`` / ``OTEL_ONLY`` env vars, then to ``"langsmith"``. otel_enabled: *Deprecated.* Use ``tracing_mode`` instead. When ``True``, interpreted as ``tracing_mode="hybrid"`` (or ``"otel"`` if the ``OTEL_ONLY`` env var is set). Will be removed in the next minor version. tracing_sampling_rate: The sampling rate for tracing. If provided, overrides the `LANGCHAIN_TRACING_SAMPLING_RATE` environment variable. Should be a float between `0` and `1`, where `1` means trace everything and `0` means trace nothing. workspace_id: The workspace ID. Required for org-scoped API keys. max_batch_size_bytes: The maximum size of a batch of runs in bytes. If not provided, the default is set by the server. headers: Additional HTTP headers to include in all requests. These headers will be merged with the default headers (User-Agent, Accept, x-api-key, etc.). Custom headers will not override the default required headers. tracing_error_callback (Optional[Callable[[Exception], None]]): Optional callback function to handle errors. Called when exceptions occur during tracing operations. disable_prompt_cache: Disable prompt caching for this client. By default, prompt caching is enabled globally using a singleton cache. Set this to `True` to disable caching for this specific client instance. To configure the global cache, use `configure_global_prompt_cache()`. !!! example ```python from langsmith import Client, configure_global_prompt_cache # Use default global cache client = Client() # Disable caching for this client client_no_cache = Client(disable_prompt_cache=True) # Configure global cache settings configure_global_prompt_cache(max_size=200, ttl_seconds=7200) ``` cache: **[Deprecated]** Control prompt caching behavior. This parameter is deprecated. Use `configure_global_prompt_cache()` to configure caching, or `disable_prompt_cache=True` to disable it. - `True`: Enable caching with the global singleton (default behavior) - `False`: Disable caching (equivalent to `disable_prompt_cache=True`) - `Cache(...)`/`PromptCache(...)`: Use a custom cache instance !!! example ```python from langsmith import Client, Cache, configure_global_prompt_cache # Old API (deprecated but still supported) client = Client(cache=True) # Use global cache client = Client(cache=False) # Disable cache # Use custom cache instance my_cache = Cache(max_size=100, ttl_seconds=3600) client = Client(cache=my_cache) # New API (recommended) client = Client() # Use global cache (default) # Configure global cache for all clients configure_global_prompt_cache(max_size=200, ttl_seconds=7200) # Or disable for a specific client client = Client(disable_prompt_cache=True) ``` Raises: LangSmithUserError: If the API key is not provided when using the hosted service. LangSmithUserError: If both `api_url` and `api_urls` are provided. z-You cannot provide both api_url and api_urls.LANGSMITH_ENDPOINTLANGCHAIN_ENDPOINTLANGSMITH_RUNS_ENDPOINTSz]You cannot provide both LANGSMITH_ENDPOINT / LANGCHAIN_ENDPOINT and LANGSMITH_RUNS_ENDPOINTS.rENDPOINTAPI_KEY WORKSPACE_IDN)api_key_headerz profile-authr)i'i`rr[F USE_DAEMONtrue)rSrT_tracer)tracer_providerz tracing_mode=zu requires OpenTelemetry packages. Install with `pip install langsmith[otel]`. Falling back to LangSmith-only tracing.r6rrRTRACING_QUEUE_MAX_SIZEr)targetargsdaemon)rrrhttp://https://r HIDE_INPUTS HIDE_OUTPUTS HIDE_METADATAizOrun_ops_buffer_size must be provided when process_buffered_run_ops is specifiedzOprocess_buffered_run_ops must be provided when run_ops_buffer_size is specifiedUSE_PYO3_CLIENTzaFailed to import `langsmith_pyo3` when PyO3 client was requested, falling back to Python impl: %s@Bdz|Failed to instantiate `langsmith_pyo3.BlockingTracingClient` when PyO3 client was requested, falling back to Python impl: %szBoth 'cache' and 'disable_prompt_cache' were provided. The 'cache' parameter is deprecated and will be removed in a future version. Using 'cache' parameter value.zThe 'cache' parameter is deprecated and will be removed in a future version. Use 'configure_global_prompt_cache()' to configure the global cache, or 'disable_prompt_cache=True' to disable caching for this client.TFAILED_TRACES_DIRFAILED_TRACES_MAX_MBii@zJInvalid value for LANGSMITH_FAILED_TRACES_MAX_MB: %r, using default 100 MBrh)rrrgetenvrr0rr'load_profile_client_configrk has_oauthrrBoauth_access_tokenrr1r2rrrrrrget_workspace_idrr nextiter get_api_url ProfileAuth X_API_KEYcurrent_auth_headers get_api_keyrhas_authrr r rr r rrrSessionrsr5 ls_schemas LangSmithInforweakreffinalizer functoolsr ratexitregisterr!r"r#r$ threadingLockr%r&r)r,rr*rget_tracer_provider get_tracerrProxyTracerProviderhasattrr ProxyTracerr] NoOpTracerrset_tracer_providerr'r(rrrr3rrThread_tracing_control_thread_funcrefstartrr0r/mountr _get_data_typer rrrrrrrrr^rrr langsmith_pyo3rarbreprBlockingTracingClientr@rrr+DeprecationWarningr-rMr.r/ OverflowErrorr3r4)9rrkrr r r6rsr7r8r9r:r;r<r=r>r?rr@rArrrrBrCrrDrErF resolved_mode env_api_url env_api_keyenv_workspace_idprofile_configapi_url_explicit_or_env_api_keyprofile_auth_enableduse_profile_oauthapi_key_ workspace_id_session_rrrrexisting_providertracerqueue_maxsize_str queue_maxsizeadapterrrMqueue_capacity batch_sizebatch_timeout_millisworker_threads _max_mb_str_max_mbs9 rerzClient.__init__s p  x -?  I* + + /1y9M/N/N i233 -0  .lVVV *75jAA 5i@@ :>JJ"=??*GG 0U~?U .5-@''k&$F;$3F0M^5M'2 $ #!(' ' L!@^%@  2C LN - -  ?C57"#=>S#T#T 47EE! >  ,---=A@D">B QS$-N$4$4!59%9"). #+#7 #E#E#O . . .3  !NN ', %CJ$B$D$D!*55h??'/"#4j6TUUM#FI66M' * 6 &&&1 M04NDU/V/V,,/G/G/I/I,6 67KLLL%1\BV%W%W%W"#- ,?))    >M>>> !  &*"%0"+LL/ -9 .7:'"!,       .  & ( 45M N N %0%&&&,  ;H%;;;D   3k$'')/     egggg!%D ()&7:     9g... :w///%DY%8%D%D%D  & & "&& K%m44> ' L%n55? ( M%o66&@  *B&)A&$7!*C*Kt'/3y{{, $ /4G4Oa  $ +0C0Oa  !   1 2 2 >!N %%%%%   6GG  !* '+$!"(6(L(L  &",& ))D%%!NNZQEI$'=$  !5  M1#        MR#     ~~59 $4 $ % 0DKKDK191E 2 2 *+ABB  >*5 > > NN'    ->D ) ) )  >##'   sV/CTA,U>=U>8]== ^9(^44^9)_11 `-;(`((`-,5d""3ee_httpx.Timeoutctj|jd|jd|jd|jdS)Nrr[)connectreadwritepool)rTimeoutr rs re_openapi_timeoutzClient._openapi_timeout4sC~M!$q!-"q!     riLangsmithOpenAPIClientcD|jt|j}|}d}|jr!t d||dt |j}t|j |j rt|j nd|||j pd||_|jSN)base_urltimeout)r tenant_idrrr http_clientrh) r3rrrkrr5_AsyncHttpxClientWrapperrrsrrrrWrrrrrs re_get_langsmith_apizClient._get_langsmith_api<s   &,T\::H++--GK% 6%#1>> #9 595GQ#d0111T! $ 5' ###D ""riSyncLangsmithOpenAPIClientcD|jt|j}|}d}|jr!t d||dt |j}t|j |j rt|j nd|||j pd||_|jSr) r4rrrkrr5_SyncHttpxClientWrapperrrsrrrrWrrs re_get_langsmith_api_synczClient._get_langsmith_api_syncQs  # +,T\::H++--GK% 5%#1>> (B 595GQ#d0111T! $ 5' (((D $''rircjt|jjd|jS)zAccess the runs resource.0.16.0 min_version)r)rrrrrs rerz Client.runsfs1 ty0hGGGG&&((--riAsyncEvaluatorsResourcecjt|jjd|jS)zAccess the evaluator resource.rr)r)rrronline_evaluatorsrs re evaluatorszClient.evaluatorsls1 ty0hGGGG&&((::rircjt|jjd|jS)z=Access the sandboxes resource (registries, snapshots, boxes).rr)r)rrr sandboxesrs rerzClient.sandboxesrs1 ty0hGGGG&&((22rircjt|jjd|jS)z8Access the v2 datasets resource (experiment_runs, etc.).rr)r)rrrdatasetsrs rerzClient.datasetsxs1 ty0hGGGG&&((11rircjt|jjd|jS)z4Access the annotation queues resource (runs, items).z0.16.14r)r)rrrannotation_queuesrs rerzClient.annotation_queues~s1 ty0iHHHH&&((::rircjt|jjd|jS)z8Access the threads resource (query, stats, list_traces).rr)r)rrrthreadsrs rerzClient.threadss1 ty0hGGGG&&((00rircjt|jjd|jS)z.Access the traces resource (query, list_runs).rr)r)rrrtracesrs rerz Client.traces1 ty0hGGGG&&((//rircjt|jjd|jS)z&Access the public shared-run resource.rr)r)rrrpublicrs rerz Client.publicrribody_fnCallable[[], bytes]r c|jsdS |}t|tr|d}||j|d||jdS#t $rYdSwxYw)uDump a failed trace payload to disk if a fallback directory is configured. *body_fn* is called lazily inside a try/except so that any serialization errors are silently swallowed — we must never raise from a failure path. Nzutf-8zruns/multipart)endpointr max_bytes)r.r rWencode_write_trace_to_fallback_dirr/r@)rrrrs re_dump_failed_tracezClient._dump_failed_traces&  F 799D$$$ ,{{7++  - -')7 .         DD sAA%% A32A3)r directoryrWrrrrcd||tj|d}dtjddt jjddd}t||z } |tvr6|j d d t ||f|d kr`t|}td | d D} | |krtd|| |dS|d} | t%j|| d| |td|dS#t,$r'} td|| Yd} ~ dSd} ~ wwxYw)a&Persist a failed trace payload to a local fallback directory. Saves a self-contained JSON file with the endpoint, the HTTP headers required for replay, and the base64-encoded request body. Can be replayed later with a simple POST: POST / Content-Type: [Content-Encoding: ] If *max_bytes* is set, new traces are dropped when the directory is already at or over the budget. r[)rrr body_base64trace_z.6frNz.jsonT)parentsexist_okrc3pK|]1}||jV2dSry)is_filestatst_size)r|fs rer z6Client._write_trace_to_fallback_dir..sOyy{{FFHH$riz trace_*.jsonzCould not write trace to fallback dir %s as it's already over size limit (%d bytes >= %d bytes). Increase LANGSMITH_FAILED_TRACES_MAX_MB if possible.z.tmpizALangSmith trace upload failed; data saved to %s for later replay.zELangSmith tracing error: could not write trace to fallback dir %s: %s)base64 b64encodedecoder^r[uuid4hexr rparentmkdiraddsumglobrarb with_suffix write_textjsondumpschmodrenamer@error) rrrrrenvelopefilenamefilepathdir_pathru temp_path write_excs rerz#Client._write_trace_to_fallback_dirs0 !+D1188::    JDIKKIIIdjll.>rr.BIII ??X-#  666%%dT%BBB&**9555$Q ??%]]>:: I%%NNO"! F ,,V44I  H!5!5 6 6 6 OOE " " "   X & & & NNS         LLW          s 8B%F"BF"" G,GGc|j}d|dS)zReturn an HTML representation of the instance with a link to the URL. Returns: The HTML representation of the instance. z LangSmith Client) _host_url)rlinks re _repr_html_zClient._repr_html_s ~W4WWWWrirr@c|jrE ||dS#t$r tddYdSwxYwdS)zInvoke the background tracing error callback if configured. Args: error: The exception that occurred during background tracing. z!Error in tracing_error_callback: Texc_infoN)r+r@rar)rrs re_invoke_tracing_error_callbackz%Client._invoke_tracing_error_callbacks  '  ,,U33333    8!   s &A  A cd|jdS)zReturn a string representation of the instance with a link to the URL. Returns: The string representation of the instance. zClient (API URL: )rkrs re__repr__zClient.__repr__ s 34<2222ric*t|jSry)rrkrs re_hostz Client._hosts$,'''ric@tj|j|jS)zThe web host url.)r get_host_urlrrkrs rerzClient._host_urls$T]DLAAAricdtjdd}||j|jr|j|t <n6|jr||jn|jr d|j|d<|jr |j|d<|S)Nz langsmith-py/application/json)z User-AgentAcceptzBearer Authorization X-Tenant-Id) rRrrDr rrvr2rr)rrs re_compute_headerszClient._compute_headerssA)*?AA(   t+,,, < L!%GI    ' L NN45 6 6 6 6  % L'K1I'K'KGO $   8%)%7GM "ri attr_namer~rct|||t|d|dS)z8Set attributes that affect headers and recalculate them.rN)object __setattr__r)rrr~s re_set_header_affecting_attrz!Client._set_header_affecting_attr/sD4E2224T-B-B-D-DEEEEEric8|js|jdS|j}t|d|t|d|jjt|d|dS)Nr2rr)rr1get_auth_headersrrrpr)r auth_headerss re_ensure_profile_authzClient._ensure_profile_auth4s < 4-5 F)::<< 4!8,GGG ');)N    4T-B-B-D-DEEEEEric|jS)z+Return the API key used for authentication.)rrs rerzClient.api_key>s }ric2|d|dS)Nrrrr~s rerzClient.api_keyCs '' E:::::ric|jS)z.Return the workspace ID used for API requests.)rrs rerBzClient.workspace_idG !!ric2|d|dS)Nrr%r&s rerBzClient.workspace_idLs ''?????ric|jS)z0Return the custom headers used for API requests.)r rs rerzClient.headersPs ##ric6|d|pidS)Nr r%r&s rerzClient.headersUs# ''(95;BGGGGGrirc  |j|jS|jdkr&|jtj|_|jS |ddddi|j}tj|tjd i| |_na#t$rT}t d|j d t|tj|_Yd}~nd}~wwxYw|jS) zGet the information about the LangSmith API. Returns: The information about the LangSmith API, or `None` if the API is not available. NrSGETz/inforr)rrzFailed to get info from : rh)rr0r&r{r|request_with_retriesr rraise_for_status_with_textr BaseExceptionrarbrkr)rresponserMs rerz Client.infoYs" : !:    ' 'D,>,J#133DJ:  400!#56 1H  / 9 9 9#1DDHMMOODDDJJ 4 4 4 NND4<DD477DD   $133DJJJJJJ  4 zsAB C<(A C77C<ls_schemas.LangSmithSettingsc|jR|dd}tj|t jdi||_|jS)ztGet the settings for the current tenant. Returns: The settings for the current tenant. Nr-z /settingsrh)rr/rr0r{LangSmithSettingsr)rr2s re _get_settingszClient._get_settingszsZ > !00 DDH  / 9 9 9'9LLHMMOOLLDN~ricontent_lengthc||jdSttj|j}|j}|sdS|jp|d}|dS||kr d|d|dSdS)Nsize_limit_byteszThe content length of z) bytes exceeds the maximum size limit of z bytes.)rrr{r|batch_ingest_configr)r{)rr7rbic size_limits re_content_above_sizezClient._content_above_sizes  !TZ%74J,dj99& 4/N377;M3N3N  4 J & &===)3=== trir[r)request_kwargsstop_after_attemptretry_on to_ignorehandle_response_contextmethod0Literal['GET', 'POST', 'PUT', 'PATCH', 'DELETE']pathnamer>Optional[Mapping]r?rr@'Optional[Sequence[type[BaseException]]]rArB1Optional[Callable[[requests.Response, int], Any]]rCrrequests.Responsec ~||pi}i|j|di| di} |j|j| } d|ji|| d| i}|dkr.d|vr*d|vr&|dds d|dd<t jt jt|j  g} g|pd t j t j t j fR} g|pd R} d}t|D]r} t jt | 5|jj|t'|j|fd d i|}dddn #1swxYwYt j||cS#t,jj$r_}t2d ||dz|krd|zt7jdzz}t9j|Yd}~d}~wt,j$rF}|| |dz|kr|||dz}|rYd}~|jdkr4t j d|jd|d|dtA|| |jdkrt j d||d||jdkr)t j!d|dtA|||jdkr)t j"d|dtA|||jdkr)t j#d|dtA|||jdkr)t j$d |dtA|||jd!kr |%}|d"d#}|d$krt j&d%n#tNtPf$rYnwxYwt j)d&|d|dtA|t j)d&|d|dtA|t j&d&|d|dtA|d}~wt,j*$r"}|jd'krd(nd)} tW|jr,t|jj,d*nd#}|-|}|r|}n#tN$rd}YnwxYw|jr!|jj,d+pd#nd#}|dd,|d-d}}d.t]d/t_|d0z z}|||}t j d1|d|d|dtA|d2|d3|| |d}~wt`$r}tc|j2}t_|dkr|dnd#}|3d4d5}|rXd64t|d/g|gzd7t_|dkr |ddngDz}n|}t j)d&|d|d|||d}~wwxYw#| $r1}|$t2d ||cYd}~cSYd}~sd}~wt j!$r|dz|kr|k tk|j,d8d9} nA#t`$r4}t26d:tA|d;} Yd}~nd}~wwxYw| d|zzt7jz} t9j| Y/| $r>|dz|krd|zt7jdzz}t9j|YpwxYwt j)d&|d|d<)=aSend a request with retries. Args: method: The HTTP request method. pathname: The pathname of the request URL. Will be appended to the API URL. request_kwargs: Additional request parameters. stop_after_attempt: The number of attempts to make. retry_on: The exceptions to retry on. In addition to: `[LangSmithConnectionError, LangSmithAPIError]`. to_ignore: The exceptions to ignore / pass on. handle_response: A function to handle the response and return whether to continue retrying. _context: The context of the request. **kwargs: Additional keyword arguments to pass to the request. Returns: The response object. Raises: LangSmithAPIError: If a server error occurs. LangSmithUserError: If the request fails. LangSmithConnectionError: If a connection error occurs. LangSmithError: If the request fails. rNrr-datafiles Content-Typer)rrhstreamFzPassing on exception %sr[r6rtizServer error (z) caused failure to  z in LangSmith API. rsz(Client took too long to send request to izRate limit exceeded for z. izAuthentication failed for izResource not found for iz Conflict for irr!org_scoped_key_requires_workspacezThis API key is org-scoped and requires workspace specification. Please provide 'workspace_id' parameter, or set LANGSMITH_WORKSPACE_ID environment variable.z Failed to rZz'Please confirm your LANGCHAIN_ENDPOINT.z(Please confirm your internet connection.zContent-Lengthr*rz#Connection error caused failure to z Content-Length: z API Key: rszsession (project) c,g|]}t|SrhrW)r|args re z/Client.request_with_retries..`sWWWCs3xxWWWriz retry-after30zInvalid retry-after header: %sz in LangSmith API.)7r"rr{r1prepare_request_headersr rFilterLangSmithRetryFilterPoolFullWarningrWrLangSmithConnectionErrorLangSmithRequestTimeoutLangSmithAPIErrorrange filter_logs_urllib3_loggerrsrequest_construct_urlrkr0r exceptions ReadTimeoutrarrandomr^sleep HTTPError status_coderLangSmithRateLimitErrorLangSmithAuthErrorLangSmithNotFoundErrorLangSmithConflictErrorrrrKeyErrorrBConnectionErrorrrr=maxrpr@rrareplacerrrb)!rrDrFr>r?r@rArBrCrrlogging_filters retry_on_ to_ignore_r2idxrM sleep_timeshould_continue error_data error_coderecommendationr7size_recrprefixrqfillermasked_api_keyramsgemsg retry_afters! rer/zClient.request_with_retriess N !!###'-2 m   B// jjB''    )(@@IIG t}    w    eOO.((~--"9-11.AA.9KN9 %n 5  ) + +  *DJ @ @ @ 6 n"6 10*6 6 7Lb6K6K +,,^ ^ C] B!-oOO#74<#7"*4<BB$$$)$- $$7AAA#OOO*6LL!:A>>>Qw"444!"C6=??S+@!AJJz***HHHH)???+*6"Qw);;;2A/(CRSG2T2T#2!-$,HHHH#/366"*"<!.1E!.!.[a!.!.$,!.!.3777!.$,!.!.## &1S88"*"B!96!9#+!9!9.6!9!9##&1S88"*"B!.8!.!.tAww!.#+!.!.##&1S88"*"=!.X!.!.a!.#+!.!.##&1S88"*"A!.(!.!.d1gg!.#+!.!.##&1S88"*"A O O ODGG OX O O##&1S88 %-5]]__ -7^^GR-H-H #-1T#T#T*2*E)^+&+&%&$U %/#9%%% $%"*"9!3V!3!3h!3!3)-a!3!3## #+"9!3V!3!3h!3!3)-a!3!3## '9XXX(XXtTUwwXX /    <+LLLBAG# .), y$C 1 5 56F G GHHH!#** $(#;#;N#K#K#6-5N%...)-.EFIU )--k::@bSU&-RaR['"##,FF C3w<>>#OOOOOOOO(''''3 ( ( (7000')&+H,<,@,@PT,U,U&V&V $)))< GG') )*AsF2V]__D  ;'''''   7000Vv}'<=  :&&&  % > > >( > > >   s5F(*F9 F(F F( F F((Y/^! ^!paramsr_rrIterator[dict]c#lK|r|ni}|dd}|dd|d< ||d<|d||}|}|sdS|Ed{Vt ||dkrdS|t |z }k) zGet a paginated list of items. Args: path: The path of the request URL. params: The query parameters. Yields: The items in the paginated list. offsetrlimitrjTr-rN)copyr{r/rrp)rr_rparams_rr2rs re_get_paginated_listzClient._get_paginated_lists$*1&++---rXq))";;w44 ! &GH 001H MMOOE        5zzGG,,, c%jj F !riPOSTr)rrequest_methoddata_keyrLiteral['GET', 'POST']rc#xK|r|ni} |||dt|i}|}|sdS||sdS||Ed{V|d}|sdS|dsdS|d|d<)aMGet a cursor paginated list of items. Args: path: The path of the request URL. body: The query body. request_method: The HTTP request method. data_key: The key in the response body that contains the items. Yields: The items in the paginated list. TrLr>Ncursorsrrcursor)rr/ _dumps_jsonrr{) rr_rrrrr2 response_bodyrs re_get_cursor_paginated_listz!Client._get_cursor_paginated_lists&"&-$))+++2 000K00 1H%MMOOM   $$X.. $X. . . . . . . .#'' 22G ;;v&&  'GH ' 0ri) description data_typedf pd.DataFramer input_keys Sequence[str] output_keysrrOptional[ls_schemas.DataType]ls_schemas.Datasetctj}||d|d|d|f|||||S)aUpload a dataframe as individual examples to the LangSmith API. Args: df: The dataframe to upload. name: The name of the dataset. input_keys: The input keys. output_keys: The output keys. description: The description of the dataset. data_type: The data type of the dataset. Returns: The uploaded dataset. Raises: ValueError: If the `csv_file` is not a `str` or `tuple`. Example: ```python from langsmith import Client import os import pandas as pd client = Client() df = pd.read_parquet("path/to/your/myfile.parquet") input_keys = ["column1", "column2"] # replace with your input column names output_keys = ["output1", "output2"] # replace with your output column names dataset = client.upload_dataframe( df=df, input_keys=input_keys, output_keys=output_keys, name="My Parquet Dataset", description="Dataset created from a parquet file", data_type="kv", # The default ) ``` F)rrzdata.csv)rrrrr)ioBytesIOto_csvseek upload_csv)rrrrrrrcsv_files reupload_dataframezClient.upload_dataframesj`:<< (% ((( a  "!##    ri)rrrr?Union[str, tuple[str, io.BytesIO], tuple[str, io.BytesIO, str]]c||d}|r||d<|r||d<|rtj||d<ttj|d<t |tr[t |d5}dt|j|dfi} | d d || } d d d n #1swxYwYnft |trB|} t|d kr|d|ddf} | d d |d| i } ntdtj | | } d| vrWd| dvrMt |tr|n|d} | dd} td| dt!jdi| |j|dS)aBUpload a CSV file to the LangSmith API. Args: csv_file: The CSV file to upload. If a string, it should be the path. If a tuple, it should be a tuple containing the filename and a `BytesIO` object. input_keys: The input keys. output_keys: The output keys. name: The name of the dataset. description: The description of the dataset. data_type: The data type of the dataset. Returns: The uploaded dataset. Raises: ValueError: If the `csv_file` is not a string or tuple. Example: ```python from langsmith import Client import os client = Client() csv_file = "path/to/your/myfile.csv" input_keys = ["column1", "column2"] # replace with your input column names output_keys = ["output1", "output2"] # replace with your output column names dataset = client.upload_csv( csv_file=csv_file, input_keys=input_keys, output_keys=output_keys, name="My CSV Dataset", description="Dataset created from a CSV file", data_type="kv", # The default ) ``` )rrrrrrrbfileztext/csvrz/datasets/upload)rLrMNr6rr[z"csv_file must be a string or tupledetailzalready existsrmDataset z already existsrrrh)rget_enum_valuerWr[rr openr rr/rrprr0rr`r{Datasetr_get_optional_tenant_id)rrrrrrrrLrfile_r2 file_tupleresult file_names rerzClient.upload_csv snj%&    DL  ."-D   C ( 7 B BD &&T h $ $ Ch%% $x.."5q*!EF44& 5               % ( ( C!J8}}!!&qk8A; C 00"z* 1HHABB B+H555 v  "2fX6F"F"F$.x$=$=N8A;I!,,R0IB BBBCC C!   n3355     s53B44B8;B8r3Union[ls_schemas.Run, dict, ls_schemas.RunLikeDict]rDrc\t|dr2tt|dr|}nt t |}d|vrt j|d<n8t|dtrt j |d|d<d|vrE|d=|rtj |d|d<| |d|d<d|vrE|d=|rtj |d|d<||d|d<d|vr&|d||d|d<d|vr&|d||d|d<d|vrlt|dt rQ|d}d |vrE|d =|rtj |d |d <||d |d <|sF|d s1t&jt&jj|d <d |vr_|d d vr|d dn1|d r|d dd|S)aLTransform the given run object into a dictionary representation. Args: run: The run object to transform. update: Whether the payload is for an "update" event. copy: Whether to deepcopy run inputs/outputs. Returns: The transformed run object as a dictionary. model_dumprinputsNoutputseventsrextrarCr serializedrun_type)rnrLgraph)rcallablegetattrrrr r[rr rWr\r deepish_copy_hide_run_inputs_hide_run_outputs_filter_new_token_events_hide_run_error_hide_run_metadatar{rrctimezoneutcr)rrrDr run_creaters re_run_transformzClient._run_transformn s 3 % % )(73 3M3M*N*N )"~~//JJdCJ z ! !#z||Jt    4(# . . ;#yD)9::Jt  z ! !j&:&F S'/'S(T(T 9%$($:$::i;P$Q$QJy ! z ! !j&:&F#'#@#@HAU#V#VJx j Z%8%D"&"6"6z'7J"K"KJw  j Z 70CT%J%J w'EU""uZ'8'DQ(0(=eJ>O(P(PE*%$($;$;E*. s)VVVdaAXDUDUQDUDUDUri) rls_envget_runtime_environmentrr  setdefaultget_langchain_env_var_metadatarrDr) rr runtime_envr run_extrarlangchain_metadataaddedrCs @re_insert_runtime_envzClient._insert_runtime_env s  )  F466  @ @JT:#8#8"#E#EFFI%00B??G#=k#=W#=Ii &11*bAAH!'!F!H!H VVVV&8&>&>&@&@VVVE @ 7 7 > >??? @ @ricL|jdStj|jkS)NT)rrjrs re_should_samplezClient._should_sample s$  # +4}!999ripatch=Iterable[Union[dict, ls_schemas.Run, ls_schemas.RunLikeDict]]rrcJ|jt|Stfd d}|rkg}|D]d}t||d}||jvr||:||d |kr|j|e|Sg}|D]}||ddp ||d }||jvr%||d |krE|r||_|j|z|||S) Nrrr}rWdefaultrXc ||S#ttf$r/|turt||cYSt|||cYSwxYwry)rrr_UNSETr)rr}rs re_valz)Client._filter_for_sampling.._val si 23xi( 2 2 2f$$"3,,,,,sC11111 2s *A A  A trace_idr)rrr}rWrrrXr) rrrrarappendremoverr)rrrrsampledrrs re_filter_for_samplingzClient._filter_for_sampling s  # +:: 4: 2 2 2 2 2  G ? ?#DDj$9$9::4#<<<NN3''''T#t__00-44X>>>NG ( (4Z66I$$sD//t8884T??h..**,,@s++++155h????NN3''''Nrirc|jS)zPer-client tracing mode.)r0rs rerzClient.tracing_mode r(riitemr*c|jJ |j|dS#t$r!td|jjdYdSwxYw)z3Put an item on the tracing queue, dropping if full.Nztracing queue full (maxsize=r )r put_nowaitr rfr)rrs re_put_tracing_queuezClient._put_tracing_queue s!---    ) )$ / / / / /    Lt/A/ILLL       s''AA) project_name revision_idrrrkrrur RUN_TYPE_Trrrc b| dd} | dd} | dd} | dd} |p&| dtj}i| ||||d}||gsdS|||dd d <||d }||gt |d ||jr| dso|j 5|j d|||| | | | df| r|  ddddS#1swxYwYdS||||| | | | dS)a0Persist a run to the LangSmith API. Args: name (str): The name of the run. inputs (Dict[str, Any]): The input values for the run. run_type (str): The type of the run, such as tool, chain, llm, retriever, embedding, prompt, or parser. project_name (Optional[str]): The project name of the run. revision_id (Optional[Union[UUID, str]]): The revision ID of the run. api_key (Optional[str]): The API key to use for this specific run. api_url (Optional[str]): The API URL to use for this specific run. service_key (Optional[str]): The service JWT key for service-to-service auth. tenant_id (Optional[str]): The tenant ID for multi-tenant requests. authorization (Optional[str]): The Authorization header value. cookie (Optional[str]): The Cookie header value. **kwargs (Any): Additional keyword arguments. Returns: None Raises: LangSmithUserError: If the API key is not provided when using the hosted service. Example: ```python from langsmith import Client import datetime from uuid import uuid4 client = Client() run_id = uuid4() client.create_run( id=run_id, project_name=project_name, name="test_run", run_type="llm", inputs={"prompt": "hello world"}, outputs={"generation": "hi there"}, start_time=datetime.datetime.now(datetime.timezone.utc), end_time=datetime.datetime.now(datetime.timezone.utc), hide_inputs=True, hide_outputs=True, ) ``` service_keyNr authorizationcookie session_name)rrrrrrCrFrris_run_ops_buffer_flushpostrkrrrrrrrkrrrr)rrget_tracer_projectrrrrr{rr%r$r_should_flush_run_ops_buffer_flush_run_ops_buffer _create_run)rrrrrrrrrkrrrrrrs re create_runzClient.create_run shv#)**]D"A"A & ; = = $*JJ$E$E #ZZ$77# vzz   ' ) )( (   (    ((*66  F  "=HJw  +M :((%(@@    *...& NN= ) )+G    ) &**=V2W2W *  $++"'.'.+6)2-:&,     44661..000%                  (   '#+      s+AFF  F rrrrrrc|d9|d#|j|j|dS|j|||||||jt dt d|}t|\} } t dd| j |jj 5t| |jt} | r.|jxjdz c_|jdddn #1swxYwYt!t#| dS|jt d|}t dd|j|j|j]|t1|d|||||||||j dS|t1|d||||||| dS|||||||| dS|||||||| dS) Nr dotted_order@Run compression is enabled but threading event is not configuredrrR5Adding compressed multipart to queue with context: %sr[/Adding to tracing queue: trace_id=%s, run_id=%srrkrrrr otel_contextr)r{r rr!r"rrArCralogcontextlockr?r1 trace_countr _close_filesrrrrrr&rr*r(r'get_current_span_create_run_non_batch) rrrrkrrrr serialized_opmultipart_form opened_filesenqueueds rerzClient._create_runn sW NN: & & 2~..: ,!,,Z88888&2OO'%!)N-5$Z!36: F F L!"  K"* +099C&.!  H  9.::a?::266888999999999999999T,"5"5"7"78899999#/ 26: F F  E!*!$  %1++(&~6)$+$+(3&/*7#))-)B)B $ 0 A A C C**    ++(&~6)$+$+(3&/*7#)        **## +'"/!+  & &'#+ '     s A D##D'*D'c Vg}t|||||g} | r|p|j} t|j||||||j} |d| dt || dtjfn#t$r} | | Yd} ~ nd} ~ wwxYw|j D]\} } t|j|ddddd} |d| dt || dtjfY#t$r} | | Yd} ~ }d} ~ wwxYw|rbt|dkr,tj|}|||||d|ddS) Nrrrrrfallback_api_keyr/runsrLr)r>rAr[)rhr)rrk_apply_auth_overridesrrr/rrrqr@rrrrpLangSmithExceptionGroupr )rrrrkrrrrerrors use_overridetarget_api_urlrrM write_api_url write_api_keyexception_groups rerzClient._create_run_non_batch sYGWk=&QRR . %$4 N+ '#+!%G !))%,,, +J 7 7#*$$ (>@* ! ! ! a         !150D0J0J0L0L % %, }%3 -$("&&*#)-G--(///$/ $;$;'.(($,#B"D.!%%%MM!$$$$$$$$%  6{{Q"*"Bf"U"U"U33ODDD%%33F1I>>>Qi  s17A;; B$BB$AD E#D==Ec|jduriS|jr6tjt |}||S|jdur|S||SNTF)rrr&loadsr)rr json_inputss rerzClient._hide_run_inputs' sr   $ $I   1!- F(;(;<>L##L11 1   & &N!!'***ric|jr||S|d|i}t|tr d|vr|dS|S)a<Apply the configured anonymizer to a run's error string. Unlike inputs/outputs, ``error`` is a plain string (``repr(exc)`` plus a formatted traceback, see ``run_helpers._format_error_with_exceptions_to_handle``) that can capture credentials the user never explicitly logged -- e.g. an HTTP client exception whose request-object repr includes an ``Authorization`` header. Route it through the same anonymizer as inputs/outputs so secrets are scrubbed client-side before upload. The value is wrapped as ``{"error": ...}`` so that both ``create_secret_anonymizer`` and a user-supplied anonymizer matching the documented ``Callable[[dict], dict]`` signature receive a dict (a bare string would break a dict-typed anonymizer), then unwrapped. Nr)rr r )rrscrubbeds rerzClient._hide_run_error; s\ 5=L##We$455 h % % %'X*=*=G$ $ rirCc|jduriS|jr6tjt |}||S|jdur|S||Sr&)rrr&r'r)rrC json_metadatas rerzClient._hide_run_metadataQ sr  $ & &I   3#M+h*?*?@@M##M22 2  % ' 'O""8,,,rirOptional[Sequence[dict]]Optional[list[dict]]c"|s|Sd|DS)aFilter content from new_token events. This prevents streaming LLM output from being uploaded via events. Args: events: The events to filter. Returns: The filtered events with kwargs removed from new_token events. cg|];}|ddkrd|Dn|Client._filter_new_token_events...l s# = = =daqH}}Q}}}ri)r{r)r|events rerZz3Client._filter_new_token_events..k s_   yy  K// > =ekkmm = = = =   rirh)rs rerzClient._filter_new_token_events[ s4 M       ric|jsdS|jt|j|jkrdS|j+t j|jz }||jdz krdSdS)zDCheck if the run ops buffer should be flushed based on size or time.FNTrZ)r$rrprr^r)rtime_since_last_flushs rerz#Client._should_flush_run_ops_bufferr sw# 5  % 1D())T-FFF4  * 6$(IKK$2V$V !$)H4)OPPturicb|jsdSt|j}|jtj|_ddlm}m} ||||}|j |j |dSdS#t$r|||YdSwxYw)z8Process and flush run ops buffer in a background thread.Nr)LANGSMITH_CLIENT_THREAD_POOL_process_buffered_run_ops_batch) r$rclearr^r&langsmith._internal._background_threadr:r;submitr#r RuntimeError)rbatch_to_processr:r;futures rerzClient._flush_run_ops_buffer s#  F  455 ""$$$/3y{{,        D188/7GF}( !!&))))))( D D D + +D2B C C C C C C Ds8BB.-B.ropslist[SerializedRunOperation]c ggd}|D]} t| trRtj| j} | jrtj| j| d<| jrtj| j| d<| jrtj| j| d<| j rtj| j | d<| j rtj| j | d<| j rtj| j | d<| j rtd|| jd | jd | jtj| fkt| t(rtd t d t+| |j} |jp"| jpid pt4} t7jt:} t7jt:}d}t=t:t>dddgD]4}t7j ||}|r|dkr|tC|ddz| krx|"tj| d|dd#||||||||d}| $|$|%\}}|tC|z }| |tj|||||6|red#d|&D}|"tj| d|z||||||dSdS)N)rrrrrrrrzBAttachments are not supported when use_multipart_endpoint is Falseztrace=z,id=z;Feedback operations are not supported in non-multipart modez&Unknown item type in tracing queue: %sr9rrrr[rVr.; )rCrkrrrrrc3PK|]!\}}|dd|V"dS)r.rEN)rr5s rer z/Client._batch_ingest_run_ops.. s>WW41a1 6 6 ! 6 6WWWWWWri)'r r=r&r'_nonerFragmentrrrrrrrarb operationrrrrr<rrr)r:r{r2 collections defaultdictrrrdequerp_post_batch_ingest_runsrr<popleftr)rrBrkrrrrrids_and_partial_bodyop curr_dictrr9 body_chunks context_ids body_sizer} body_dequecurr_id curr_bodyrs re_batch_ingest_run_opszClient._batch_ingest_run_ops s>    Q QB"455 Q#M"(33 9F*1*:29*E*EIh':H+2+;BJ+G+GIi(9F*1*:29*E*EIh'8D)0)9"()C)CIg&8D)0)9"()C)CIg&=N.5.>r}.M.MIl+>NN#%R\2996bk66ru66 i8P8PQB ;<< QQ EtBxxPPPPy  & !(.B334FGG !   ;F:QRV:W:W :E:QRV:W:W  W_568IJJ 1 1C$*+?+DEEJ 1MM!C 1 a(8$9$99g|]}|dS)Frrr|rrs rerZz,Client.batch_ingest_runs..i s,OOO++Ce+<<OOOric@g|]}|ddS)TF)rDrr]r^s rerZz,Client.batch_ingest_runs..j s;   BED  Du  = =   rirrz:Batch ingest requires trace_id and dotted_order to be set.c36K|]}td|VdSrNrAr|rs rer z+Client.batch_ingest_runs.. s-QQS+FC88QQQQQQric36K|]}td|VdSrNrbrcs rer z+Client.batch_ingest_runs.. s-RRc+GS99RRRRRRri)rrr{rrrrrrr=r>rrmrX)rrYrD create_dicts update_dictsrserialized_opss` rebatch_ingest_runszClient.batch_ingest_runs s>~ f  F**6+>Y??**6+>Yd*KK f  FOOOOOOO     IO      C77:&& cggn.E.E 1P    C77:&& cggn.E.E 1P   ) R R#D$B$B<$P$PQQ  R#D$B$B<$P$PQQ     !<=== ' ( /OQQLQQQRR\RRR      "">22222ric t|||||g} | r/|pj} | tij|||||jfg} n%fdjD} | D]\} } d| d|| dtj fd|4#t$r} tj t| | }d|}t d |n:#t$r-t d t%| YnwxYw| Yd} ~ d} ~ wwxYwdS) Nrc Tg|]$\}}|tij|dddddf%SNrrrr|r!target_api_keyrs rerZz2Client._post_batch_ingest_runs.. b3NN#))4=) .$("&&*#)- rirz /runs/batchrr)r>rAr?rCrzFailed to batch ingest runs: )rrkrrrrrr/rrqr@ tracebackformat_exception_onlyrrrnrarbrr )rrrCrkrrrrrr r! endpointsrrMexc_desc_linesexc_descs` rerMzClient._post_batch_ingest_runs sGWk=&QRR  $4 N#))4=) '$/"+&3%)-  II7;6J6P6P6R6RI (1 7 7 #NG 7))%222 $#*$$ (>@'(%*     7 7 7N%.%DT!WWa%P%PN!ww~66==??HNN#M8#M#MNNNN NNNNN#L477#L#LMMMMMN33A66666666 7 7 7s75,B"" E--A&DE(4E E( E  E((E-@list[Union[SerializedRunOperation, SerializedFeedbackOperation]]c ,g}i} |D]} t| tr=t| \} } || | | Tt| t r#|t | tdt| t|} | r | | ||||||nF#tj $r4d|_d|D}|r||||||||YnwxYwt!t#| dS#t!t#| wxYwdS)Nz+Unknown operation type in tracing queue: %srTc<g|]}t|t|Srh)r r=)r|rPs rerZz0Client._multipart_ingest_ops.. s(VVV"z">T/U/UV2VVVri)r r=rCrrDr<rBrarrr;_send_multipart_reqrrpr,rXrrr)rrBrkrrrrrpartsopened_files_dictrPpartr acc_multipartrun_opss re_multipart_ingest_opszClient._multipart_ingest_ops s13:< V VB"455 VLBOO  T"""!((6666B ;<< V PQSTT JDQSHHUUUU8??  ? ?((!## +'"/!)2   ,0(VVVVV .. ' '$/"+&3%/ "T"3":":"<"<==>>>>> T"3":":"<"<==>>>>9 ? ?s+C+*E!+AD.+E!-D..E!!0F)rc  |s|sdS|pt}|ptd}|s|sdSfd|D}fd|D}|rD|D]@}|dr|dstjdA~|rD|D]@}|dr|dstjd A~|re|rcd |D}g}|D]O}|d |vr.|D]\} } | | ||d | <:||P~|}|s|sdS||tttj d |Dd |D} | D],} t| trt| j|-| dS)a Batch ingest/upsert multiple runs in the Langsmith system. Args: create (Optional[Sequence[Union[ls_schemas.Run, RunLikeDict]]]): A sequence of `Run` objects or equivalent dictionaries representing runs to be created / posted. update (Optional[Sequence[Union[ls_schemas.Run, RunLikeDict]]]): A sequence of `Run` objects or equivalent dictionaries representing runs that have already been created and should be updated / patched. Raises: LangsmithAPIError: If there is an error in the API request. !!! note The run objects MUST contain the `dotted_order` and `trace_id` fields to be accepted by the API. Example: ```python from langsmith import Client import datetime from uuid import uuid4 client = Client() _session = "__test_batch_ingest_runs" trace_id = uuid4() trace_id_2 = uuid4() run_id_2 = uuid4() current_time = datetime.datetime.now(datetime.timezone.utc).strftime( "%Y%m%dT%H%M%S%fZ" ) later_time = ( datetime.datetime.now(datetime.timezone.utc) + timedelta(seconds=1) ).strftime("%Y%m%dT%H%M%S%fZ") runs_to_create = [ { "id": str(trace_id), "session_name": _session, "name": "run 1", "run_type": "chain", "dotted_order": f"{current_time}{str(trace_id)}", "trace_id": str(trace_id), "inputs": {"input1": 1, "input2": 2}, "outputs": {"output1": 3, "output2": 4}, }, { "id": str(trace_id_2), "session_name": _session, "name": "run 3", "run_type": "chain", "dotted_order": f"{current_time}{str(trace_id_2)}", "trace_id": str(trace_id_2), "inputs": {"input1": 1, "input2": 2}, "error": "error", }, { "id": str(run_id_2), "session_name": _session, "name": "run 2", "run_type": "chain", "dotted_order": f"{current_time}{str(trace_id)}." f"{later_time}{str(run_id_2)}", "trace_id": str(trace_id), "parent_run_id": str(trace_id), "inputs": {"input1": 5, "input2": 6}, }, ] runs_to_update = [ { "id": str(run_id_2), "dotted_order": f"{current_time}{str(trace_id)}." f"{later_time}{str(run_id_2)}", "trace_id": str(trace_id), "parent_run_id": str(trace_id), "outputs": {"output1": 4, "output2": 5}, }, ] client.multipart_ingest(create=runs_to_create, update=runs_to_update) ``` NTrc:g|]}|Srhr]r^s rerZz+Client.multipart_ingest..v s'CCCS++C00CCCric>g|]}|dS)T)rDr]r^s rerZz+Client.multipart_ingest..w s,PPP#++C+==PPPrirrzNMultipart ingest requires trace_id and dotted_order to be set in create dicts.zNMultipart ingest requires trace_id and dotted_order to be set in update dicts.c i|] }|d| Srrhrcs rerz+Client.multipart_ingest.. sCCCsCIsCCCrirc36K|]}td|VdSrarbrcs rer z*Client.multipart_ingest.. s-MM'44MMMMMMric36K|]}td|VdSrerbrcs rer z*Client.multipart_ingest.. s-NN#'55NNNNNNri)rrr{rrrrrr>rrrmr r=rrr) rrYrDrrfrgr create_by_idstandalone_updatesrrrhrPs ` remultipart_ingestzClient.multipart_ingest s| &  F**6+>Y??**6+>Yd*KK f  FCCCCFCCC PPPPPPP  #  wwz**#''.2I2I"56   #  wwz**#''.2I2I"56   .L .CClCCCL-/ #  t9 ,, # ;;1=9:LT3A6;'--c2222-L L  F   ...   ...= MM MMMNNNNN    !  B"455 .N$@ "">22222rir)attemptsrkrrrrraccr:rc |j|j} t|||||g} | r/|pj} | t ij|||||jfg} n%fdjD} | D]\} } td|dzD]} tj t}|j dkr|n|d| di| d|jit"d d| n`#t$j$rYnMt$jt$jt$jf$r5}||kr t.d ||}nYd}~Yd}~nd}~wt2$r} t5jt9||}d |}t.d |n:#t2$r-t.d t?|YnwxYw|}Yd}~nd}~wwxYw fd ddti!|dS)Nrc Tg|]$\}}|tij|dddddf%Srlrmrns rerZz.Client._send_multipart_req.. rprir[boundary-1r/runs/multipartrNrLrrr>r?rCz!Failed to multipart ingest runs: rcttrn,tjtS)Nr)r rrqtb_multipartMultipartEncoderr1 to_string)rLrzsrez,Client._send_multipart_req..sA%dE22&+<!I#)++ rimultipart/form-data; boundary=)"rzrrrkrrrrrrcrrr1rprr/ content_type_TRACING_SEND_TIMEOUTrrqr`rarbrarbr@rqrrrrrnrrr )rrrrkrrrrrrCr r!rsheaders_for_endpointryencoderr _fail_excrMrtrurLrzs` @@reryzClient._send_multipart_req se ;GWk=&QRR  $4 N#))4=) '$/"+&3%)-  II7;6J6P6P6R6RI 5>7 7  0N0Q1 --6 6 (",=eiXXXG{j00&0022&--):::$(("6( .0D(((= ((,-!).   E6EE54. ! ! ! h'P3'P'PQQQ/2  " !"""V)2)HaRS)T)T#%77>#:#:#A#A#C#C'U8'U'UVVVV$VVV'T4PQ77'T'TUUUUUV !IIIIII"''$%Qi%Q%QR   33I>>>o7 7 sJA.D  H/%H/&E44 H/A&G)(H*)4H H*H H**H/r data_stream io.BytesIOcompressed_traces_infoOptional[tuple[int, int]]c Xdt|dg}|jD]j\}}|dt d|dzD]:} t |j|ddddd}dt|d<d |d <|rt|dnd |d <|rt|dnd |d <t d|| d|d||tdd|n#tj$rYnztjtjtjf$rK} ||kr td| | } n|dYd} ~ (Yd} ~ nd} ~ wt($r} t+jt/| | } d | } td| n:#t($r-tdt3| YnwxYw| } Yd} ~ nd} ~ wwxYw|d||jdtd d|| ldS)zASend a zstd-compressed multipart form data stream to the backend.rErrr[NrrrNzstdContent-EncodingrzX-Pre-Compressed-SizezX-Post-Compressed-Sizez3Sending compressed multipart request with context: rrrrz,Failed to send compressed multipart ingest: )rNr)rrrrrrcrrr1rWrarr/rrrqr`rarbrbr@rqrrrrnrrrr )rrrrrCrkrryrrrrMrtrus re_send_compressed_multipart_reqz%Client._send_compressed_multipart_reqs '+y""E"EFF $ 4 : : < <L L  GW   Q   Q1 --I I ="3 '$("&&*#)-GEDDN+39G./:PX215666VX34;QX215666VX45LLXhXX--"333$/'.'<(( ,-!).   E6EE54. ! ! ! hP3PP03 #((+++ " ! " " " )2)HaRS)T)T#%77>#:#:#A#A#C#CU8UU%T4PQ77TT!"IIIIII "  ###''$(T(T(T,233I>>>YL L sJ0B(DI.%I;F I'A&H I4I II II)rrrend_timerrrrrtagsrrreference_example_idrrkrrrrrrOptional[RUN_TYPE_T]rOptional[datetime.datetime]rrrOptional[list[str]]r Optional[ls_schemas.Attachments]rstr | uuid.UUID | Nonec t|d|||dd|dd|dd| | |dd|ddd }|||d <| rt| | | |d <|jdus |jduo|dduo |ddu}||gd sdS|||d <nCtjtj j |d <|| ||d<|| ||d<|.|stj|}|||d<| || |d<|drQ|ddx}r|||dd<||g|||d<|jr|dso|j5|jd|||||||df|r| ddddS#1swxYwYdS||||||||dS)a Update a run in the LangSmith API. Args: run_id (Union[UUID, str]): The ID of the run to update. name (Optional[str]): The name of the run. run_type (Optional[str]): The type of the run (e.g., llm, chain, tool). start_time (Optional[datetime.datetime]): The start time of the run. end_time (Optional[datetime.datetime]): The end time of the run. error (Optional[str]): The error message of the run. inputs (Optional[Dict]): The input values for the run. outputs (Optional[Dict]): The output values for the run. events (Optional[Sequence[dict]]): The events for the run. extra (Optional[Dict]): The extra information for the run. tags (Optional[List[str]]): The tags for the run. attachments (Optional[Dict[str, Attachment]]): A dictionary of attachments to add to the run. The keys are the attachment names, and the values are Attachment objects containing the data and mime type. reference_example_id (Optional[Union[str, uuid.UUID]]): ID of the example that was the source of the run inputs. Used for runs that were part of an experiment. api_key (Optional[str]): The API key to use for this specific run. api_url (Optional[str]): The API URL to use for this specific run. service_key (Optional[str]): The service JWT key for service-to-service auth. tenant_id (Optional[str]): The tenant ID for multi-tenant requests. authorization (Optional[str]): The Authorization header value. cookie (Optional[str]): The Cookie header value. **kwargs (Any): Kwargs are ignored. Returns: None Examples: ```python from langsmith import Client import datetime from uuid import uuid4 client = Client() project_name = "__test_update_run" start_time = datetime.datetime.now() revision_id = uuid4() run: dict = dict( id=uuid4(), name="test_run", run_type="llm", inputs={"text": "hello world"}, project_name=project_name, api_url=os.getenv("LANGCHAIN_ENDPOINT"), start_time=start_time, extra={"extra": "extra"}, revision_id=revision_id, ) # Create the run client.create_run(**run) run["outputs"] = {"output": ["Hi"]} run["extra"]["foo"] = "bar" run["name"] = "test_run_updated" # Update the run client.update_run(run["id"], **run) ``` rrN parent_run_idrrr) rrrrrrrrrrrrTrrrrrrrrCrrrrr)rarrrrr!rrrcrrrrrrrrr{rrrr%r$rrr _update_run)rrrrrrrrrrrrrrrrrkrrrrrrL use_multipartrCs re update_runzClient.update_runvsn68,,  :t44#ZZ>>"JJ~t<< **\488"JJ~t<<   !!+!5!5!7!7D   . *;8T U U U"-D   t + Qt/ET/Q 1Z , 1^$D0  (($t(<<  F  '1133D  '044X5F5JKKUUWWD    0077DM  !226::DN    9"/88"44W==DO  !::6BBDN = -=,,Z888x N,0,C,CH,M,MW j)  $ $dV , , ,  ++?D' (  ) &**=V2W2W *  $++'.'.+6)2-:&,     44661..000%                  (   '#+      sAJ))J-0J- run_updatecz|jdus |jduo|dduo |ddu}|j|j|dS|rt d|} |j||||||t | \} } t dd| j|jj 5|j tdt| |jt} | r.|jxjdz c_|j dddn #1swxYwYt!t#| dS|jt dd | j| j|j]|t/|d| ||||||||j dS|t/|d| |||||| dSdS|||||||| dS) Nrrr)rIpayloadrRr rr[r r r)rr!r rrArCrar rrr"rr?r1rrrrrrrr&rr*r(r'r_update_run_non_batch) rrrrkrrrrrrrrrs rerzClient._update_runsA t + Qt/ET/Q 7:&d2 7>*$6    (   ( ( 4 4 4 4 4 Q .*UUUM&2OO'%!)N L!"  K"* +0 9 919(^ D&.!  H  9.::a?::266888 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9T,"5"5"7"78899999#/ E!*!$  %1++(&~6)$+$+(3&/*7#))-)B)B $ 0 A A C C**    ++(&~6)$+$+(3&/*7#)        10/J  & &'#+ '     s0A"DD"%D"c t|||||g}|r^|p|j} t|j||||||j} |d| d|dt || ddS|jD]S\} } t|j| ddddd} |d| d|dt || dTdS)NrPATCH/runs/rrr) rrkrrrr/rrr) rrrrkrrrrr r!rr"r#s rerzClient._update_run_non_batchsfGWk=&QRR ( $4 N+ '#+!%G  % %!;;D)9;;' 33&   &     150D0J0J0L0L  , }/M) $""&%)))$>>Jt,<>> +J 7 7#*$$*  rirc|jdS|jtdddlm}m}||dd\}}|dd} ||j|||}|j|n(#t$r||||YnwxYw|jrIt|j}tj ||\} } |j | dSdS) aForce flush the currently buffered compressed runs. Args: attempts: Retry attempts for the send request. timeout: Maximum seconds to wait for in-flight sends to complete. None (default) waits indefinitely. Bounds only the wait on already-submitted sends; the synchronous drain and submit steps above are not bounded by this timeout. Nz>Run compression is enabled but request pool futures is not setr)r:'_tracing_thread_drain_compressed_bufferr[)r<r9rr)r!r#rr=r:rr>rrr?rcfwaitdifference_update) rrrr:rfinal_data_streamrrAfuturesdoners reflush_compressed_traceszClient.flush_compressed_tracessy  ! ) F = P           4 3 Q     "  (F 5<<7%*% =  !!&))))   33%'=4  = 24=))Gggw777GD! M + +D 1 1 1 1 1  2 2s8A77"BBc|tj|znd}|jr:|j5|jr|dddn #1swxYwY|j||jnx|jj5|jj rH|tjz }|dkrn+|jj ||jj Hdddn #1swxYwY|j @|$td|tjz nd}| |dSdS)aFlush the tracing queue and compressed buffer. Args: timeout: Maximum seconds to wait for pending traces to drain. None (default) waits indefinitely. A timeout of 0 returns immediately without waiting. NrrQr)r^ monotonicrr%r$rrrall_tasks_doneunfinished_tasksrr!rtr)rrdeadlinewait_for remainings reflushz Client.flushs291D4>##g--$  ) 1* 1 1'1..000 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1   )"''))))'6II,=I#+dn.>.>#>#q==!*9>>xHHH ,=IIIIIIIIIIIIIIII  ! -9A9MCDN$4$44555SW   ( ( ( ; ; ; ; ; . -s$AAA AC,,C03C0ls_schemas.RuncFtj|jj}|tjjkrt jdt5| d|j |j }dddn #1swxYwYtj t}i}t|j}t#|dD]}|jt jd|jdd |jd D}||vr:|j|jkr*||j||||j<||jg|_|D]\} } | || _|S) aLoad child runs for a given run. Args: run (Run): The run to load child runs for. Returns: Run: The run with loaded child runs. Raises: LangSmithError: If a child run has no parent, or on SmithDB-only backends (no ClickHouse query support), where loading child runs isn't supported. uLoading child runs is not supported on SmithDB-only backends (no ClickHouse query support). See https://docs.langchain.com/langsmith/smithdb-sdk-migration#load-a-run’s-child-runsF)is_rootrrNc|jSry)rrs rerz)Client._load_child_runs..5s!.rir}z Child run z has no parentcNh|]"}d|v|ddd#S)Zr[)r`)r|segs re z*Client._load_child_runs..;s<#:: #q!!!$::ri.)r(rGrrrFrrrB_suppress_deprecation_warning list_runsrrrJrKrrWrrrrr`rr child_runsr) rrbackendrtreemapr run_id_str child_run ancestor_idsrchildrens re_load_child_runszClient._load_child_runss&7 8PQQ )6C C C)- + , ,  #.3<(J                 #D ) ) 13[[  ((    / /I&.-.W9<.W.W.WXXX$177<<L \))ilcf.D.D /077 BBB%.Y\" SVR00 '  / / FH&.DL # # s#BB  B zread_run() is deprecated and will be removed after Jan 31, 2027. Use client.runs.retrieve() instead. See https://docs.langchain.com/langsmith/smithdb-sdk-migration#runs-retrieve for the migration guide. project_idrload_child_runsrOptional[ID_TYPE]cjt|d}tj|jj}|tjjkr|dd|}t| d|j }tj dd|i| d|j i} |r|| } | S|tjd |tjd |rtjd tj||t|d | S)aoRead a run from the LangSmith API. .. admonition:: Deprecated Use :meth:`langsmith.Client.runs.retrieve` instead. See https://docs.langchain.com/langsmith/smithdb-sdk-migration#runs-retrieve for the migration guide. Will be removed after Jan 31, 2027. Args: run_id (Union[UUID, str]): The ID of the run to read. load_child_runs (bool, default=False): Whether to load nested child runs. **Deprecated**: this will be removed in a future release, and raises on SmithDB-only backends (no ClickHouse query support), where child runs are never loaded. project_id (Optional[Union[UUID, str]]): The ID of the project (session) that owns the run. Required on SmithDB-only backends (no ClickHouse query support). start_time (Optional[datetime]): The run's start time. Required on SmithDB-only backends (no ClickHouse query support). Returns: Run: The run read from the LangSmith API. Examples: ```python from langsmith import Client # Existing run run_id = "your-run-id" client = Client() stored_run = client.read_run(run_id) ``` rr-rs3_urlsattachments_keyrkrrNzTread_run requires project_id on SmithDB-only backends (no ClickHouse query support).zTread_run requires start_time on SmithDB-only backends (no ClickHouse query support).uload_child_runs is not supported on SmithDB-only backends (no ClickHouse query support). See https://docs.langchain.com/langsmith/smithdb-sdk-migration#load-a-run’s-child-runsrrrh)rar(rGrrrFrr//_convert_stored_attachments_to_attachments_dictrrkr{rrrrrB _read_run_v2) rrrrrrun_id_rr2rrs reread_runzClient.read_runHs|f68,,%7 8PQQ )6C C C008J8J8JKKHI DLK.'+3==??FJnC 1++C00J  )2   )2   )-  #/   L99!     rizread_thread() is deprecated and will be removed after Jan 31, 2027. Use client.threads.list_traces() instead. See https://docs.langchain.com/langsmith/smithdb-sdk-migration#threads-list-traces for the migration guide.ascrrrrselectfilterorderr+Optional[Union[ID_TYPE, Sequence[ID_TYPE]]]#Optional[Union[str, Sequence[str]]]rrrOptional[Sequence[str]]rrLiteral['asc', 'desc']Iterator[ls_schemas.Run]c  |s|stdtjt|} d| d} |r d| d|dn| } t 5|jd|||||| |d| cdddS#1swxYwYdS) aRead runs for a single thread. .. admonition:: Deprecated Use :meth:`langsmith.Client.threads.list_traces` instead. See https://docs.langchain.com/langsmith/smithdb-sdk-migration#threads-list-traces for the migration guide. Will be removed after Jan 31, 2027. Args: thread_id: Thread id (required). project_id: Project id(s) (required when not using project_name). project_name: Project name(s) (required when not using project_id). is_root: If True, return only root runs. Default True. limit: Maximum number of runs to return. select: Fields to select. filter: Additional filter expression. order: Sort order for runs (e.g. "asc" for chronological). Default "asc". **kwargs: Additional arguments passed to the runs query. Yields: Runs in the thread. Examples: ```python for run in client.read_thread( thread_id="thread_abc123", project_name="My Project", limit=50, ): print(run.id) ``` z-thread_id requires project_id or project_namezeq(thread_id, r zand(rrNrh)rrrrWrr) rrrrrrrrrrthread_id_escaped thread_filtercombined_filters re read_threadzClient.read_threadsf Nl NLMM M Js9~~66=):=== ?EX;;;&;;;;=+ , ,  !4> %)&                      sA88A<?A<zlist_runs() is deprecated and will be removed after Jan 31, 2027. Use client.runs.query() instead. See https://docs.langchain.com/langsmith/smithdb-sdk-migration#runs-query for the migration guide.)rrrrrqueryr trace_filter tree_filterrrrrrun_idsrrrrrrrrOptional[Sequence[ID_TYPE]]c+Kg}t|tjtfr||n*t|t r|||9t|tr|g}|fd|Dgd}|p|}d|vrtjdt|r|nd||r|gnd|||| | | | r| nd| ||||d|}d| D}t d| D]H\}}t|d j }t!jdd |i|d jiV| |dz|krdSIdS)aList runs from the LangSmith API. .. admonition:: Deprecated Use :meth:`langsmith.Client.runs.query` instead. See https://docs.langchain.com/langsmith/smithdb-sdk-migration#runs-query for the migration guide. Will be removed after Jan 31, 2027. Args: project_id: The ID(s) of the project to filter by. project_name: The name(s) of the project to filter by. run_type: The type of the runs to filter by. trace_id: The ID of the trace to filter by. reference_example_id: The ID of the reference example to filter by. query: The query string to filter by. filter: The filter string to filter by. trace_filter: Filter to apply to the ROOT run in the trace tree. This is meant to be used in conjunction with the regular `filter` parameter to let you filter runs by attributes of the root run within a trace. tree_filter: Filter to apply to OTHER runs in the trace tree, including sibling and child runs. This is meant to be used in conjunction with the regular `filter` parameter to let you filter runs by attributes of any run within a trace. is_root: Whether to filter by root runs. parent_run_id: The ID of the parent run to filter by. start_time: The start time to filter by. error: Whether to filter by error status. run_ids: The IDs of the runs to filter by. select: The fields to select. limit: The maximum number of runs to return. **kwargs: Additional keyword arguments. Yields: The runs. Examples: ```python # List all runs in a project project_runs = client.list_runs(project_name="") # List LLM and Chat runs in the last 24 hours todays_llm_runs = client.list_runs( project_name="", start_time=datetime.now() - timedelta(days=1), run_type="llm", ) # List root traces in a project root_runs = client.list_runs(project_name="", is_root=1) # List runs without errors correct_runs = client.list_runs(project_name="", error=False) # List runs and only return their inputs/outputs (to speed up the query) input_output_runs = client.list_runs( project_name="", select=["inputs", "outputs"] ) # List runs by run ID run_ids = [ "a36092d2-4ad5-4fb4-9c0d-0dba9a2ed836", "9398e6be-964f-4aa4-8ae9-ad78cd4b7074", ] selected_runs = client.list_runs(id=run_ids) # List all "chain" type runs that took more than 10 seconds and had # `total_tokens` greater than 5000 chain_runs = client.list_runs( project_name="", filter='and(eq(run_type, "chain"), gt(latency, 10), gt(total_tokens, 5000))', ) # List all runs called "extractor" whose root of the trace was assigned feedback "user_score" score of 1 good_extractor_runs = client.list_runs( project_name="", filter='eq(name, "extractor")', trace_filter='and(eq(feedback_key, "user_score"), eq(feedback_score, 1))', ) # List all runs that started after a specific timestamp and either have "error" not equal to null or a "Correctness" feedback score equal to 0 complex_runs = client.list_runs( project_name="", filter='and(gt(start_time, "2023-07-15T12:34:56Z"), or(neq(error, null), and(eq(feedback_key, "Correctness"), eq(feedback_score, 0.0))))', ) # List all runs where `tags` include "experimental" or "beta" and `latency` is greater than 2 seconds tagged_runs = client.list_runs( project_name="", filter='and(or(has(tags, "experimental"), has(tags, "beta")), gt(latency, 2))', ) ``` NcFg|]}|jSr) read_projectr)r|rrs rerZz$Client.list_runs..ls-RRRT"""558RRRri)app_pathcompletion_costcompletion_tokensrrrrrfeedback_statsfirst_token_timerrrrrparent_run_ids prompt_cost prompt_tokensrrrrstatusr total_cost total_tokensr child_run_idszOThe child_run_ids field is deprecated and will be removed in following versions)rsrreference_examplerrrrr parent_runrrrrrrci|] \}}||| Sryrhr5s rerz$Client.list_runs..KKKtq!Q]a]]]ri /runs/queryrrrrrr[rh)r r[r\rWrrextendrrrrr enumeraterrrkr{rr)rrrrrrrrrrrrrrrrrr project_idsdefault_select body_queryirrs` rerzClient.list_runssKv j49c"2 3 3 +   z * * * *  D ) ) +   z * * *  #,,, . ,~   RRRR\RRR      <)> f $ $ Ma"    '2;{{t *>H%&&D(&'4>H*..000D#& & $%& (LKz'7'7'9'9KKK   + +M + K K   FAsJY K.'+.:>.    QUe^^  rizlist_threads() is deprecated and will be removed after Jan 31, 2027. Use client.threads.query() instead. See https://docs.langchain.com/langsmith/smithdb-sdk-migration#threads-query for the migration guide.r)rrrrrrrlist[ListThreadsItem]c ||td||td|||j}|Jtt |d}|Dt jt jjt j dz }|j %| t jj}gd }|gd d d || d } ||| d<d| D} tjt } |d| D]4} | d} | r| | | 5g} | D]\}}|dg}|D]H} t+| d|j}|t/jdd|i| d|jiId|D}| ||t5||rt7|nd|rt9|ndd| dd |dkr | |d} | | d|} | S)apList threads and fetch the runs for each thread. .. admonition:: Deprecated Use :meth:`langsmith.Client.threads.query` instead. See https://docs.langchain.com/langsmith/smithdb-sdk-migration#threads-query for the migration guide. Will be removed after Jan 31, 2027. Args: project_id: The project (session) id. project_name: The project name (alternative to project_id). limit: Maximum number of threads to return. Default None (no limit). offset: Pagination offset for threads. Default 0. filter: Optional filter for threads and runs. start_time: Only include runs from this time. Default: 1 day ago. Returns: List of thread items, each with "thread_id", "runs", "count", "min_start_time", and "max_start_time". Nz2Either project_id or project_name must be providedz1Provide exactly one of project_id or project_namerrr[)daystzinfo)rrrrrrrrrrrrrrrrr r Trjdesc)rsrrrrrrci|] \}}||| Sryrhr5s rerz'Client.list_threads.. rrirrrc^|dpd|dpdfS)Nrrrrzrs rerz%Client.list_threads..s1EE,''-2EE.))/Rrirrrrrcbg|],}|dt|d-S)r)r{rWr|rs rerZz'Client.list_threads..%s@&&&55&&2AlO$$222ri)rrrdrrc0|dpdS)Nrrrz)ts rerz%Client.list_threads..5s!%% 0117Rri)r}reverserrh)rrrrWrarrcrr timedeltar%rurrrJrKrrr{rsortrrkr{rrrpminrt)rrrrrrrr run_selectr threads_maprun_dicttidrr run_dictsrr start_timess re list_threadszClient.list_threadss\H  ,"6QRR R  !l&>PQQ Q  #** *EEHJ%%%*l;;<<  !*..!%"***+J   $#++83D3H+IIJ   *#| $..00 & &   #)Jx KKz'7'7'9'9KKK -8-DT-J-J 77 J7WW 2 2H,,{++C 2C ''111(*$/$5$5$7$7   Iy NN     D%  Mi  N$/"#'.&&"&&&K MM!*  YY:E&Oc+&6&6&64:E&Oc+&6&6&64       77     A::FGG_F  FUF^F ri)rrrr project_namesrreference_example_idsrrrrrrrrdata_source_typerOptional[list[ID_TYPE]]rrr8rr9r:cddlm}m}|pg}|rl|5fd|D}||D].}||j/ dddn #1swxYwY|st d|||||||| | | | | |||d}d|D}dd d t|i }tj || S) aGet aggregate statistics over queried runs. Takes in similar query parameters to `list_runs` and returns statistics based on the runs that match the query. Args: id (Optional[List[Union[UUID, str]]]): List of run IDs to filter by. trace (Optional[Union[UUID, str]]): Trace ID to filter by. parent_run (Optional[Union[UUID, str]]): Parent run ID to filter by. run_type (Optional[str]): Run type to filter by. project_names (Optional[List[str]]): List of project names to filter by. project_ids (Optional[List[Union[UUID, str]]]): List of project IDs to filter by. reference_example_ids (Optional[List[Union[UUID, str]]]): List of reference example IDs to filter by. start_time (Optional[str]): Start time to filter by. end_time (Optional[str]): End time to filter by. error (Optional[bool]): Filter by error status. query (Optional[str]): Query string to filter by. filter (Optional[str]): Filter string to apply. trace_filter (Optional[str]): Trace filter string to apply. tree_filter (Optional[str]): Tree filter string to apply. is_root (Optional[bool]): Filter by root run status. data_source_type (Optional[str]): Data source type to filter by. Returns: Dict[str, Any]: A dictionary containing the run statistics. rThreadPoolExecutor as_completedcHg|]}j|Sr)r>r)r|rexecutorrs rerZz(Client.get_run_stats..qs<OOD$5DOIIriNz>At least one of project_names or project_ids must be provided.)rrrrrsrrrrrrrrrr:ci|] \}}||| Sryrhr5s rerz(Client.get_run_stats..sEEEDAqq}1a}}}rirz /runs/statsrLr) concurrent.futuresr>r?rrrrrr/rrr0r)rrrrrr8rr9rrrrrrrrr:r>r?rrArr2rAs` @re get_run_statszClient.get_run_stats>s\ HGGGGGGG!'R  ;##%% ; -+l733;;F&&v}}'9::::;  ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; P $ "!6$ (& 0  &FEGMMOOEEE,,   G,,-   +H555}}sA A22A69A6zget_run_url() is deprecated and will be removed after Jan 31, 2027. Use client.runs.get_url() instead. See https://docs.langchain.com/langsmith/smithdb-sdk-migration#runs-get-url for the migration guide.)rrls_schemas.RunBasec2||||S)aGet the URL for a run. Not recommended for use within your agent runtime. More for use interacting with runs after the fact for data analysis or ETL workloads. .. admonition:: Deprecated Use :meth:`langsmith.Client.runs.get_url` instead. See https://docs.langchain.com/langsmith/smithdb-sdk-migration#runs-get-url for the migration guide. Will be removed after Jan 31, 2027. Args: run (RunBase): The run. project_name (Optional[str]): The name of the project. project_id (Optional[Union[UUID, str]]): The ID of the project. Returns: str: The URL for the run. )rrr)_construct_run_url)rrrrs re get_run_urlzClient.get_run_urls)B&&,:'   rict|ddx}rnt|ddx}r||j}nQ||}nL|||j}n.tj}||j}t |d}|jd|d|d|jdS) zBuild a run's UI URL locally, without calling the backend. Kept for backends that predate the ``/runs/{run_id}/url`` v2 endpoint. rNrrz/o/z /projects/p/z/r/z ?poll=true)rrrrrrar_get_tenant_id)rrrrrr session_id_s rerGzClient._construct_run_urls!lD99 9: I $S.$?? ?\ I** *EEHJJ  ##JJ  %** *EEHJJ#688L** *EEHJz<88 ~ $ $$"5"5"7"7 $ $[ $ $ $ $ $ rizshare_run() is deprecated and will be removed after Jan 31, 2027. Use client.runs.share.create() instead. See https://docs.langchain.com/langsmith/smithdb-sdk-migration#share-and-read-public-runs for the migration guide.)share_idrLc@t|d}t||pttjd}|dd|d|j|}t j||d}|j d|d S) aRGet a share link for a run. .. admonition:: Deprecated Use :meth:`langsmith.Client.runs.share.create` instead. See https://docs.langchain.com/langsmith/smithdb-sdk-migration#share-and-read-public-runs for the migration guide. Will be removed after Jan 31, 2027. Args: run_id (Union[UUID, str]): The ID of the run to share. share_id (Optional[Union[UUID, str]]): Custom share ID. If not provided, a random UUID will be generated. Returns: str: The URL of the shared run. r)r share_tokenPUTr/sharerrrN/public//r) rarWr[rr/rrr0rr)rrrLrrLr2rNs re share_runzClient.share_runs.68,,'ll#8s4:<<'8'8  ,,  $W $ $ $M -   +H555mmoom4 .99+9999rizunshare_run() is deprecated and will be removed after Jan 31, 2027. Use client.runs.share.delete() instead. See https://docs.langchain.com/langsmith/smithdb-sdk-migration#share-and-read-public-runs for the migration guide.c|ddt|dd|j}tj|dS)aDelete share link for a run. .. admonition:: Deprecated Use :meth:`langsmith.Client.runs.share.delete` instead. See https://docs.langchain.com/langsmith/smithdb-sdk-migration#share-and-read-public-runs for the migration guide. Will be removed after Jan 31, 2027. Args: run_id (Union[UUID, str]): The ID of the run to unshare. Returns: None DELETErrrPrNr/rarrr0)rrr2s re unshare_runzClient.unshare_runsW*,,  7Xfh// 7 7 7M-   +H55555rizread_run_shared_link() is deprecated and will be removed after Jan 31, 2027. Use client.runs.retrieve(selects=["SHARE_URL"]) instead. See https://docs.langchain.com/langsmith/smithdb-sdk-migration#share-and-read-public-runs for the migration guide.c|ddt|dd|j}tj||}|d|vrdS|jd|dd S) a-Retrieve the shared link for a specific run. .. admonition:: Deprecated Use :meth:`langsmith.Client.runs.retrieve` with ``selects=["SHARE_URL"]`` instead. See https://docs.langchain.com/langsmith/smithdb-sdk-migration#share-and-read-public-runs for the migration guide. Will be removed after Jan 31, 2027. Args: run_id (Union[UUID, str]): The ID of the run. Returns: Optional[str]: The shared link for the run, or None if the link is not available. r-rrrPrWNrNrRrS)r/rarrr0rr)rrr2rs reread_run_shared_linkzClient.read_run_shared_links,,,  7Xfh// 7 7 7M-   +H555 >]&884.CC&*?CCCCrict5|t|d}dddn #1swxYwY|duS)zGet share state for a run. Args: run_id (Union[UUID, str]): The ID of the run. Returns: bool: True if the run is shared, False otherwise. rN)rr[ra)rrrs re run_is_sharedzClient.run_is_shared>s+ , , I I,,Xfh-G-GHHD I I I I I I I I I I I I I I I4s$?AAzread_shared_run() is deprecated and will be removed after Jan 31, 2027. Use client.public.runs.retrieve() instead. See https://docs.langchain.com/langsmith/smithdb-sdk-migration#share-and-read-public-runs for the migration guide.rNUnion[ID_TYPE, str]c"t|dd\}}d|d}||dt|dz }|d ||j }t j|t jd i|d |j iS) alGet shared runs. .. admonition:: Deprecated Use :meth:`langsmith.Client.public.runs.retrieve` instead. See https://docs.langchain.com/langsmith/smithdb-sdk-migration#share-and-read-public-runs for the migration guide. Will be removed after Jan 31, 2027. Args: share_token (Union[UUID, str]): The share token or URL of the shared run. run_id (Optional[Union[UUID, str]]): The ID of the specific run to retrieve. If not provided, the full shared run will be returned. Returns: Run: The shared run. rrrVrR/runNrmrr-rWrrh) rfrar/rrr0r{rrr)rrNrrrer_r2s reread_shared_runzClient.read_shared_runKs2,K%HHH :*****   42244 4D,,  M-   +H555~JJ JJ4>JJJJrizlist_shared_runs() is deprecated and will be removed after Jan 31, 2027. Use client.public.runs.query() instead. See https://docs.langchain.com/langsmith/smithdb-sdk-migration#share-and-read-public-runs for the migration guide.c#K|rd|ini}t|dd\}}|d|d|D]}tjd i|d|jiVd S) aGet shared runs. .. admonition:: Deprecated Use :meth:`langsmith.Client.public.runs.query` instead. See https://docs.langchain.com/langsmith/smithdb-sdk-migration#share-and-read-public-runs for the migration guide. Will be removed after Jan 31, 2027. Args: share_token (Union[UUID, str]): The share token or URL of the shared run. run_ids (Optional[List[str]]): A list of run IDs to filter the results by. Yields: A shared run. rrrr`rRrrrNrh)rfrr{rr)rrNrrrrers relist_shared_runszClient.list_shared_runsps0#*1gr+K%HHH :22 .z . . .T3   B BC.AA3AA$.AAA A A A A B Bri dataset_name dataset_idrfls_schemas.DatasetShareSchemac ||td|||j}|ddt |dd|j}t j||}ttj i|d |j d t |d d d iS)aRetrieve the shared schema of a dataset. Args: dataset_id (Optional[Union[UUID, str]]): The ID of the dataset. Either `dataset_id` or `dataset_name` must be given. dataset_name (Optional[str]): The name of the dataset. Either `dataset_id` or `dataset_name` must be given. Returns: ls_schemas.DatasetShareSchema: The shared schema of the dataset. Raises: ValueError: If neither `dataset_id` nor `dataset_name` is given. N/Either dataset_id or dataset_name must be givenrer- /datasets/rgrPrWrgrRrNzresponse.share_token/d) r read_datasetrr/rarrr0rrr{DatasetShareSchemar)rrgrfr2ds reread_dataset_shared_schemaz!Client.read_dataset_shared_schemas(  ,"6NOO O  ** *EEHJ,,  C*l;; C C CM-   +H555 MMOO  )  $.JJAm,.DEEJJJ     ric ||td|||j}dt|i}|ddt |dd|j|}tj|| }ttj i|d |j d |d d iS) a Get a share link for a dataset. Args: dataset_id (Optional[Union[UUID, str]]): The ID of the dataset. Either `dataset_id` or `dataset_name` must be given. dataset_name (Optional[str]): The name of the dataset. Either `dataset_id` or `dataset_name` must be given. Returns: ls_schemas.DatasetShareSchema: The shared schema of the dataset. Raises: ValueError: If neither `dataset_id` nor `dataset_name` is given. NrjrergrOrkrPrQrgrRrNrl)rrmrrWr/rarrr0rrr{rnr)rrgrfrLr2ros re share_datasetzClient.share_datasets(  ,"6NOO O  ** *EEHJ #j// ,,  C*l;; C C CM -   +H555--//  ) Iq I%DNHHAm4DHHH I I   ric|ddt|dd|j}tj|dS)zDelete share link for a dataset. Args: dataset_id (Union[UUID, str]): The ID of the dataset to unshare. Returns: None rVrkrgrPrWNrX)rrgr2s reunshare_datasetzClient.unshare_datasetsW,,  C*l;; C C CM-   +H55555rict||j\}}|dd|d|j}t j|t jdi||j d|ddS)zGet shared datasets. Args: share_token (Union[UUID, str]): The share token or URL of the shared dataset. Returns: Dataset: The shared dataset. r-rR /datasetsrWrl)r _public_pathrh) rfrkr/rrr0r{rrr)rrNrrer2s reread_shared_datasetzClient.read_shared_datasets,KFF :,,  ,z , , ,M-   +H555!  mmoo n3K333     ri) example_idsrryIterator[ls_schemas.Example]c #Ki}|d|D|d<t|dt|dd|D]/\}}tjd i|d|jiV| |d z|krdS0dS) aGet shared examples. Args: share_token (Union[UUID, str]): The share token or URL of the shared dataset. example_ids (Optional[List[UUID, str]], optional): The IDs of the examples to filter by. limit (Optional[int]): Maximum number of examples to return, by default None. Returns: List[ls_schemas.Example]: The list of shared examples. Nc,g|]}t|SrhrX)r|rs rerZz/Client.list_shared_examples..s:::CGG:::rirrRrN /examplesrrr[rh)rrrar{Exampler)rrNryrrr examples relist_shared_exampleszClient.list_shared_exampless"  "::k:::F4L#  $ $J8K??JJJ %      JAw $IIwII$.III I I I QUe^^  ri)rr name_containsrdataset_share_tokenr(Iterator[ls_schemas.TracerSessionResult]c#K|||d}t|d}t|d|d|D]/\}} tjd i| d|jiV| |dz|krdS0dS) a<List shared projects. Args: dataset_share_token (str): The share token of the dataset. project_ids (Optional[List[Union[UUID, str]]]): List of project IDs to filter the results, by default None. name (Optional[str]): Name of the project to filter the results, by default None. name_contains (Optional[str]): Substring to search for in project names, by default None. limit (Optional[int]): Maximum number of projects to return, by default None. Yields: The shared projects. )rrrrrRz/datasets/sessionsrrNr[rh)rarrr{TracerSessionResultr) rrrrrrrrNr projects relist_shared_projectszClient.list_shared_projects&s*$TMRR24IJJ #  $ $:;::: %      JAw 0UU7UUdnUUU U U U QUe^^  ri)rrCupsert project_extrareference_dataset_id num_examplesnum_repetitionsevaluator_keysrrrrrrls_schemas.TracerSessionc|jd} |} |r i| pid|i} || |ttjd} i} |rd| d<||| d<||| d<||| d <| r| | d <|d | i|jd d it | | }tj|tj di| d|j iS)aCreate a project on the LangSmith API. Args: project_name (str): The name of the project. project_extra (Optional[dict]): Additional project information. metadata (Optional[dict]): Additional metadata to associate with the project. description (Optional[str]): The description of the project. upsert (bool, default=False): Whether to update the project if it already exists. reference_dataset_id (Optional[Union[UUID, str]): The ID of the reference dataset to associate with the project. num_examples (Optional[int]): The expected number of examples that will be run against this project. Used by the backend to populate experiment progress; sent as a transport-only field and not round-tripped as a response field. num_repetitions (Optional[int]): The number of repetitions per example. Combined with ``num_examples`` to compute expected run count for progress tracking. Transport-only. evaluator_keys (Optional[list[str]]): Feedback keys produced by the row-level evaluators that will run against this project. Used by the backend to populate per-evaluator experiment progress. Transport-only. Returns: TracerSession: The created project. /sessionsrC)rrrrTrNrrrrrrNr)rrLrrrh) rkrWr[rr/rrrr0r{ TracerSessionrr)rrrrCrrrrrrrrrrr2s recreate_projectzClient.create_projectGs@Jl---  <; ;j(;;E &djll##      $#F8   ++?D' (  ##/D  &&5D" #  4%3D! ",,  It}In6HIIT"" -   +H555'TT(--//TTT^TTTTri)rrrCrrcn|jdt|d}|}|r i|pid|i}||||r|ndd} |d|i|jddit |  } t j| tj d i| d |j iS) a,Update a LangSmith project. Args: project_id (Union[UUID, str]): The ID of the project to update. name (Optional[str]): The new name to give the project. This is only valid if the project has been assigned an end_time, meaning it has been completed/closed. description (Optional[str]): The new description to give the project. metadata (Optional[dict]): Additional metadata to associate with the project. project_extra (Optional[dict]): Additional project information. end_time (Optional[datetime.datetime]): The time the project was completed. Returns: TracerSession: The updated project. /sessions/rrCN)rrrrrrNrrrLrrh) rkrarr/rrrr0r{rrr) rrrrrCrrrrrr2s reupdate_projectzClient.update_projects<lRRhz<.P.PRR  <; ;j(;;E&08B**,,,d    ,,  It}In6HIIT"" -   +H555'TT(--//TTT^TTTTrirc|j|jS |ddddi}|}t|trFt |dkr3t jd i|dd|ji}|j |_|jSnA#t$r4}t dt|d Yd}~nd}~wwxYwdS) Nr-rrr[rrrz*Failed to get tenant ID from LangSmith: %sTr rh)rr/rr rrpr{rrrr@rarr)rr2rtracer_sessionrMs rerzClient._get_optional_tenant_ids ? &? " 00{GQ<1H]]__F&$'' 'CKK!OO!+!?""Qi""+/>"""#1":&    LL?? ?ri)rrrr include_statsrls_schemas.TracerSessionResultcd}ddi}||dt|dz }n|||d<ntd||d <|d || }|}t |t rKt |d krtjd |dtj di|d d|j iStj di|d|j iS)aRead a project from the LangSmith API. Args: project_id (Optional[str]): The ID of the project to read. project_name (Optional[str]): The name of the project to read. Only one of project_id or project_name may be given. include_stats (bool, default=False): Whether to include a project's aggregate statistics in the response. Returns: TracerSessionResult: The project. rrr[Nrmrr'Must provide project_name or project_idrr-rrzProject not foundrrh) rarr/rr rrprrpr{rr)rrrrr_rr2rs rerzClient.read_projects7*")1  ! <\::<<
K|]}tj|VdSry)r(_v2_run_to_schemar*s rer z*Client.get_test_results..s.LL'9!<<LLLLLLricH|}d|DS)N)rycjg|]0}d|jid|jpiD1S) example_idc i|] \}}d|| S)z reference.rhr5s rerzNClient.get_test_results..fetch_examples...s'WWWtq!'A''WWWri)rrrr|rs rerZzCClient.get_test_results..fetch_examples..s\ !'*WW8M27T7T7V7VWWWri) list_examples)batchexamplesrs refetch_examplesz/Client.get_test_results..fetch_exampless<))e)<.s$LLL41a|||QLLLric i|] \}}d|| S)zoutputs.rhr5s rerz+Client.get_test_results..s$OOOTQ~!~~qOOOri)execution_timerrc|i|]9\}}|dkr|d d||d:S)notecommentsz feedback.avgrzr5s rerz+Client.get_test_results..sO $1$%KKAEE*4E4EK,OOQUU5\\$/KKrirrnoteszRun z has no reference example ID.c@g|]}|D]}|Srh)r)r|rAoutputs rerZz+Client.get_test_results..sA   V]]__  ;AF    ri) left_index right_indexrecords)orient)2rr UserWarningrCr>r?pandasr(rGrrrFrrrrrrrrrrWrrrrrrrrr total_secondsrr rDr{rrarbrpr> DataFrame set_indexmergejson_normalizerrr rto_dict)rrrr>r?pdrrpagerresultsryrrrrArrrow note_statsrr result_dfexample_outputs example_dfs` reget_test_resultszClient.get_test_results0s.  3[Q     HGGGGGGG%7 8PQQ )6C C C/00  15)!-     2@22               "!l&>!..L.IIL ! !VWWW00227== __-    (0!Qx'8'<   #>E0MLeLLLD           ! !' VXG! $! $ !"8 LLAHN3I3I3K3KLLL POaio25L5L5N5NOOO  :"*+,"al2AACCC!W$   # 4JJ()(8(>(>(@(@&'%5%9%9&%A%AAz4'1~~j'A'AA84+3CL)O&&q'=>>>>NN#M!$#M#M#MNNN{##j0A55 +FVj5H,H IIu-x~u'M'MNNN*,s####677# Vx~{677?STTUUUO' V' V' V' V' V' V' V' V' V' V' V' V' V' V' VP!BL))33LAA   !-g!6!6     W%o66@@NNJ"((tQU(VVI!r d38n%y'8'8 '8'J'J K K   s%+BBBG9MM #M reference_dataset_namereference_freedataset_versionOptional[dict[str, Any]]c #Kd| t| dndi} ||| d<||| d<||| d<||td|| d<n"| || j}|| d<||| d <||| d <||| d <| t j| | d <t |d| D]/\} } tj di| d|j iV| | dz| krdS0dS)a|List projects from the LangSmith API. Args: project_ids (Optional[List[Union[UUID, str]]]): A list of project IDs to filter by, by default None name (Optional[str]): The name of the project to filter by, by default None name_contains (Optional[str]): A string to search for in the project name, by default None reference_dataset_id (Optional[List[Union[UUID, str]]]): A dataset ID to filter by, by default None reference_dataset_name (Optional[str]): The name of the reference dataset to filter by, by default None reference_free (Optional[bool]): Whether to filter for only projects not associated with a dataset. limit (Optional[int]): The maximum number of projects to return, by default None metadata (Optional[Dict[str, Any]]): Metadata to filter by. Yields: The projects. Raises: ValueError: If both reference_dataset_id and reference_dataset_name are given. rNrjrrrzGOnly one of reference_dataset_id or reference_dataset_name may be givenreference_datasetrerrrrCrrrr[rh) r0rrmrrrrrr{rr)rrrrrrrrrrrCrr rs re list_projectszClient.list_projectssP (9S___s"   "&F4L  !F6N  $&3F? #  +%1 ;+?F& ' ' # /#'#4#43$5$$ !+?F& '  %'5F# $  $&3F? #  &(7F$ %  !%H!5!5F: #  $ $[ $ @ @    JAw0UU7UUdnUUU U U U QUe^^   ric|)t||j}n|td|ddt |d|j}tj|dS)amDelete a project from LangSmith. Args: project_name (Optional[str]): The name of the project to delete. project_id (Optional[str]): The ID of the project to delete. Returns: None Raises: ValueError: If neither project_name or project_id is provided. NrrrVrrrW) rWrrrr/rarrr0)rrrr2s redelete_projectzClient.delete_projects$  #T..L.IILMMJJ  FGG G,,  =*l;; = =M-   +H55555ri)rr inputs_schemaoutputs_schematransformationsrCls_schemas.DataTyperrr0Optional[list[ls_schemas.DatasetTransformation]]cdtji|pi}||j|dtji|piddd}|||d<|||d<|||d<|d d i|jd d it j| } tj| | } | dp|| d<tj di| |j |dS)aCreate a dataset in the LangSmith API. Args: dataset_name (str): The name of the dataset. description (Optional[str]): The description of the dataset. data_type (DataType, default=DataType.kv): The data type of the dataset. inputs_schema (Optional[Dict[str, Any]]): The schema definition for the inputs of the dataset. outputs_schema (Optional[Dict[str, Any]]): The schema definition for the outputs of the dataset. transformations (Optional[List[DatasetTransformation]]): A list of transformations to apply to the dataset. metadata (Optional[dict]): Additional metadata to associate with the dataset. Returns: Dataset: The created dataset. Raises: requests.HTTPError: If the request to create the dataset fails. rsdk)rCsource)rrrrNrinputs_schema_definitionoutputs_schema_definitionrrvrNrrrCrrh)rrr~r/rr&rrr0rr{r{rrr) rrfrrrrrrCrRr2 json_responses recreate_datasetzClient.create_dataset,saFv=??THNPRT ".v=??~2  # #   "%0GM "  $2?G. /  %3AG/ 0,,  It}In6HIIw'' -   +H555  $1$5$5j$A$A$MX j!!   n3355     rirfrgcb |||dS#tj$rYdSwxYw)aGCheck whether a dataset exists in your tenant. Args: dataset_name (Optional[str]): The name of the dataset to check. dataset_id (Optional[Union[UUID, str]]): The ID of the dataset to check. Returns: bool: Whether the dataset exists. rTF)rmrrp)rrfrgs re has_datasetzClient.has_datasetusJ"    . s"RRR4:d##RRRrirV)rmrrr/ratextrr`)rrgrfr_r2rRs reread_dataset_openai_finetuningz%Client.read_dataset_openai_finetunings(  !   %** *EEHJJFGG G,,  D Dhz<88 D D D  SR 0C0C0E0E0K0KD0Q0QRRRri) dataset_idsrrfdataset_name_containsrCrrr Iterator[ls_schemas.Dataset]c#nKd|t|dndi}|||d<|||d<|||d<|||d<|tj||d<t|d | D]B\}} t jd i| |j|d V| |d z|krdSCdS)aList the datasets on the LangSmith API. Args: dataset_ids (Optional[List[Union[UUID, str]]]): A list of dataset IDs to filter the results by. data_type (Optional[str]): The data type of the datasets to filter the results by. dataset_name (Optional[str]): The name of the dataset to filter the results by. dataset_name_contains (Optional[str]): A substring to search for in the dataset names. metadata (Optional[Dict[str, Any]]): A dictionary of metadata to filter the results by. limit (Optional[int]): The maximum number of datasets to return. Yields: The datasets. rNrjrrrrrCrvrrr[rh) r0rrrrr{rrr) rrrrfr rCrrr rRs re list_datasetszClient.list_datasets#s)< (9S___s"   "&F4L  "+F;   #)F6N ,&;F? #  !%H!5!5F: #  $ $[ $ @ @   JAw$.7799     QUe^^  ri)rgrfc|||j}|td|ddt |d|j}t j|dS)a&Delete a dataset from the LangSmith API. Args: dataset_id (Optional[Union[UUID, str]]): The ID of the dataset to delete. dataset_name (Optional[str]): The name of the dataset to delete. Returns: None NrerrVrkrgrW)rmrrr/rarrr0)rrgrfr2s redelete_datasetzClient.delete_datasetXs$  #** *EEHJ  EFF F,,  =*l;; = =M-   +H55555rias_ofdatetime.datetimetagc|||j}|td|ddt |dd|j||d }tj|dS) ayUpdate the tags of a dataset. If the tag is already assigned to a different version of this dataset, the tag will be moved to the new version. The as_of parameter is used to determine which version of the dataset to apply the new tags to. It must be an exact version of the dataset to succeed. You can use the read_dataset_version method to find the exact version to apply the tags to. Args: dataset_id (Optional[Union[UUID, str]]): The ID of the dataset to update. dataset_name (Optional[str]): The name of the dataset to update. as_of (datetime.datetime): The timestamp of the dataset to apply the new tags to. tag (str): The new tag to apply to the dataset. Returns: None Examples: ```python dataset_name = "my-dataset" # Get the version of a dataset <= a given timestamp dataset_version = client.read_dataset_version( dataset_name=dataset_name, as_of=datetime.datetime(2024, 1, 1) ) # Assign that version a new tag client.update_dataset_tags( dataset_name="my-dataset", as_of=dataset_version.as_of, tag="prod", ) ``` NrerrOrkrg/tagsrrrQ) rmrrr/rarrrr0rrgrfrrr2s reupdate_dataset_tagzClient.update_dataset_tagusZ  #** *EEHJ  EFF F,,  B*l;; B B BM** -   +H55555ri)rgrfsearchrr#Iterator[ls_schemas.DatasetVersion]c #,K|||j}||t|dndd}t|dt |dd|D]&\}}t jd i|V| |d z|krdS'dS) asList dataset versions. Args: dataset_id (Optional[Union[UUID, str]]): The ID of the dataset. dataset_name (Optional[str]): The name of the dataset. search (Optional[str]): The search query. limit (Optional[int]): The maximum number of versions to return. Yields: The dataset versions. Nrerj)rrrkrgz /versionsrr[rh)rmrr0rrrar{DatasetVersion)rrgrfrrrr rs relist_dataset_versionszClient.list_dataset_versionss&  ** *EEHJ(-(9S___s  $  $ $JXj,??JJJ %      JAw +66g66 6 6 6 QUe^^  ri)rgrfrrls_schemas.DatasetVersionc |||j}|r|s||td|ddt |dd||d }t jd i|S) a6Get dataset version by `as_of` or exact tag. Use this to retrieve the dataset version to a timestamp or for a given tag. Args: dataset_id (Optional[ID_TYPE]): The ID of the dataset. dataset_name (Optional[str]): The name of the dataset. as_of (Optional[datetime.datetime]): The timestamp of the dataset to retrieve. tag (Optional[str]): The tag of the dataset to retrieve. Returns: DatasetVersion: The dataset version. Examples: ```python # Get the latest version of a dataset client.read_dataset_version(dataset_name="my-dataset", tag="latest") # Get the version of a dataset <= a given timestamp client.read_dataset_version( dataset_name="my-dataset", as_of=datetime.datetime(2024, 1, 1), ) # Get the version of a dataset with a specific tag client.read_dataset_version(dataset_name="my-dataset", tag="prod") ``` Nrez/Exactly one of as_of and tag must be specified.r-rkrgz/versionrrrh)rmrrr/rar{rrrs reread_dataset_versionzClient.read_dataset_versionsL  ** *EEHJ  Pc Pu}NOO O,,  E*l;; E E E"3//-   (;;8==??;;;ri)source_api_urlrf token_or_urlrc|p|j}t||\}}t|d}||}|p|j} ||}t d|d|S#tj $rYnwxYw t| |}| ||j |jptjj|j|j|j} |d|Dd|D|j n2#t.$r%} t d |d | d } ~ wwxYw ~n#~wxYw|S) aClone a public dataset to your own langsmith tenant. This operation is idempotent. If you already have a dataset with the given name, this function will do nothing. Args: token_or_url (str): The token of the public dataset to clone. source_api_url (Optional[str]): The URL of the langsmith server where the data is hosted. Defaults to the API URL of your current client. dataset_name (Optional[str]): The name of the dataset to create in your tenant. Defaults to the name of the public dataset. Returns: Dataset: The cloned dataset. placeholder)rkrrerz) already exists in your tenant. Skipping.)rfrrrrrcg|] }|j Srh)rr|rMs rerZz/Client.clone_public_dataset..<s777AH777ricg|] }|j Srh)rr$s rerZz/Client.clone_public_dataset..=s9991QY999ri)rrrgz)An error occurred while creating dataset z . You should delete it manually.N)rkrfrrxrrmrarrrprrrrrr{DataTypekvrrrcreate_examplesrr1r) rr rrfre source_clientdsrrRrMs reclone_public_datasetzClient.clone_public_datasets,(74<%8~%V%V" #!     . .z : :#.rw  "" "==B KKR<RRR   I.    D  M>>zJJKKH)))N,@**=*@ .!0 " 2 *G $$77h77799999&z% !     5 555  "    sC 5BBBA'E(1D32E(3 E"= EE""E((E+c<||}|jS)N)rg)rmr)rrgrRs rerzClient._get_data_typeOs!##z#::  rirL generation created_atls_schemas.Examplec>|d|id|i|||S)aFAdd an example (row) to an LLM-type dataset. Args: prompt (str): The input prompt for the example. generation (Optional[str]): The output generation for the example. dataset_id (Optional[Union[UUID, str]]): The ID of the dataset. dataset_name (Optional[str]): The name of the dataset. created_at (Optional[datetime.datetime]): The creation timestamp of the example. Returns: Example: The created example inputrrrrgrfr.)create_example)rrLr-rgrfr.s recreate_llm_examplezClient.create_llm_exampleSs:4""V$z*!%! #   rimessages:list[Union[Mapping[str, Any], ls_schemas.BaseMessageLike]] generations>Optional[Union[Mapping[str, Any], ls_schemas.BaseMessageLike]]c g}|D]~}tj|r@|tjt t j|V|t t|d}|Vtj|r-tjt t j|}nt t|}|d|i|d|ind|||S)aAdd an example (row) to a Chat-type dataset. Args: messages (List[Union[Mapping[str, Any], BaseMessageLike]]): The input messages for the example. generations (Optional[Union[Mapping[str, Any], BaseMessageLike]]): The output messages for the example. dataset_id (Optional[Union[UUID, str]]): The ID of the dataset. dataset_name (Optional[str]): The name of the dataset. created_at (Optional[datetime.datetime]): The creation timestamp of the example. Returns: Example: The created example Nr1rr2) ris_base_message_likerconvert_langchain_messagerr{BaseMessageLiker r3) rr5r7rgrfr. final_inputmessagefinal_generationss recreate_chat_examplezClient.create_chat_exampleus$8  8 8G,W55 8""6Z7AA ""4g#6#67777   ",[99 <$,$F3[AA%%!!%){$;$;!""[)1B1N,--TX!%!#   ricr|||j}d}||}|tjjkr|jdkrtd|jd tj |j }n-#t$r td|jd|j wxYwd|i}|j sd}n[ tj |j } n-#t$r td |jd |j wxYwd | i}n |tjj kr|jdkrtd|jd  dtj|j i}n-#t$r td |jd|j wxYw|j sd}n d tj|j i}nd#t$r td|jd |j wxYw|tjjkr|j }|j }ntd|d||||||S)aAdd an example (row) to a dataset from a run. Args: run (Run): The run to create an example from. dataset_id (Optional[Union[UUID, str]]): The ID of the dataset. dataset_name (Optional[str]): The name of the dataset. created_at (Optional[datetime.datetime]): The creation timestamp of the example. Returns: Example: The created example Nrernz Run type z+ is not supported for dataset of type 'LLM'z2Error converting LLM run inputs to prompt for run z with inputs r1z7Error converting LLM run outputs to generation for run z with outputs rz, is not supported for dataset of type 'chat'z9Error converting LLM run inputs to chat messages for run z=Error converting LLM run outputs to chat generations for run z Dataset type z not recognized.r2)rmrr r{r&rnrrrget_prompt_from_inputsrrget_llm_generation_from_outputschatget_messages_from_inputs#get_message_generation_from_outputsr'r3) rrrgrfr. dataset_typerLrrr-s recreate_example_from_runzClient.create_example_from_runs5$  ** *EEHJL11*== :.2 2 2|u$$ 1 111 !8DD    :::-0Z::  '.v%6F; 148!)!I#+!V!VJJ!$@F@@25+@@ $Z0 Z05 5 5|u$$ 2 222 !8#DSZ#P#PQ    :::-0Z::  ;   ("NK##GG "$H$'FHH:=+HH Z03 3 3ZFkGGK\KKKLL L""!%! #   s0.B*B2C*D  E''*FF;;*G%rwUnion[list[ls_schemas.ExampleCreate] | list[ls_schemas.ExampleUpsertWithAttachments] | list[ls_schemas.ExampleUpdate],]include_dataset_id/tuple[Any, bytes, dict[str, io.BufferedReader]]c :g}i}|r>> 3.    !1%)LLL ;* $ $$$&&DDD///s/J>>&K'&K')updatesrr`(Optional[list[ls_schemas.ExampleUpdate]]!ls_schemas.UpsertExamplesResponsec2||||S)zUpdate examples using multipart. .. deprecated:: 0.3.9 Use Client.update_examples instead. Will be removed in 0.4.0. rgr`r)_update_examples_multipart)rrgr`rs reupdate_examples_multipartz Client.update_examples_multipart*..!)E/   ric6|jjpiddstd|g}||d|\}}} |dt |j||i|jd|j id }tj |tt|n3#tt|wxYw|S) a;Update examples using multipart. Args: dataset_id (Union[UUID, str]): The ID of the dataset to update. updates (Optional[List[ExampleUpdate]]): The updates to apply to the examples. Raises: ValueError: If the multipart examples endpoint is not enabled. "dataset_examples_multipart_enabledFzYour LangSmith deployment does not allow using the latest examples endpoints, please upgrade your deployment to the latest version or downgrade your SDK to langsmith<0.3.9.NrJrrrNrrrrr{rr_r/_dataset_examples_pathrkrrrr0rrrr)rrgr`rrrLr{r2s rerez!Client._update_examples_multipartsK  (.B33 0%   &  ?G+/+G+G $)E,H, , ((  ;00&t|Z@@  - &(<    1  H  / 9 9 9 /668899 : : : :L/668899 : : : :}} AC0D)uploadsrrn(Optional[list[ls_schemas.ExampleCreate]]c2||||S)zUpload examples using multipart. .. deprecated:: 0.3.9 Use Client.create_examples instead. Will be removed in 0.4.0. rgrnr)_upload_examples_multipart)rrgrnrs reupload_examples_multipartz Client.upload_examples_multipartrgrirls_schemas.ExampleCreatecvd}|jr$|tt|jz }|jr$|tt|jz }|jr$|tt|jz }|jr|jD]\}}t|tr |d}n|\}}t|tr@ |tj |z }n.#ttf$r|dz }YnwxYw|t|z }|dz }|S)z?Estimate the size of an example in bytes for batching purposes.rZrLri)rrprrrCrrr r r rr_rUrVOSError)rrsizerrr\s re_estimate_example_sizezClient._estimate_example_sizesU > 5 C GN3344 4D ? 6 C GO4455 5D   7 C G$45566 6D   !(!4!:!:! {"example_ids": [... ``` r[rz'max_concurrency must be between 1 and 3rcg|]}d|d S)rrh)r|rs rerZz*Client.create_examples..s #=#=#=HHHH#=#=#=rizCannot specify z when 'examples' is specified.>idsrsplitsrrCsource_run_idsz(Received unsupported keyword arguments: rz3Either dataset_id or dataset_name must be provided.rec^g|]*}t|trtjdi|n|+Srh)r r r{rSr|rs rerZz*Client.create_examples../sM2.As}FCy&#~(  "%#'$-!'!6S%6%6)7   rirc3K|]}dVdSryrhr|rs rer z)Client.create_examples..Ns"-M-Mqd-M-M-M-M-M-MrirCc3K|]}dVdSryrhrs rer z)Client.create_examples..Os".N.Nt.N.N.N.N.N.Nrirc3K|]}dVdSryrhrs rer z)Client.create_examples..Ps",L,LaT,L,L,L,L,L,Lrirc3K|]}dVdSryrhrs rer z)Client.create_examples..Qs")I)I1$)I)I)I)I)I)Irirc3K|]}dVdSryrhrs rer z)Client.create_examples..Rs"4T4TaT4T4T4T4T4T4Trirryrdr)rrr differencerrmrr{rprziprcr{UpsertExamplesResponser!_upload_examples_batches_parallel)rrfrgrrrr kwarg_keyssupported_kwargs unsupportedrnr input_lenarg_name arg_valuer}s rer(zClient.create_examplesspZO((((q((((FGG G  h #=#=f#=#=#=>>JL*LLL      c&kk&<&<=M&N&NN{ P5;M;MPPP  Il IRSS S I** *EEHJ & !GGZZ))F P !NOOOF I'-||~~  #)(S^^y-H-H$;X;;Y;;.7;;;JMJJy))M-M-ME)_upload_single_batch)r|rrrgrArs rerZz.nsH -0 riryrdrr) langsmith.utilsrrr?rrr{r{r)rr}rgrrall_examples_ids total_count latest_as_ofrrrAr2rrAs` `` @rerz(Client._upload_examples_batches_parallelds  ====== & &? C C C )x%G/'22 ) )!==?? '' ]B(G(GHHHx||GQ777 ! W--)l2el6J6J#(L ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ),0( <    sB C  CCc |jjpiddr2|t t j||}|S|D]-}t|dd|_tj d.| ddi|j d d itfd |D }tj||}d |Dt#|ddS)zIUpload a single batch of examples (used by both sequential and parallel).riFrqrNz7Must upgrade your LangSmith version to use attachments.r/examples/bulkrNrcXg|]&}it|ddti'S)T exclude_nonerg) dump_modelrW)r|uploadrgs rerZz/Client._upload_single_batch..sU # (dCCC(#j//rircg|] }|d Srrh)r|rLs rerZz/Client._upload_single_batch..sEEEtT EEErir)rrr{rrrr[r\rrrrr/rrrr0rrp)rrrgrr2r response_datas ` rerzClient._upload_single_batchs] I $ * / / 0%  & 66 :66-I7H O   6=11=)-F&MQ00 M4=M.:LMM  ',  1  H  / 9 9 9$MMOOMEE}EEE]++ riOptional[Mapping[str, Any]]r`Optional[str | list[str]]rrPrNrOc || std|||j}tjdi||||| | | | d} |r|| _|r+t |trtj |n|ntj | _|j j pi ddr3||| g|| jSt!| d d| _t%jd |d d i|jd dit-idt/| Ddt|i}t3j||}tjdi||j|dS)a-Create a dataset example in the LangSmith API. Examples are rows in a dataset, containing the inputs and expected outputs (or other reference information) for a model or chain. Args: inputs (Mapping[str, Any]): The input values for the example. dataset_id (Optional[Union[UUID, str]]): The ID of the dataset to create the example in. dataset_name (Optional[str]): The name of the dataset to create the example in. created_at (Optional[datetime.datetime]): The creation timestamp of the example. outputs (Optional[Mapping[str, Any]]): The output values for the example. metadata (Optional[Mapping[str, Any]]): The metadata for the example. split (Optional[str | List[str]]): The splits for the example, which are divisions of your dataset such as 'train', 'test', or 'validation'. example_id (Optional[Union[UUID, str]]): The ID of the example to create. If not provided, a new example will be created. source_run_id (Optional[Union[UUID, str]]): The ID of the source run associated with this example. use_source_run_io (bool): Whether to use the inputs, outputs, and attachments from the source run. use_source_run_attachments (Optional[List[str]]): Which attachments to use from the source run. If use_source_run_io is True, all attachments will be used regardless of this param. attachments (Optional[Attachments]): The attachments for the example. Returns: Example: The created example. Nz/Must provide either inputs or use_source_run_iore)rrrCr`rPrNrOrriF)rgrn)rrz6Must upgrade your LangSmith version to use attachmentsrr}rNrci|] \}}||| Sryrhr5s rerz)Client.create_example..sVVVDAq 1a rirgrrrh)rrmrr{rSr.r rWr[r\rrrr{rr read_examplerrrrr/rrrrrr0rr~rr)rrrgrfr.rrCr`rrPrNrOrrLr2rs rer3zClient.create_examples=l >"3>NOO O  ** *EEHJ'    "$!.%6.H*       )(DO jS&A&A QTYz " " "zz  I $ * / / 0%     + +zD6 + R R R$$$88 8t]++7#'  VWWW00M4=M.:LMM VVJt,<,<,B,B,D,DVVV$c*oo 1  H  / 9 9 9]]__F%.7799 ri)rcn|ddt|dd|r|ndi}|}t |d|j}t jd id |D||j | d S) aRead an example from the LangSmith API. Args: example_id (Union[UUID, str]): The ID of the example to read. as_of (Optional[datetime.datetime]): The dataset version tag OR timestamp to retrieve the example as of. Response examples will only be those that were present at the time of the tagged (or timestamped) version. Returns: Example: The example. r- /examples/rrNrattachment_urlsrc&i|]\}}|dk ||Srrhr5s rerz'Client.read_example..:s)JJJ118I3I3Iq!3I3I3Irirrrrh) r/rarrrrkr{r~rrr)rrrr2rrs rerzClient.read_examples,,  =*l;; = =e=***-  --//E %6    !  JJ JJJ #n3355      ri)rrrCrinclude_attachments'Optional[Union[datetime.datetime, str]]rinline_s3_urlsrc +nKi| ||t|tjr|n||||t|dnd| d} | t | | d<||| d<n$|!||j}|| d<n | rgd| d<t|d | D]v\}}t|d |j }tj did | D||j|dV| |dz|krdSwdS)a Retrieve the example rows of the specified dataset. Args: dataset_id (Optional[Union[UUID, str]]): The ID of the dataset to filter by. dataset_name (Optional[str]): The name of the dataset to filter by. example_ids (Optional[Sequence[Union[UUID, str]]): The IDs of the examples to filter by. as_of (Optional[Union[datetime.datetime, str]]): The dataset version tag OR timestamp to retrieve the examples as of. Response examples will only be those that were present at the time of the tagged (or timestamped) version. splits (Optional[Sequence[str]]): A list of dataset splits, which are divisions of your dataset such as 'train', 'test', or 'validation'. Returns examples only from the specified splits. inline_s3_urls (bool, default=True): Whether to inline S3 URLs. offset (int, default=0): The offset to start from. Defaults to 0. limit (Optional[int]): The maximum number of examples to return. metadata (Optional[dict]): A dictionary of metadata to filter by. filter (Optional[str]): A structured filter string to apply to the examples. include_attachments (bool, default=False): Whether to include the attachments in the response. **kwargs (Any): Additional keyword arguments are ignored. Yields: The examples. Examples: List all examples for a dataset: ```python from langsmith import Client client = Client() # By Dataset ID examples = client.list_examples( dataset_id="c9ace0d8-a82c-4b6c-13d2-83401d68e9ab" ) # By Dataset Name examples = client.list_examples(dataset_name="My Test Dataset") ``` List examples by id ```python example_ids = [ "734fc6a0-c187-4266-9721-90b7a025751a", "d6b4c1b9-6160-4d63-9b61-b034c585074f", "4d31df4e-f9c3-4a6e-8b6c-65701c2fed13", ] examples = client.list_examples(example_ids=example_ids) ``` List examples by metadata ```python examples = client.list_examples( dataset_name=dataset_name, metadata={"foo": "bar"} ) ``` List examples by structured filter ```python examples = client.list_examples( dataset_name=dataset_name, filter='and(not(has(metadata, \'{"foo": "bar"}\')), exists(metadata, "tenant_id"))', ) ``` Nrj)rrrrrrrrCrRre)rrrCrr}rrrc&i|]\}}|dk ||Srrhr5s rerz(Client.list_examples..s)NNNDAqq.sFFF1 q! ri)rgr`rrrrNrci|] \}}||| Sryrhr5s rerz)Client.update_example..s+ $1 }q,}}rirgrrh)rrr{rr r{rTrrrgrer/rarrrrWrr0r) rrrrrCr`rgrWr example_dictrr2s reupdate_examplezClient.update_examplesH " -I,2774e !V#9#   *  FF 2 2 4 4FFF    **:66AJ I $ * / / 0%   #//)G90 00AXj,??AAM4=M.:LMM (27(;(;(A(A(C(C %c*oo    1H  / 9 9 9==?? "ri)rfrgr`r str | NoneID_TYPE | None3Optional[Sequence[ls_schemas.ExampleUpdate | dict]]c j|r"|r tdt|d|s|std|r|rtd|r||j}|r;d|D}|s+|djr|dj}ntdn|d d }|std |sd |vr!||dj}n`tt|d d krtd|d dstd|d d}|j j pid}|s:|dp|d tdt|} | D]@\} } | 9t| | kr&td| dt| d| dAdt||ddt| D|ddt| D|ddt| D|dd t| D|d d!t| D|dd"t| D|dd#t| DD}d } |j j pidd$rM|tt j|||%} d&| d'dd(i| S|d)d*i|jd+d,it)d-|D.} t+j| | S)/a Update multiple examples. Examples are expected to all be part of the same dataset. Args: dataset_name (str | None): The name of the dataset to update. Should specify exactly one of 'dataset_name' or 'dataset_id'. dataset_id (UUID | str | None): The ID of the dataset to update. Should specify exactly one of 'dataset_name' or 'dataset_id'. updates (Sequence[ExampleUpdate | dict] | None): The example updates. Overwrites any specified fields and does not update any unspecified fields. dangerously_allow_filesystem (bool): Whether to allow using filesystem paths as attachments. **kwargs (Any): Legacy keyword args. Should not be specified if 'updates' is specified. - example_ids (Sequence[UUID | str]): The IDs of the examples to update. - inputs (Sequence[dict | None] | None): The input values for the examples. - outputs (Sequence[dict | None] | None): The output values for the examples. - metadata (Sequence[dict | None] | None): The metadata for the examples. - splits (Sequence[str | list[str] | None] | None): The splits for the examples, which are divisions of your dataset such as 'train', 'test', or 'validation'. - attachments_operations (Sequence[AttachmentsOperations | None] | None): The operations to perform on the attachments. - dataset_ids (Sequence[UUID | str] | None): The IDs of the datasets to move the examples to. Returns: The LangSmith JSON response. Includes 'message', 'count', and 'example_ids'. !!! warning "Behavior changed in `langsmith` 0.3.9" Updated to ... Example: ```python from langsmith import Client client = Client() dataset = client.create_dataset("agent-qa") examples = [ { "inputs": {"question": "what's an agent"}, "outputs": {"answer": "an agent is..."}, "metadata": {"difficulty": "easy"}, }, { "inputs": { "question": "can you explain the agent architecture in this diagram?" }, "outputs": {"answer": "this diagram shows..."}, "attachments": {"diagram": {"mime_type": "image/png", "data": b"..."}}, "metadata": {"difficulty": "medium"}, }, # more examples... ] response = client.create_examples(dataset_name="agent-qa", examples=examples) example_ids = response["example_ids"] updates = [ { "id": example_ids[0], "inputs": {"question": "what isn't an agent"}, "outputs": {"answer": "an agent is not..."}, }, { "id": example_ids[1], "attachments_operations": [ {"rename": {"diagram": "agent_diagram"}, "retain": []} ], }, ] response = client.update_examples(dataset_name="agent-qa", updates=updates) # -> {"example_ids": [... ``` z&Must pass in either 'updates' or args z , not both.z6Please pass in a non-empty sequence for arg 'updates'.z;Must pass in exactly one of 'dataset_name' or 'dataset_id'.rec^g|]*}t|trtjdi|n|+Sr)r r r{rTrs rerZz*Client.update_examples..{sM2.sujCdIv{LRi(  !"%#'&1$-!''32I    rirc3K|]}dVdSryrhrs rer z)Client.update_examples..")L)L1$)L)L)L)L)L)Lrirc3K|]}dVdSryrhrs rer z)Client.update_examples..s"*M*MA4*M*M*M*M*M*MrirCc3K|]}dVdSryrhrs rer z)Client.update_examples..s"+N+NQD+N+N+N+N+N+Nrirc3K|]}dVdSryrhrs rer z)Client.update_examples..rric3K|]}dVdSryrhrs rer z)Client.update_examples..".Q.Qt.Q.Q.Q.Q.Q.Qric3K|]}dVdSryrhrs rer z)Client.update_examples..rric3K|]}dVdSryrhrs rer z)Client.update_examples..s"2U2UA42U2U2U2U2U2UriFrdr>rdz examples updatedrrrNrcdg|]-}dt|D.S)ci|] \}}||| Sryrhr5s rerz5Client.update_examples...s+$(Aq#$=!"1#0==ri)rrrs rerZz*Client.update_examples..sU !( ,6w,?,?,E,E,G,Grir)rrrmrrgr{rrprrrrrrcrerr[r\r/rrrr0r) rrfrgr`rr updates_objrymultipart_enabled examples_lenrrr2s reupdate_exampleszClient.update_exampless1p  g SvSSS  W' WUVV V  IJ IM  I** *EEHJ M  K  q>,!,Q!:JJ$W  !**]D99K  O : ..!%!2!2;q>!B!B!MJJVM23344q88$%TUUU .q1:$%RSSS!' !6q!9J!%!9!?R D D4!! & ZZ 899VVZZ =V=V!X {++L'-||~~  #)(S^^|-K-K$@X@@Y@@0<@@@nqJJx)L)Ll8K8K)L)L)LMMJJy*M*M|9L9L*M*M*MNNJJz+N+N% :M:M+N+N+NOOJJx)L)Ll8K8K)L)L)LMMJJ}.Q.QU<=P=P.Q.Q.QRRJJ}.Q.QU<=P=P.Q.Q.QRRJJ02U2U|ATAT2U2U2U n nK6 I $ * / / 0%  ! #66 :66#-I7Hhll7A66III  00 M4=M.:LMM ,7    1H"  / 9 9 9==?? "ric|ddt|d|j}tj|dS)zDelete an example by ID. Args: example_id (Union[UUID, str]): The ID of the example to delete. Returns: None rVrrrWNrX)rrr2s redelete_examplezClient.delete_examplesT,,  =*l;; = =M-   +H55555ri) hard_deleteSequence[ID_TYPE]rc|rdt|Ddd}t|jd}t|j|}||jd|i|jddit||j }n?d d t|Di}| d d i|jddi| }tj |dS)a"Delete multiple examples by ID. Parameters ---------- example_ids : Sequence[ID_TYPE] The IDs of the examples to delete. hard_delete : bool, default=False If True, permanently delete the examples. If False, soft delete them. c Vg|]&\}}tt|d|d'Sz example_ids[]rWrar|r rs rerZz*Client.delete_examples..J   3&9Q&9&9&9::;;   riT)ryrzdatasets/examples/deleterrNr)rrLrryc Vg|]&\}}tt|d|d'Srrrs rerZz*Client.delete_examples..#rrirVr}rN) r_platform_pathrkrgr"rsrfrrr r/rr0)rryrrr_full_urlr2rs redelete_exampleszClient.delete_examplessD #    "+K"8"8    $ D"$,0JKKD%dlD99H  % % ' ' '|++M4=M.:LMM && ,HH  "+K"8"8   &F 00M4=M.:LMM 1H +H55555ri)rgrfr'Optional[Union[str, datetime.datetime]] list[str]cp|,|td||j}i}|3t|tjr|n||d<|ddt|dd| }tj || S) aGet the splits for a dataset. Args: dataset_id (Optional[Union[UUID, str]]): The ID of the dataset. dataset_name (Optional[str]): The name of the dataset. as_of (Optional[Union[str, datetime.datetime]]): The version of the dataset to retrieve splits for. Can be a timestamp or a string tag. Defaults to "latest". Returns: List[str]: The names of this dataset's splits. NMust provide dataset name or IDrerr-rkrg/splitsr) rrmrr rrr/rarr0r)rrgrfrrr2s relist_dataset_splitszClient.list_dataset_splits0s&  # !BCCC** *EEHJ  %/x7H%I%IT!!!u 7O,,  D*l;; D D D-   +H555}}ri)rgrfr split_name list[ID_TYPE]rc|,|td||j}|dt|D|d}|ddt |dd | }t j|dS) aFUpdate the splits for a dataset. Args: dataset_id (Optional[Union[UUID, str]]): The ID of the dataset to update. dataset_name (Optional[str]): The name of the dataset to update. split_name (str): The name of the split to update. example_ids (List[Union[UUID, str]]): The IDs of the examples to add to or remove from the split. remove (Optional[bool]): If True, remove the examples from the split. If False, add the examples to the split. Returns: None Nrrec Vg|]&\}}tt|d|d'Srrrs rerZz0Client.update_dataset_splits..rsJAsHS"5"5"5"56677ri)r rrrOrkrgrr)rrmrrr/rarr0)rrgrfr ryrrLr2s reupdate_dataset_splitszClient.update_dataset_splitsUs.  # !BCCC** *EEHJ$' 44   ,, K\ B BKKKRV-   +H55555ri@Union[V2Run, ls_schemas.Run, ls_schemas.RunBase, str, uuid.UUID]0Union[V2Run, ls_schemas.Run, ls_schemas.RunBase]ct|ttjfrt j|jj}|tjj kr-|tj d|tj dn|tj dtdt5|||||}dddn #1swxYwYn|}|S)aResolve the run ID. Args: run (Union[V2Run, Run, RunBase, str, UUID]): The run to resolve. load_child_runs (bool): Whether to load child runs. project_id (Optional[Union[UUID, str]]): The ID of the project (session) that owns the run. Required to look the run up on SmithDB-only backends. start_time (Optional[datetime]): The run's start time. Required to look the run up on SmithDB-only backends. Returns: Run: The resolved run. Raises: TypeError: If the run type is invalid. Nztproject_id is required to resolve a run from an ID when ClickHouse query support is disabled (SmithDB-only backend).ztstart_time is required to resolve a run from an ID when ClickHouse query support is disabled (SmithDB-only backend).zResolving a run from an ID without passing project_id is deprecated and will raise an error in a future release. Pass project_id explicitly.rrrrr)r rWr[r\r(rGrrrFrrrBrrrrr)rrrrrrrun_s re_resolve_run_idzClient._resolve_run_id~sQ8 cC+ , ," );DI._cast_results-.. F0A=0A -//I]4R4R/<[/IM),!!+t444&=+<+._is_eval_resultssgt,,Eg1E Eric&g|] }|Srhrh)r|rr(s rerZz/Client._select_eval_results..s!DDDA QDDDriz!Invalid evaluation results type: z.. Must be EvaluationResult, EvaluationResults.)rr rXr!)rrrXr))langsmith.evaluationrr r%r rrr)rrrr+results_r(r's ` @@re_select_eval_resultszClient._select_eval_resultss CBBBBB ! ! ! ! ! ! ! F F F F g|< = = yHH  g & & DDDD1CDDDHH  & & $ T$%8%899:HH@DMM@@@ rizevaluate_run() is deprecated and will be removed after Jan 31, 2027. There is no replacement: run the evaluator yourself and log the result with client.create_feedback().)rr source_inforrrls_evaluator.RunEvaluatorr0r9Optional[Union[ls_schemas.Example, str, dict, uuid.UUID]]r!c|||||}|||} |ttj|| } || |||} | dS)aEvaluate a run. .. admonition:: Deprecated There is no replacement. Run the evaluator yourself and log the result with :meth:`langsmith.Client.create_feedback`. Will be removed after Jan 31, 2027. Args: run (Union[V2Run, Run, RunBase, str, UUID]): The run to evaluate. Passing `schemas.Run` (the legacy read-side datamodel) is deprecated; pass the v2 `Run` datamodel (from `client.runs.retrieve`) instead. evaluator (RunEvaluator): The evaluator to use. project_id (Optional[Union[UUID, str]]): The ID of the project (session) that owns the run. Omitting it is deprecated and will raise in a future release. start_time (Optional[datetime]): The run's start time, passed to `runs.retrieve` when resolving `run` from an ID. Required when resolving a run from an ID on SmithDB-only backends (no ClickHouse query support). source_info (Optional[Dict[str, Any]]): Additional information about the source of the evaluation to log as feedback metadata. reference_example (Optional[Union[Example, str, dict, UUID]]): The example to use as a reference for the evaluation. If not provided, the run's reference example will be used. load_child_runs (bool, default=False): Whether to load child runs when resolving the run ID. **Deprecated**: see `read_run`. Returns: Feedback: The feedback object created by the evaluation. rrr0rr)rr evaluate_runrr{r_log_evaluation_feedback rrrrrr0rrrrevaluator_responsers rer6zClient.evaluate_runsj## +!! $   "556GNN&33  & &&4  //  #! 0  qzri) _executorr9:Optional[Union[V2Run, ls_schemas.Run, ls_schemas.RunBase]]r:Optional[cf.ThreadPoolExecutor]c |}fd}|"t|ddpt|ddnd}|D] } |pi} | jr i| j| } d} | jr| j} n ||j} | j| jddnd} |did| d| jd| jd| j d | j d | j d | d | j d tttj| jdtjjd||ndd| jd|rt|ddndd|p|d|r|jndd|  |S)NcPrjjfi|dSjdi|dS)Nrh)r>create_feedback)rr:rs re_submit_feedbackz9Client._log_evaluation_feedback.._submit_feedback\sN /  !5@@@@@@@$$..v.....rirrrrr}rr~r$ correctionr0rPfeedback_configfeedback_source_typerrrrh)r/revaluator_info target_run_idrrr{r}rr~r$rArPrrr{FeedbackConfigrBFeedbackSourceTypeMODELr) rr9rr0rr:rr@run_session_idres source_info_rrs ` ` rer7zClient._log_evaluation_feedbackOs*++,>?? / / / / / /S, - - QlD1Q1Q    C&,"L! FE#"4E E G  !+&47I4ICIMM'4000tE     w GG ii ii      >>  )L "// !%Z679L!!! &0%B%H%H *-::$ ii >d# $e%   (rizaevaluate_run() is deprecated and will be removed after Jan 31, 2027. There is no replacement: run the evaluator yourself and log the result with client.create_feedback().cK|||||}|||} |ttj|| d{V} || |||} | dS)aEvaluate a run asynchronously. .. admonition:: Deprecated There is no replacement. Run the evaluator yourself and log the result with :meth:`langsmith.Client.create_feedback`. Will be removed after Jan 31, 2027. Args: run (Union[V2Run, Run, RunBase, str, UUID]): The run to evaluate. Passing `schemas.Run` (the legacy read-side datamodel) is deprecated; pass the v2 `Run` datamodel (from `client.runs.retrieve`) instead. evaluator (RunEvaluator): The evaluator to use. project_id (Optional[Union[UUID, str]]): The ID of the project (session) that owns the run. Omitting it is deprecated and will raise in a future release. start_time (Optional[datetime]): The run's start time, passed to `runs.retrieve` when resolving `run` from an ID. Required when resolving a run from an ID on SmithDB-only backends (no ClickHouse query support). source_info (Optional[Dict[str, Any]]): Additional information about the source of the evaluation to log as feedback metadata. reference_example (Optional[Union[Example, str, dict, UUID]]): The example to use as a reference for the evaluation. If not provided, the run's reference example will be used. load_child_runs (bool, default=False): Whether to load child runs when resolving the run ID. **Deprecated**: see `read_run`. Returns: EvaluationResult: The evaluation result object created by the evaluation. rr4Nr5r)rr aevaluate_runrr{rr7r8s rerMzClient.aevaluate_runsj## +!! $   "556GNN#,#:#:  & &&$;$ $        //  #! 0   qzriunnamedr)rr~rrAr$r0rCrP feedback_idrBr?rcomparative_experiment_idfeedback_group_idrrrrextend_trace_retentionr}rr(Union[float, int, bool, str, dict, None]rAUnion[dict, None]r$Union[str, None]rC)Union[ls_schemas.FeedbackSourceType, str]rOrB#Optional[ls_schemas.FeedbackConfig]rPrQrrRls_schemas.Feedbackc  |p|}||td||td||t|j|r*tjdt |t  t| tj stj | } | tj j krtj |}n=| tj j krtj |}ntd| |j|jni|_| "d|jvrdt| i|jd<|jrd|jvr|jd}t!|d r1t#t%|d r|}d|vr1tt)|jddd |d<||jd<t+||n|d }tjd)id t+| dt+|d dt+|d d|dt/|d|d|d|d|dt0jt0jjdt0jt0jjd| d|d|dt+|d dt+|d d|d|d|}|j o!|jjpidd }|r$|r!|jjrtAj!|jjd r|j"|j#|j$|j%tM|}|j#tO|}|j#j(5tS||j#tT}|r5|j#xj+d!z c_+|j,r|j,-dddn #1swxYwYn|j"5|.t_t|j0|nJtc|d "}|2d#d$d%|i| t@j3f&tj4d)i|S#tj$r#} tl7d'd (| d} ~ wwxYw)*aCreate feedback for a run. !!! note To enable feedback to be batch uploaded in the background you must specify `trace_id`. *We highly encourage this for latency-sensitive environments.* Args: key (str): The name of the feedback metric. score (Optional[Union[float, int, bool]]): The score to rate this run on the metric or aspect. value (Optional[Union[float, int, bool, str, dict]]): The display value or non-numeric value for this feedback. run_id (Optional[Union[UUID, str]]): The ID of the run to provide feedback for. At least one of run_id, trace_id, or project_id must be specified. trace_id (Optional[Union[UUID, str]]): The ID of the trace (i.e. root parent run) of the run to provide feedback for (specified by run_id). If run_id and trace_id are the same, only trace_id needs to be specified. **NOTE**: trace_id is required feedback ingestion to be batched and backgrounded. correction (Optional[dict]): The proper ground truth for this run. comment (Optional[str]): A comment about this feedback, such as a justification for the score or chain-of-thought trajectory for an LLM judge. source_info (Optional[Dict[str, Any]]): Information about the source of this feedback. feedback_source_type (Union[FeedbackSourceType, str]): The type of feedback source, such as model (for model-generated feedback) or API. source_run_id (Optional[Union[UUID, str]]): The ID of the run that generated this feedback, if a "model" type. feedback_id (Optional[Union[UUID, str]]): The ID of the feedback to create. If not provided, a random UUID will be generated. feedback_config (Optional[FeedbackConfig]): The configuration specifying how to interpret feedback with this key. Examples include continuous (with min/max bounds), categorical, or freeform. stop_after_attempt (int, default=10): The number of times to retry the request before giving up. project_id (Optional[Union[UUID, str]]): The ID of the project (or experiment) to provide feedback on. This is used for creating summary metrics for experiments. comparative_experiment_id (Optional[Union[UUID, str]]): If this feedback was logged as a part of a comparative experiment, this associates the feedback with that experiment. feedback_group_id (Optional[Union[UUID, str]]): When logging preferences, ranking runs, or other comparative feedback, this is used to group feedback together. extra (Optional[Dict]): Metadata for the feedback. session_id (Optional[Union[UUID, str]]): The session (project) ID of the run. Required for run-level feedback; omitting it is deprecated. See https://docs.langchain.com/langsmith/smithdb-sdk-migration#feedback-create start_time (Optional[datetime]): The start time of the run. Better performance if provided. extend_trace_retention (bool, default=True): If false, create the feedback without extending the trace's retention tier. **kwargs (Any): Additional keyword arguments. Returns: Feedback: The created feedback object. Example: ```python from langsmith import trace, traceable, Client @traceable def foo(x): return {"y": x * 2} @traceable def bar(y): return {"z": y - 1} client = Client() inputs = {"x": 1} with trace(name="foobar", inputs=inputs) as root_run: result = foo(**inputs) result = bar(**result) root_run.outputs = result trace_id = root_run.id child_runs = root_run.child_runs # Provide feedback for a trace (a.k.a. a root run) client.create_feedback( key="user_feedback", score=1, trace_id=trace_id, ) # Provide feedback for a child run foo_run_id = [run for run in child_runs if run.name == "foo"][0].id client.create_feedback( key="correctness", score=0, run_id=foo_run_id, # trace_id= is optional but recommended to enable batched and backgrounded # feedback ingestion. trace_id=trace_id, ) ``` Nz8One of run_id, trace_id, or project_id must be providedz?project_id cannot be provided if run_id or trace_id is providedzLThe following arguments are no longer used in the create_feedback endpoint: )rCzUnknown feedback source type __runrrz+feedback_source.metadata['__run']['run_id']Trrrr}rr~rAr$feedback_sourcer. modified_atrBrrrPrQrrrRuse_multipart_endpointz0.8.10r[rr /feedbackrL)r>r?r@zError creating feedbackr rh)8rrrrrrrr r{rGAPIAPIFeedbackSourcerHModelFeedbackSourcerCrWrrrrrarFeedbackCreaterrrcrrr,r:r{rris_version_greater_or_equalrr!rr&r@rBrr?r1rr"rrr*rrr/rpFeedbackr@rar)!rrr}rr~rrAr$r0rCrPrOrBr?rrPrQrrrrrRrr[ _run_meta _session_idfeedbackrrrrfeedback_blockrMs! rer?zClient.create_feedbacksg\#8 >j0WXX X  *"8Q   *"4 &ty 1 1 1   M/$Vnn//"    r 2J4QRR '1'D((($$z'D'HHH0+FFF &)F)LLL","@+"V"V"V !WAU!W!WXXX,;,D,P((VX  $(WOG7O,O,O/>.Fw.O 9l337I|44997!* 4 4 6 6Iy((*- +4W=hGI++Ih' 5>(1&(4 *RVK"0 ,,,$F====  &hDAAAA  C -U333e&: !0$,001B1FGGG%-11(2C2GHHH!0'; &:!"+7-4++++#(#//@d"S"S"S"S)*e+,e-.(>'=/H4!% 882 -3c*D11  * ** I%* 89JHUU *  &2d6L6X%1&. 7 A A )5T)# /4 A A#G* 2%$$ $A 2>>!C>>#9A $ : > > @ @ @ A A A A A A A A A A A A A A A'3++(X[)9)9=II"-X-@-@d-@-S-S!T!T))$(:&=?*&??)<)<)>)>?? ?    LL2TL B B BG s?0L R%AO/# R%/O33R%6O37B-R%% S/S  S)rr~rAr$ci}|t||d<|||d<|||d<|||d<|ddt|di|jd d it | }t j|dS) ajUpdate a feedback in the LangSmith API. Args: feedback_id (Union[UUID, str]): The ID of the feedback to update. score (Optional[Union[float, int, bool]]): The score to update the feedback with. value (Optional[Union[float, int, bool, str, dict]]): The value to update the feedback with. correction (Optional[dict]): The correction to update the feedback with. comment (Optional[str]): The comment to update the feedback with. Returns: None Nrr~rAr$r /feedback/rOrNrr)rr/rarrrr0)rrOrr~rAr$feedback_updater2s reupdate_feedbackzClient.update_feedback s4+-  '=e'D'DOG $  ',OG $  !,6OL )  )0OI &,,  ?+}== ? ?It}In6HII_-- -   +H55555ric|ddt|d}tjdi|S)zRead a feedback from the LangSmith API. Args: feedback_id (Union[UUID, str]): The ID of the feedback to read. Returns: Feedback: The feedback. r-rjrOrh)r/rar{rdrrrOr2s re read_feedbackzClient.read_feedback!sR,,  ?+}== ? ?  "55X]]__555ri)r feedback_keyrCrrp1Optional[Sequence[ls_schemas.FeedbackSourceType]]Iterator[ls_schemas.Feedback]c+K||t|dndd|}|||d<|||d<t|d|D]&\}}tjd i|V| |dz|krdS'dS) aList the feedback objects on the LangSmith API. Args: run_ids (Optional[Sequence[Union[UUID, str]]]): The IDs of the runs to filter by. feedback_key (Optional[Sequence[str]]): The feedback key(s) to filter by. Examples: 'correctness' The query performs a union of all feedback keys. feedback_source_type (Optional[Sequence[FeedbackSourceType]]): The type of feedback source, such as model or API. limit (Optional[int]): The maximum number of feedback to return. **kwargs (Any): Additional keyword arguments. Yields: The feedback objects. Nrj)rrr}rr^rr[rh)r0rrr{rd) rrrprCrrrr rgs re list_feedbackzClient.list_feedback!s8(-(9S___s     #(F5M  +3F8 $  $ $[ $ @ @    KAx%1111 1 1 1 QUe^^   ric|ddt|d|j}tj|dS)zDelete a feedback by ID. Args: feedback_id (Union[UUID, str]): The ID of the feedback to delete. Returns: None rVrjrOrWNrXrns redelete_feedbackzClient.delete_feedbackJ!sT,,  ?+}== ? ?M-   +H55555ri)r~rAr$rCrTc t||jd\}}||jkrtd||ddt |t |||||d|j} tj| dS) aCreate feedback from a presigned token or URL. Args: token_or_url (Union[str, uuid.UUID]): The token or URL from which to create feedback. score (Optional[Union[float, int, bool]]): The score of the feedback. value (Optional[Union[float, int, bool, str, dict]]): The value of the feedback. correction (Optional[dict]): The correction of the feedback. comment (Optional[str]): The comment of the feedback. metadata (Optional[dict]): Additional metadata for the feedback. Raises: ValueError: If the source API URL is invalid. Returns: None r[)rUzInvalid source API URL. rz/feedback/tokens/)rr~rAr$rCrN) rfrkrr/rarrrr0) rr rr~rAr$rCrrer2s recreate_feedback_from_tokenz!Client.create_feedback_from_token[!s8&9 $,!& & & "  T\ ) )HHHII I,,  6 4 4 6 6""",& (   M-   +H55555ri) expirationrBrOry0Optional[datetime.datetime | datetime.timedelta]ls_schemas.FeedbackIngestTokenc~||||pttjd}|tjddd|d<nt |t jr||d<nlt |t jr3tj|j |j dz|j d zd z|d<ntd t|| d d t| }tj|tjdi|S)aCreate a pre-signed URL to send feedback data to. This is useful for giving browser-based clients a way to upload feedback data directly to LangSmith without accessing the API key. Args: run_id (Union[UUID, str]): The ID of the run. feedback_key (str): The key of the feedback to create. expiration (Optional[datetime.datetime | datetime.timedelta]): The expiration time of the pre-signed URL. Either a datetime or a timedelta offset from now. Default to 3 hours. feedback_config (Optional[FeedbackConfig]): If creating a feedback_key for the first time, this defines how the metric should be interpreted, such as a continuous score (w/ optional bounds), or distribution over categorical values. feedback_id (Optional[Union[UUID, str]): The ID of the feedback to create. If not provided, a new feedback will be created. Returns: FeedbackIngestToken: The pre-signed URL for uploading feedback data. )rrprBrNrrr#hoursminutes expires_in expires_atrPUnknown expiration type: r/feedback/tokens)rLrh)rWr[rr{TimeDeltaInputr rrr.r#secondsrrr/rrr0FeedbackIngestTokenr)rrrpryrBrOrr2s recreate_presigned_feedback_tokenz&Client.create_presigned_feedback_token!sdF(.2TZ\\!2!2     !+!:"""D    H$5 6 6 M!+!5!5!7!7D    H$6 7 7 M!+!:_ (D0#+r1R7"""D   Kj9I9IKKLL L,,  T""-   +H555-@@ @@@ri)ryfeedback_configs feedback_keysr7Optional[Sequence[Optional[ls_schemas.FeedbackConfig]]](Sequence[ls_schemas.FeedbackIngestToken]c |/t|t|krtd|sdgt|z}d\ |tjddd nt |t jr| nit |t jr0tj|j|j dz|j dzdz ntd t|t fd t||D d fd g}tjtj5 fdjD}tj|D]5}|}|d|D6 dddn #1swxYwY|S)aCreate a pre-signed URL to send feedback data to. This is useful for giving browser-based clients a way to upload feedback data directly to LangSmith without accessing the API key. Args: run_id (Union[UUID, str]): The ID of the run. feedback_keys (Sequence[str]): The key of the feedback to create. expiration (Optional[datetime.datetime | datetime.timedelta]): The expiration time of the pre-signed URL. Either a datetime or a timedelta offset from now. Default to 3 hours. feedback_configs (Optional[Sequence[Optional[FeedbackConfig]]]): If creating a feedback_key for the first time, this defines how the metric should be interpreted, such as a continuous score (w/ optional bounds), or distribution over categorical values. Returns: Sequence[FeedbackIngestToken]: The pre-signed URL for uploading feedback data. NzBThe length of feedback_keys and feedback_configs must be the same.NNrrr}rrPrc&g|] \}}||dS))rrprBrrrh)r|rprBrrrs rerZz;Client.create_presigned_feedback_tokens.. "sE   2L/ %$0'6",",    rirkrWrrrXrcd|dijt|pjid}t j||S)Nrrrr)r/rrvrrr0r)rkrr2rrs rereqz4Client.create_presigned_feedback_tokens..req"s00,,,  - !7#:dl    1  H  / 9 9 9==?? "rircDg|]\}}||Srhr>)r|rkrrArs rerZz;Client.create_presigned_feedback_tokens..)"s=$GWWg66ric0g|]}tjdi|Sr)r{r)r|r|s rerZz;Client.create_presigned_feedback_tokens..0"s)QQQZ3;;d;;QQQri)rkrWrrrXr)rprr{rr rrr.r#rrrrrr>rrr?rr)rrrryrtokensfutsfutr2rrArrrs`` @@@@@re create_presigned_feedback_tokensz'Client.create_presigned_feedback_tokens!sD  'C ,>,>#FVBWBW,W,WT   ; $vM(:(:: !+ J  #2JJ  H$5 6 6 M#--//JJ  H$6 7 7 M#2_ (D0#+r1R7JJ Kj9I9IKKLL L      69!#366       # # # # # # #  "s43G/H/H I I I X(,(<(B(B(D(DDt,,  ::<< QQQQQ                 s A1GG"%G")r(Iterator[ls_schemas.FeedbackIngestToken]c#Kt|d|t|dndd}t|d|D]&\}}t jdi|V| |dz|krdS'dS) a List the feedback ingest tokens for a run. Args: run_id (Union[UUID, str]): The ID of the run to filter by. limit (Optional[int]): The maximum number of tokens to return. Yields: The feedback ingest tokens. rNrj)rrrrr[rh)rar0rrr{r)rrrrr tokens relist_presigned_feedback_tokensz%Client.list_presigned_feedback_tokens4"s vx00(-(9S___s  "  $ $%7 $ G G    HAu099599 9 9 9 QUe^^   riz{Composite feedback formulas are no longer supported in the SDK. Add composite feedback scores via the LangSmith UI instead.)rgrrr$Iterator[ls_schemas.FeedbackFormula]c*t|j)aList feedback formulas. .. admonition:: Deprecated Composite feedback formulas are no longer supported in the SDK. Add composite feedback scores via the LangSmith UI instead. This method now raises ``NotImplementedError``. NotImplementedError!_FEEDBACK_FORMULA_DEPRECATION_MSG)rrgrrrs relist_feedback_formulaszClient.list_feedback_formulasY"s("$"HIIIrifeedback_formula_idls_schemas.FeedbackFormulac*t|j)aGet a feedback formula by ID. .. admonition:: Deprecated Composite feedback formulas are no longer supported in the SDK. Add composite feedback scores via the LangSmith UI instead. This method now raises ``NotImplementedError``. rrrs reget_feedback_formula_by_idz!Client.get_feedback_formula_by_ido"s"$"HIIIri)rgraggregation_typeLiteral['sum', 'avg'] formula_partsASequence[Union[ls_schemas.FeedbackFormulaWeightedVariable, dict]]c*t|j)aCreate a feedback formula. .. admonition:: Deprecated Composite feedback formulas are no longer supported in the SDK. Add composite feedback scores via the LangSmith UI instead. This method now raises ``NotImplementedError``. r)rrprrrgrs recreate_feedback_formulazClient.create_feedback_formula"s."$"HIIIric*t|j)aUpdate a feedback formula. .. admonition:: Deprecated Composite feedback formulas are no longer supported in the SDK. Add composite feedback scores via the LangSmith UI instead. This method now raises ``NotImplementedError``. r)rrrprrs reupdate_feedback_formulazClient.update_feedback_formula"s,"$"HIIIric*t|j)aDelete a feedback formula by ID. .. admonition:: Deprecated Composite feedback formulas are no longer supported in the SDK. Add composite feedback scores via the LangSmith UI instead. This method now raises ``NotImplementedError``. rrs redelete_feedback_formulazClient.delete_feedback_formula"s"$"HIIIri)is_lower_score_betterls_schemas.FeedbackConfigrls_schemas.FeedbackConfigSchemac|||d}|dd|}tj|tjdi|S)aDCreate a feedback configuration. Defines how feedback with a given key should be interpreted. If an identical configuration already exists for the key, it is returned unchanged. If a different configuration already exists for the key, an error is raised. Args: feedback_key (str): The feedback key to configure. feedback_config (FeedbackConfig): The configuration defining type, bounds, and categories. is_lower_score_better (Optional[bool]): Whether a lower score is considered better. Defaults to False. Returns: FeedbackConfigSchema: The created or existing feedback configuration. Raises: requests.HTTPError: If a conflicting configuration already exists for the given key (HTTP 400). Example: .. code-block:: python from langsmith import Client client = Client() config = client.create_feedback_config( feedback_key="user-rating", feedback_config={ "type": "continuous", "min": 0.0, "max": 5.0, }, ) )rprBrr/feedback-configsrrhr/rr0r{FeedbackConfigSchemarrrprBrrr2s recreate_feedback_configzClient.create_feedback_config"sq^).%:   ,,  -   +H555.AAAAAri)rprrr)Iterator[ls_schemas.FeedbackConfigSchema]c#K|t|dnd|d}|||d<|||d<t|d|D]&\}}tjd i|V| |dz|krdS'dS) a)List feedback configurations. Args: feedback_key (Optional[Sequence[str]]): Filter by specific feedback keys. name_contains (Optional[str]): Filter by substring match on the feedback key. limit (Optional[int]): The maximum number of configurations to return. offset (int): The number of configurations to skip. Defaults to 0. Yields: FeedbackConfigSchema: The feedback configurations. Example: .. code-block:: python from langsmith import Client client = Client() for config in client.list_feedback_configs(): print(f"{config.feedback_key}: {config.feedback_config}") Nrjrrr}rrrr[rh)r0rrr{r)rrprrrrr rs relist_feedback_configszClient.list_feedback_configs"sB).(9S___s" "   #(F5M  $&3F? #"  $ $%8 $ H H    IAv1;;F;; ; ; ; QUe^^   ri)rBrcd|i}|||d<|||d<|dd|}tj|tjdi|S) aUpdate a feedback configuration. Only the provided fields will be updated; others remain unchanged. Args: feedback_key (str): The feedback key of the configuration to update. feedback_config (Optional[FeedbackConfig]): The new configuration values. is_lower_score_better (Optional[bool]): Whether a lower score is considered better. Returns: FeedbackConfigSchema: The updated feedback configuration. Raises: LangSmithNotFoundError: If no configuration exists for the given feedback key (HTTP 404). Example: .. code-block:: python from langsmith import Client client = Client() config = client.update_feedback_config( "user-rating", is_lower_score_better=True, ) rpNrBrrrrrhrrs reupdate_feedback_configzClient.update_feedback_config,#sL L   &&5D" # ,,AD( ),,  -   +H555.AAAAAricb|ddd|i}tj|dS)aDelete a feedback configuration. This performs a soft delete. The configuration can be recreated later with the same key. Args: feedback_key (str): The feedback key of the configuration to delete. Example: .. code-block:: python from langsmith import Client client = Client() client.delete_feedback_config("user-rating") rVrrprNr/rr0)rrpr2s redelete_feedback_configzClient.delete_feedback_config`#sE$,,  "L1-   +H55555ri) queue_idsrrrr$Iterator[ls_schemas.AnnotationQueue]c#K|dt|Dnd|||t|dndd}t|d|D]&\}}tjdi|V| |dz|krdS'dS) aList the annotation queues on the LangSmith API. Args: queue_ids (Optional[List[Union[UUID, str]]]): The IDs of the queues to filter by. name (Optional[str]): The name of the queue to filter by. name_contains (Optional[str]): The substring that the queue name should contain. limit (Optional[int]): The maximum number of queues to return. Yields: The annotation queues. Nc<g|]\}}t|d|dS)z queue_ids[r)rars rerZz1Client.list_annotation_queues..#s2UUUfa#0A00011UUUrirj)rrrr/annotation-queuesrr[rh)rr0rr{AnnotationQueue)rrrrrrr queues relist_annotation_queueszClient.list_annotation_queues{#s4(VU )@T@TUUUU*(-(9S___s   "  $ $%9& $ I I    HAu,    QUe^^  ri)rqueue_idrubric_instructions rubric_itemsrrr4Optional[list[ls_schemas.AnnotationQueueRubricItem]]%ls_schemas.AnnotationQueueWithDetailscR|||t|nttj|d}|||d<|ddd|D}t j|tjdi| S) a<Create an annotation queue on the LangSmith API. Args: name (str): The name of the annotation queue. description (Optional[str]): The description of the annotation queue. queue_id (Optional[Union[UUID, str]]): The ID of the annotation queue. rubric_instructions (Optional[str]): The rubric instructions for the annotation queue. rubric_items (Optional[list[AnnotationQueueRubricItem]]): The feedback configs to assign to this queue's rubric. Each item specifies a feedback_key and optional per-queue customization like description and value_descriptions. Returns: AnnotationQueue: The created annotation queue object. N)rrrrrrrci|] \}}||| Sryrhr5s rerz2Client.create_annotation_queue..#sAAA41a1=!Q===rirrh) rWr[rr/rrr0r{AnnotationQueueWithDetailsr)rrrrrrrr2s recreate_annotation_queuezClient.create_annotation_queue#s:&#+#7#h---S=N=N#6     ##/D ,,  AA4::<<AAA-   +H5554  mmoo   rils_schemas.AnnotationQueuecdt|d}|d||j}tj|t jdi|S)zRead an annotation queue with the specified `queue_id`. Args: queue_id (Union[UUID, str]): The ID of the annotation queue to read. Returns: AnnotationQueue: The annotation queue object. /annotation-queues/rr-rWrh)rar/rrr0r{rr)rrrr2s reread_annotation_queuezClient.read_annotation_queue#suJ(J)G)GII,, MM-   +H5554GGx}}GGGri)rrrrci}|||d<|||d<|||d<|||d<|ddt|d| }tj|dS) a{Update an annotation queue with the specified `queue_id`. Args: queue_id (Union[UUID, str]): The ID of the annotation queue to update. name (Optional[str]): The new name for the annotation queue. description (Optional[str]): The new description for the annotation queue. rubric_instructions (Optional[str]): The new rubric instructions for the annotation queue. rubric_items (Optional[list[AnnotationQueueRubricItem]]): The feedback configs to assign to this queue's rubric. Returns: None Nrrrrrrrrr/rarr0)rrrrrrrr2s reupdate_annotation_queuezClient.update_annotation_queue#s0 "  DL  ""-D   **=D& '  ##/D ,,  B(8Z"@"@ B B-   +H55555ric|ddt|dddi|j}tj|dS)zDelete an annotation queue with the specified `queue_id`. Args: queue_id (Union[UUID, str]): The ID of the annotation queue to delete. Returns: None rVrrrrrWNrX)rrr2s redelete_annotation_queuezClient.delete_annotation_queue $s_,,  B(8Z"@"@ B B1CT]C-   +H55555ri)rr%Optional[Sequence[ls_schemas.RunKey]]c||tjd|.dt|dd}dt|D}n`|Jt jdt d dt|dd }d t|D}ntjd|d || }tj|dS)aAdd runs to an annotation queue with the specified `queue_id`. Provide exactly one of `runs` or `run_ids`: - `runs` (preferred): each entry carries the run's full lookup key (`run_id`, `session_id`, `start_time`, and an optional `source_proposed_example_id`). This lets the run be located directly, without a scan, and is required for workspaces served by SmithDB. - `run_ids`: a plain list of run IDs. This path is deprecated and will be removed after Jan 31, 2027; prefer `runs`. See https://docs.langchain.com/langsmith/smithdb-sdk-migration#annotation-queues-add-runs. Args: queue_id (Union[UUID, str]): The ID of the annotation queue. run_ids (Optional[List[Union[UUID, str]]]): The IDs of the runs to be added to the annotation queue. runs (Optional[Sequence[RunKey]]): The runs to add, each with its full lookup key. Returns: None Nz+Provide exactly one of `runs` or `run_ids`.rrz /runs/by-keyc4g|]\}}t||Srh)r)r|r rs rerZz7Client.add_runs_to_annotation_queue..A$s'RRR3+C33RRRriaThe run_ids parameter of add_runs_to_annotation_queue() is deprecated and will be removed after Jan 31, 2027. Use the runs parameter with RunKey objects instead. See https://docs.langchain.com/langsmith/smithdb-sdk-migration#annotation-queues-add-runs for the migration guide.r6rrc Vg|]&\}}tt|d|d'S)zrun_ids[rrrs rerZz7Client.add_runs_to_annotation_queue..K$sC8>3HS/Q///2233rirr) rrrarrrrr/r0)rrrrr_ json_bodyr2s readd_runs_to_annotation_queuez#Client.add_runs_to_annotation_queue$s#:   3-=   U(J)G)GUUUDRR)D//RRRII  ME#     O(J)G)GNNNDBKGBTBTII-= ,,VT ,JJ+H55555ric |ddt|ddt|d}tj|dS)aCDelete a run from an annotation queue with the specified `queue_id` and `run_id`. Args: queue_id (Union[UUID, str]): The ID of the annotation queue. run_id (Union[UUID, str]): The ID of the run to be added to the annotation queue. Returns: None rVrrrrNr)rrrr2s re delete_run_from_annotation_queuez'Client.delete_run_from_annotation_queueU$s^,,  d(8Z"@"@ d dQWYaHbHb d d   +H55555rir%ls_schemas.RunWithAnnotationQueueInfocdt|dd}|d|d||j}tj|t jdi|S)aGet a run from an annotation queue at the specified index. Args: queue_id (Union[UUID, str]): The ID of the annotation queue. index (int): The index of the run to retrieve. Returns: RunWithAnnotationQueueInfo: The run at the specified index. Raises: LangSmithNotFoundError: If the run is not found at the given index. LangSmithError: For other API-related errors. rrrar-rmrWrh)rar/rrr0r{RunWithAnnotationQueueInfor)rrrrr2s reget_run_from_annotation_queuez$Client.get_run_from_annotation_queueh$s N(J)G)GMMM,,  ! !% ! !M-   +H5554GGx}}GGGri)rrrHOptional[Literal['needs_my_review', 'needs_others_review', 'completed']]/Iterator[ls_schemas.RunWithAnnotationQueueInfo]c#Kd|t|dndi}|||d<dt|dd}t|||D]&\}}t jd i|V| |d z|krdS'dS) aLList runs in an annotation queue with the specified `queue_id`. Args: queue_id (Union[UUID, str]): The ID of the annotation queue. status (Optional[Literal["needs_my_review", "needs_others_review", "completed"]]): Filter runs by review status. If None, returns runs across all review states. limit (Optional[int]): The maximum number of runs to return. Yields: The runs currently in the annotation queue, with queue metadata (e.g. ``added_at``, ``last_reviewed_time``). rNrjrrrrrr[rh)r0rarrr{r)rrrrrr_r rs relist_runs_from_annotation_queuez&Client.list_runs_from_annotation_queue$s. (9S___s   %F8 JXh %C%CJJJ 8 8f 8 M MNN  FAs7>>#>> > > > QUe^^  ri)rrr.rCr experimentsr ls_schemas.ComparativeExperimentcR|std|!||dj}|std|ptt j|||||p-t jt jj id}| ||dd<td | D} | d d d | i } tj| | } t!jdi| S)aCreate a comparative experiment on the LangSmith API. These experiments compare 2 or more experiment results over a shared dataset. Args: name (str): The name of the comparative experiment. experiments (Sequence[Union[UUID, str]]): The IDs of the experiments to compare. reference_dataset (Optional[Union[UUID, str]]): The ID of the dataset these experiments are compared on. description (Optional[str]): The description of the comparative experiment. created_at (Optional[datetime.datetime]): The creation time of the comparative experiment. metadata (Optional[Dict[str, Any]]): Additional metadata for the comparative experiment. id (Optional[Union[UUID, str]]): The ID of the comparative experiment. Returns: ComparativeExperiment: The created comparative experiment object. z$At least one experiment is required.Nrrz A reference dataset is required.)rrexperiment_idsrrr.rrrCci|]\}}|| Srhrhr5s rerz8Client.create_comparative_experiment..$s999DAq1a999rirz/datasets/comparativerLrrh)rrrrWr[rrrcrrrrr/rr0rr{ComparativeExperiment) rrrrrr.rCrrserr2 response_ds recreate_comparative_experimentz$Client.create_comparative_experiment$sN6 ECDD D  $ $ 1 1&q>!2!!" ! A?@@ @)DJLL)))$5&$T(9(=(=h>O>S(T(T    (0DM* %99DJJLL999::,,  #-   +H555]]__ /==*===rircL|}|dkp |j|kS)zCheck if the current workspace has the same handle as owner. Args: owner (str): The owner to check against. Returns: bool: True if the current tenant is the owner, False otherwise. r)r6 tenant_handle)rrsettingss re_current_tenant_is_ownerzClient._current_tenant_is_owner$s,%%''|>x5>>riactionls_utils.LangSmithUserErrorcjtjd|d|jd|S)NzCannot z% for another tenant. Current tenant: z, Requested tenant: )rrr6r)rrrs re_owner_conflict_errorzClient._owner_conflict_error$sS* )f ) )#1133A ) )!& ) )   riprompt_owner_and_namec|dd|d||d}|d}|r|ddnd S) aGet the latest commit hash for a prompt. Args: prompt_owner_and_name (str): The owner and name of the prompt. limit (int, default=1): The maximum number of commits to fetch. Defaults to 1. offset (int, default=0): The number of commits to skip. Defaults to 0. Returns: Optional[str]: The latest commit hash, or None if no commits are found. r- /commits/rmrrcommitsrrFN)r/r)rrrrr2rs re_get_latest_commit_hashzClient._get_latest_commit_hash$sh,,  0- 0 0 0"f55-   --//),,3=wqz-((=ri commit_idUnion[str, list[str]]c"t|tr|gn|}dfd tj5fd|D}tj|D]}| ddddS#1swxYwYdS)aUpdate tags for a prompt commit. Args: prompt_owner_and_name (str): The owner and name of the prompt in the format 'owner/repo'. commit_id (str): The commit hash/ID to tag. tags (Union[str, list[str]]): A single tag or list of tags to apply to the commit. Raises: requests.exceptions.HTTPError: If the request fails. rrWcr|d}ddd|}tj|dS)N)tag_namerr/repos/rrr)rrr2rrrs re create_tagz.Client._create_commit_tags..create_tag%s^&G00>"7>>>W1H  / 9 9 9 9 9ric<g|]}|Srhr)r|rr rAs rerZz.Client._create_commit_tags..$%s'LLLCxz377LLLriN)rrW)r rWrr>r?r) rrrrtag_listrrAr rAs ``` @@re_create_commit_tagszClient._create_commit_tags%s(c22?? ?.   44 sAAA21A2rreadmerr r1ls_schemas.Promptc|}|r|jstjdtj|\}}} ||s|d|||pd|pd|pg|d} |dd| } | tj d i| d S) aCreate a new prompt. Does not attach prompt object, just creates an empty prompt. Args: prompt_identifier (str): The identifier of the prompt. The identifier should be in the formatof owner/name:hash, name:hash, owner/name, or name description (Optional[str]): A description of the prompt. readme (Optional[str]): A readme for the prompt. tags (Optional[Sequence[str]]): A list of tags for the prompt. is_public (bool): Whether the prompt should be public. Returns: Prompt: The created prompt object. Raises: ValueError: If the current tenant is not the owner. HTTPError: If the server request fails. Cannot create a public prompt without first creating a LangChain Hub handle. You can add a handle by creating a public prompt at: https://smith.langchain.com/prompts)rzcreate a promptr) repo_handlerr1rr rr rrErh) r6rrrrrrr/rr{r.r) rrrr1rr rrrrrr2s re create_promptzClient.create_prompt%s8%%''  X3 -6 !) @AR S S{A,,5,99 G,,->FF F'&,"lJB" < < ,,VYT,JJ!!### ;;8==??6#:;;;riparent_commit_hashrrrr8c||stjdt|}t |t r d|vrd|vr|}nD ddlm}n#t$rtdwxYw||} tj | }tj |\} } } | d| } |dks|| | }||d }|||d <| d d | |}| d}|d}|d}|r|| |||| d|S)aCreate a commit for an existing prompt. Args: prompt_identifier (str): The identifier of the prompt. object (Any): The LangChain object to commit. parent_commit_hash (Optional[str]): The hash of the parent commit. Defaults to latest commit. tags (Optional[str | list[str]]): A single tag or list of tags to apply to the commit. Defaults to None. description (Optional[str]): Optional human-readable description for the commit (max 1000 chars). Defaults to None. Returns: str: The url of the prompt commit. Raises: HTTPError: If the server request fails. ValueError: If the prompt does not exist. z0Prompt does not exist, you must create it first.rrr)rvThe client.create_commit function requires the langchain-corepackage to run. Install with `pip install langchain-core`rmlatestN) parent_commitrrrrrcommitrFri)rrrpprep_obj_for_pushr r langchain_core.loadrrrr'rrr/r r)rrrr8rrprepped manifest_dictr json_objectrrrr request_dictr2 commit_jsonrFrs re create_commitzClient.create_commit%s8""#455 1B  $F++ gt $ $ 4TW__#MM 5555555   !Q   %..K J{33M ( @AR S S{A#( 8 8; 8 8  ) )-?-G!%!=!=>S!T!T 0%( (   "*5L ',, 7 577l-  mmooh/ !-0 %  M  $ $%:It L L L##'<$L$L{$L$LMMMs A!!A;)rr1rr r!c`|}|r|jstdi}|||d<|||d<|||d<|||d<|||d<tj|\} } } |dd | d | | } | | S) a$Update a prompt's metadata. To update the content of a prompt, use push_prompt or create_commit instead. Args: prompt_identifier (str): The identifier of the prompt to update. description (Optional[str]): New description for the prompt. readme (Optional[str]): New readme for the prompt. tags (Optional[Sequence[str]]): New list of tags for the prompt. is_public (Optional[bool]): New public status for the prompt. is_archived (Optional[bool]): New archived status for the prompt. Returns: Dict[str, Any]: The updated prompt data as returned by the server. Raises: ValueError: If the prompt_identifier is empty. HTTPError: If the server request fails. r4Nrr1r r!rrr rmr)r6rrrrr/rr) rrrr1rr r!rrrrrr2s re update_promptzClient.update_prompt<&s:%%''  X3 6 <>  ""-D   #DN  )D   ""-D   DL ( @AR S S{A,, 4u44{444-   !!###}}rictj|\}}}||s|d||dd|d|}|dS)a6Delete a prompt. Args: prompt_identifier (str): The identifier of the prompt to delete. Returns: bool: True if the prompt was successfully deleted, False otherwise. Raises: ValueError: If the current tenant is not the owner of the prompt. zdelete a promptrVr rmN)rrrrr/rr/s re delete_promptzClient.delete_promptv&s!) @AR S S{A,,U33 G,,->FF F,,X7V7V7V7V7VWW!!#####rirc|rdnd}||S)zGenerate a cache key for a prompt. Args: prompt_identifier: The prompt identifier. include_model: Whether model info is included. Returns: The cache key string. z :with_modelrrh)rrrrqs re_get_cache_keyzClient._get_cache_key&s##07R#-V---rils_schemas.PromptCommitc tj|\}}}|dd|d|d||rdnd}tjdi||d|S)zFetch a prompt directly from the API (no cache). Args: prompt_identifier: The prompt identifier. include_model: Whether to include model information. Returns: The fetched PromptCommit. r-rrmz?include_model=truerrrErh)rrr/r{ PromptCommitr)rrrrrrFr2s re_fetch_prompt_from_apizClient._fetch_prompt_from_api&s+3*J + + '{K,, DEDDKDD+D,9A((rDD  &  {FFhmmooF   rir skip_cacherrRrcZt||t|j||}|s<|j5|||}|j||}||S|}|s9|j2|||}|j||||S)amPull a prompt object from the LangSmith API. Public prompts referenced by owner/name cross a trust boundary because the prompt manifest may contain serialized LangChain objects and configuration that affect runtime behavior. For example, a prompt can intentionally configure a model with a custom base URL, headers, model name, or other constructor arguments. These are supported features, but they also mean the prompt contents should be treated as executable configuration rather than plain text. Set `dangerously_pull_public_prompt=True` only after reviewing and trusting the prompt contents, not merely the publishing account. Prompts from your own or your organization's account can still be unsafe if that account or prompt was compromised. Args: prompt_identifier: The identifier of the prompt. include_model: Whether to include model information. skip_cache: Whether to skip the prompt cache. Defaults to `False`. dangerously_pull_public_prompt: Set to `True` to allow pulling a public prompt by owner/name, for example `username/promptname`. Defaults to `False`. Returns: The prompt object. Raises: ValueError: If no commits are found for the prompt. )r)rr rPr-rKr{r) rrrrRr refresh_func cache_keycachedrs repull_prompt_commitzClient.pull_prompt_commit&sJ % +I      '):M   dk5++,=}MMI[__Y ==F!  =dk5++,=}MMI KOOIv| < < < rirrr'Iterator[ls_schemas.ListedPromptCommit]c#Ktj|\}}}|td|n|||d}d} ||d<|dd|d |d | } | } | d } | d } | sdS| D]*}| | |krdSt jdi||d |V| dz } +|t| z }|| krdS)aList commits for a given prompt. Args: prompt_identifier (str): The identifier of the prompt in the format 'owner/repo_name'. limit (Optional[int]): The maximum number of commits to return. If None, returns all commits. offset (int, default=0): The number of commits to skip before starting to return results. include_model (bool, default=False): Whether to include the model information in the commit data. Yields: A ListedPromptCommit object for each commit. !!! note This method uses pagination to retrieve commits. It will make multiple API calls if necessary to retrieve all commits or up to the specified limit. NrjrXrTrr-rrmrrrurNr[rh)rrr0r/rr{ListedPromptCommitrp)rrrrrrrrrr r2valrruits relist_prompt_commitszClient.list_prompt_commits&sO0!) @AR S S{A).(9Se___u*    %F8 002E22K2221H --//C NELE   $eFF 3 %{AAbAQ c%jj F/ ri)rr r"rRrrr r!r"cZ|||||}t||||S)a Pull a prompt and return it as a LangChain `PromptTemplate`. This method requires [`langchain-core`](https://pypi.org/project/langchain-core). Args: prompt_identifier: The identifier of the prompt. include_model: Whether to include model configuration in the loaded prompt. secrets: A map of secrets to use for explicit serialized LangChain secret references in the manifest, e.g. `{'OPENAI_API_KEY': 'sk-...'}`. If a manifest secret reference is not found in the map, it will be loaded from the environment only if `secrets_from_env` is `True`. Deserialized model integrations may still use their own environment-based credential defaults during initialization. secrets_from_env: Whether explicit serialized LangChain secret references in the manifest may be loaded from environment variables during deserialization. skip_cache: Whether to skip the prompt cache. Defaults to `False`. dangerously_pull_public_prompt: Set to `True` to allow pulling a public prompt by owner/name (for example, `username/promptname`). Only do this for trusted prompts. Defaults to `False`. Returns: Any: The prompt object in the specified format. !!! warning "Security note" Pulled prompts should be treated as executable configuration, not plain text. The `secrets` and `secrets_from_env` arguments only control explicit serialized LangChain secret references in the manifest. They do not prevent deserialized model integrations from using their own environment-based credential defaults during initialization. For example, a deserialized OpenAI chat model may still use `OPENAI_API_KEY` from the environment if its constructor supports that default. Avoid pulling public prompts or prompts outside your own organization unless you have reviewed and trust their contents. When you do pull a trusted external prompt, prefer pinning to a specific commit SHA rather than following a mutable latest version. This is especially important when `include_model=True`. !!! warning "Behavior changed in `langsmith` 0.5.1" Updated to take arguments `secrets` and `secrets_from_env` which default to None and False, respectively. By default, explicit serialized LangChain secret references in a pulled manifest are not resolved from environment variables unless you specify `secrets_from_env=True`. These updates were made to remediate vulnerability [GHSA-c67j-w6g6-q2cm](https://github.com/langchain-ai/langchain/security/advisories/GHSA-c67j-w6g6-q2cm) in the `langchain-core` package which this method (but not the entire langsmith package) depends on. rQ)rr r")rWrQ)rrrr r"rRrrs re pull_promptzClient.pull_prompt-'sPH// '!+I 0  ( '-     rir;)rr8r rr1r commit_tagscommit_description Optional[Any]rarbc@||r8td||||fDr||||||n||||nd||||||S|||||| } | S)aPush a prompt to the LangSmith API. Can be used to update prompt metadata or prompt content. If the prompt does not exist, it will be created. If the prompt exists, it will be updated. Args: prompt_identifier (str): The identifier of the prompt. object (Optional[Any]): The LangChain object to push. parent_commit_hash (str): The parent commit hash. Defaults to "latest". is_public (Optional[bool]): Whether the prompt should be public. If None (default), the current visibility status is maintained for existing prompts. For new prompts, None defaults to private. Set to True to make public, or False to make private. description (Optional[str]): A description of the prompt. Defaults to an empty string. readme (Optional[str]): A readme for the prompt. Defaults to an empty string. tags (Optional[Sequence[str]]): A list of tags for the prompt. Defaults to an empty list. commit_tags (Optional[str | list[str]]): A single tag or list of tags for the prompt commit. Defaults to an empty list. commit_description (Optional[str]): Optional human-readable description for the commit (max 1000 chars). Defaults to None. Returns: str: The URL of the prompt. c3K|]}|duV dSryrh)r|params rer z%Client.push_prompt..'s8&+T!rir0NF)r rr1r)rr7)rrrGr6rrE) rrrr8r rr1rrarbrgs re push_promptzClient.push_prompt~'s X   0 1 1 09;PT/U ""% +!' #   !'0'<))%'     >'':K'LL L   1* !   rir identifierrls_schemas.AgentContextcp||d|}tj|S)a Pull an agent from Hub. Args: identifier: Repo identifier (owner/name:hash, owner/name, or name). version: Commit hash or tag; overrides any hash in identifier. Returns: AgentContext: The agent snapshot. agentrh)_pull_hub_directoryr{ AgentContextmodel_validaterrirrLs re pull_agentzClient.pull_agent's5'' GW'MM&55d;;;ri)r<rr1rr rM%dict[str, Optional[ls_schemas.Entry]]r<c <||d||||||S)z=Push an agent to Hub, creating the repo if it does not exist.rlrMr<rr1rr _push_hub_directoryrrirMr<rr1rr s re push_agentzClient.push_agent'9''  '#(   rils_schemas.SkillContextcp||d|}tj|S)zPull a skill from Hub.skillrh)rmr{ SkillContextrorps re pull_skillzClient.pull_skill's5'' GW'MM&55d;;;ric <||d||||||S)zPush a skill to Hub.r|rtrurws re push_skillzClient.push_skill'ryric0||dS)z/Delete an agent and its owned child file repos.N_delete_hub_directoryrris re delete_agentzClient.delete_agent( "":.....ric0||dS)z.Delete a skill and its owned child file repos.Nrrs re delete_skillzClient.delete_skill(rric^tj|\}}}|||S)zCheck if an agent repo exists.rparse_hub_identifier_hub_repo_existsrrirrrs re agent_existszClient.agent_exists(.!6zBBtQ$$UD111ric^tj|\}}}|||S)zCheck if a skill repo exists.rrs re skill_existszClient.skill_exists"(rrirrr r!rc8|d|||||S)zList agents with pagination.rlr_list_hub_reposrrrr r!rs re list_agentszClient.list_agents'(3## # $   ric8|d|||||S)zList skills with pagination.r|rrrs re list_skillszClient.list_skills:(rri repo_typeLiteral['agent', 'skill']c tj|\}}}||n |dkr|nd}d|i}|r||d<|dt|jd|d|d|} | S) zDFetch hub directory payload, merged with owner/repo from identifier.Nr;rr=r-rm /directoriesr)rrr/r7rkr) rrirrrrr=r`rr2s rermzClient._pull_hub_directoryM(s';JGGtV*GG6X;M;MSW #.y!9  &%F8 ,,  .. K K K K K K K-   }}ric t|tj|\} } } || s|d|| || | r9t d||||fDr|| | ||||natj | s%tj d| dtj d| | ||||t|i} |D]&\} }|d| | < |d | | <'d | i}|||d <|d t#|jd | d | d|}|dd}|}t+|j| ||jS)zFCreate a hub directory commit, creating the repo if it does not exist.zpush c3K|]}|duV dSryrh)r|rs rer z-Client._push_hub_directory..w(s&QQQ1D=QQQQQQrir0zInvalid repo_handle z : must match rNTrrMr<rrmz/directories/commitsrr=rF)r8rrrrrr_update_hub_repo_metadatar5matchrpattern_create_hub_reporrrr/r7rkrr6r6rr)rrirrMr<rr1rr rrr request_filesr_entryrr2rFrs rervzClient._push_hub_directoryc(sR }---!6zBBtQ,,U33 I,,-@Y-@-@%HH H   - - QQ;i*PQQQQQ .. +!' /',T22 1A4AA"5"=AAA  ! !'y// "   >@  ;;== J JKD%}&* d##&+&6&6D&6&I&I d## '7  $$1D !,,  .. S S S S S S S-   mmooh/ > %%''k8;OOOric tj|\}}}||s|d||dt |jd|d|ddS)zDelete a hub directory repo.deleterVrmrN)rrrrr/r7rkrs rerzClient._delete_hub_directory(s!6zBBtQ,,U33 >,,Xu== = !!  .. K K K K K K K     ric||||rdndd}| |rdnd|d<|r ||d<d|d<|dt| }tjd i|S) uList hub repos filtered by type. Returns ``ListPromptsResponse`` because ``/repos`` is polymorphic — the list shape is shared across prompt, agent, and skill repos. r\r()rrrr!Nr rr)r-rrh)r/r4r{r*r) rrrrr r!rrr2s rerzClient._list_hub_repos(s "%0=66g " "   ,5"B&&7F;   ,#F7O%+F> ",,UC,GG-@@ @@@ricz |dtd|d|dS#tj$rYdSwxYw)zCheck if a hub repo exists.r-rmTF)r/r4rrp)rrrs rerzClient._hub_repo_exists(s]   % %e-D-De-D-Dd-D-D E E E4.   55 s #'::c|||d}|||d<|||d<|t||d< |dd|dS#tj$rYdSwxYw) z(Create a new hub repo of the given type.)r5rr Nrr1rrr r)rr/rrq)rrrrr1rr rs rerzClient._create_hub_repo(s ""    ""-D   #DN  ::DL   % %fid % C C C C C.    DD sAAAci}|||d<|||d<|t||d<|||d<|r'|dtd|d||dSdS) z=Patch hub repo metadata fields that were explicitly provided.Nrr1rr rrmr)rr/r4)rrrrr1rr rs rerz Client._update_hub_repo_metadata(s "  ""-D   #DN  ::DL  )D   S  % %g#/F/F/F/F/F/FT % R R R R R S Sric ||n2#t$r%}td|Yd}~nd}~wwxYwd|_|j|jdSdS)aManually trigger cleanup of background threads. Drains pending traces via ``flush()`` before stopping the background threads. Pass ``timeout=0`` to skip the drain entirely (e.g. in error paths or signal handlers where blocking on network I/O is unacceptable). Args: timeout: Maximum seconds to wait for pending traces to flush. None (default) waits indefinitely. rz(Error flushing traces during cleanup: %sNT)rr@rarbrr-shutdown)rrrMs recleanupzClient.cleanup)s J JJwJ ' ' ' ' J J J NNEq I I I I I I I I J# ; " K " " " " " # "s AAAc ||n2#t$r%}td|Yd}~nd}~wwxYw|j}|O t j|n2#t$r%}td|Yd}~nd}~wwxYwd|_|j}|E t|dS#t$r&}td|Yd}~dSd}~wwxYwdS)aVRelease resources held by this client. Calls :meth:`cleanup` to drain pending traces and stop background threads, then closes the underlying ``requests.Session`` and unregisters the ``atexit`` handler so the session is not pinned in memory for the remainder of the process lifetime. Safe to call multiple times. Args: timeout: Forwarded to :meth:`cleanup` / :meth:`flush`. Maximum seconds to wait for pending traces to flush. ``None`` (default) waits indefinitely. Pass ``0`` to skip the drain. rz-Error during cleanup while closing client: %sNz&Error unregistering atexit handler: %sz Error closing client session: %s) rr@rarbrr unregisterrrsr)rrrMhandlerrss rerz Client.close)sV O LLL ) ) ) ) O O O NNJA N N N N N N N N O&   J!'**** J J J EqIIIIIIII J#'D ,   Fg&&&&& F F FA1EEEEEEEEE F  sD AAAA** B4BB-B>> C.C))C.rLOptional[DATA_T]rOptional[Sequence[EVALUATOR_T]]summary_evaluators'Optional[Sequence[SUMMARY_EVALUATOR_T]]experiment_prefixblocking experimentOptional[EXPERIMENT_T]upload_resultsr`'Union[TARGET_T, Runnable, EXPERIMENT_T]rc  dSryrhrr`rLrrrCrrrrrrrrs reevaluatezClient.evaluate9)s " Cri+Optional[Sequence[COMPARATIVE_EVALUATOR_T]](Union[tuple[EXPERIMENT_T, EXPERIMENT_T]]rc  dSryrhrs rerzClient.evaluateL)s "(+srir IOptional[Union[Sequence[EVALUATOR_T], Sequence[COMPARATIVE_EVALUATOR_T]]]error_handlingLiteral['log', 'ignore']JUnion[TARGET_T, Runnable, EXPERIMENT_T, tuple[EXPERIMENT_T, EXPERIMENT_T]]6Union[ExperimentResults, ComparativeExperimentResults]c <ddlm}||f|||||||| || | | | d |S)aEvaluate a target system on a given dataset. Args: target (Union[TARGET_T, Runnable, EXPERIMENT_T, Tuple[EXPERIMENT_T, EXPERIMENT_T]]): The target system or experiment(s) to evaluate. Can be a function that takes a `dict` and returns a `dict`, a langchain `Runnable`, an existing experiment ID, or a two-tuple of experiment IDs. data (DATA_T): The dataset to evaluate on. Can be a dataset name, a list of examples, or a generator of examples. evaluators (Optional[Union[Sequence[EVALUATOR_T], Sequence[COMPARATIVE_EVALUATOR_T]]]): A list of evaluators to run on each example. The evaluator signature depends on the target type. Default to None. summary_evaluators (Optional[Sequence[SUMMARY_EVALUATOR_T]]): A list of summary evaluators to run on the entire dataset. Should not be specified if comparing two existing experiments. metadata (Optional[dict]): Metadata to attach to the experiment. experiment_prefix (Optional[str]): A prefix to provide for your experiment name. description (Optional[str]): A free-form text description for the experiment. max_concurrency (Optional[int], default=0): The maximum number of concurrent evaluations to run. If `None` then no limit is set. If `0` then no concurrency. blocking (bool, default=True): Whether to block until the evaluation is complete. num_repetitions (int, default=1): The number of times to run the evaluation. Each item in the dataset will be run and evaluated this many times. Defaults to 1. experiment (Optional[EXPERIMENT_T]): An existing experiment to extend. If provided, `experiment_prefix` is ignored. For advanced usage only. Should not be specified if target is an existing experiment or two-tuple fo experiments. upload_results (bool, default=True): Whether to upload the results to LangSmith. error_handling (str, default="log"): How to handle individual run errors. `'log'` will trace the runs with the error message as part of the experiment, `'ignore'` will not count the run as part of the experiment at all. **kwargs (Any): Additional keyword arguments to pass to the evaluator. Returns: ExperimentResults: If target is a function, Runnable, or existing experiment. ComparativeExperimentResults: If target is a two-tuple of existing experiments. Examples: Prepare the dataset: ```python from langsmith import Client client = Client() dataset = client.clone_public_dataset( "https://smith.langchain.com/public/419dcab2-1d66-4b94-8901-0357ead390df/d" ) dataset_name = "Evaluate Examples" ``` Basic usage: ```python def accuracy(outputs: dict, reference_outputs: dict) -> dict: # Row-level evaluator for accuracy. pred = outputs["response"] expected = reference_outputs["answer"] return {"score": expected.lower() == pred.lower()} ``` ```python def precision(outputs: list[dict], reference_outputs: list[dict]) -> dict: # Experiment-level evaluator for precision. # TP / (TP + FP) predictions = [out["response"].lower() for out in outputs] expected = [ref["answer"].lower() for ref in reference_outputs] # yes and no are the only possible answers tp = sum([p == e for p, e in zip(predictions, expected) if p == "yes"]) fp = sum([p == "yes" and e == "no" for p, e in zip(predictions, expected)]) return {"score": tp / (tp + fp)} def predict(inputs: dict) -> dict: # This can be any function or just an API call to your app. return {"response": "Yes"} results = client.evaluate( predict, data=dataset_name, evaluators=[accuracy], summary_evaluators=[precision], experiment_prefix="My Experiment", description="Evaluating the accuracy of a simple prediction model.", metadata={ "my-prompt-version": "abcd-1234", }, ) ``` Evaluating over only a subset of the examples ```python experiment_name = results.experiment_name examples = client.list_examples(dataset_name=dataset_name, limit=5) results = client.evaluate( predict, data=examples, evaluators=[accuracy], summary_evaluators=[precision], experiment_prefix="My Experiment", description="Just testing a subset synchronously.", ) ``` Streaming each prediction to more easily + eagerly debug. ```python results = client.evaluate( predict, data=dataset_name, evaluators=[accuracy], summary_evaluators=[precision], description="I don't even have to block!", blocking=False, ) for i, result in enumerate(results): # doctest: +ELLIPSIS pass ``` View the evaluation results for experiment:... Evaluating a LangChain object: ```python from langchain_core.runnables import chain as as_runnable @as_runnable def nested_predict(inputs): return {"response": "Yes"} @as_runnable def lc_predict(inputs): return nested_predict.invoke(inputs) results = client.evaluate( lc_predict, data=dataset_name, evaluators=[accuracy], description="This time we're evaluating a LangChain object.", summary_evaluators=[precision], ) ``` Comparative evaluation: ```python results = client.evaluate( # The target is a tuple of the experiment IDs to compare target=( "12345678-1234-1234-1234-123456789012", "98765432-1234-1234-1234-123456789012", ), evaluators=[accuracy], summary_evaluators=[precision], ) ``` Evaluate an existing experiment: ```python results = client.evaluate( # The target is the ID of the experiment we are evaluating target="12345678-1234-1234-1234-123456789012", evaluators=[accuracy], summary_evaluators=[precision], ) ``` !!! version-added "Added in `langsmith` 0.2.0" r)r rLrrrCrrrrclientrrrr)langsmith.evaluation._runnerr)rr`rLrrrCrrrrrrrrr evaluate_s rerzClient.evaluate_)sl\ GFFFFFy  !1/#++!))     riNUnion[DATA_T, AsyncIterable[schemas.Example], Iterable[schemas.Example], None]4Optional[Sequence[Union[EVALUATOR_T, AEVALUATOR_T]]]6Optional[Union[schemas.TracerSession, str, uuid.UUID]]VUnion[ATARGET_T, AsyncIterable[dict], Runnable, str, uuid.UUID, schemas.TracerSession]rcLKddlm}||f|||||||| || | | | d |d{VS)aiEvaluate an async target system on a given dataset. Args: target (Union[ATARGET_T, AsyncIterable[dict], Runnable, str, uuid.UUID, TracerSession]): The target system or experiment(s) to evaluate. Can be an async function that takes a `dict` and returns a `dict`, a langchain `Runnable`, an existing experiment ID, or a two-tuple of experiment IDs. data (Union[DATA_T, AsyncIterable[Example]]): The dataset to evaluate on. Can be a dataset name, a list of examples, an async generator of examples, or an async iterable of examples. evaluators (Optional[Sequence[EVALUATOR_T]]): A list of evaluators to run on each example. summary_evaluators (Optional[Sequence[SUMMARY_EVALUATOR_T]]): A list of summary evaluators to run on the entire dataset. metadata (Optional[dict]): Metadata to attach to the experiment. experiment_prefix (Optional[str]): A prefix to provide for your experiment name. description (Optional[str]): A description of the experiment. max_concurrency (Optional[int], default=0): The maximum number of concurrent evaluations to run. If `None` then no limit is set. If `0` then no concurrency. num_repetitions (int, default=1): The number of times to run the evaluation. Each item in the dataset will be run and evaluated this many times. Defaults to 1. blocking (bool, default=True): Whether to block until the evaluation is complete. experiment (Optional[TracerSession]): An existing experiment to extend. If provided, `experiment_prefix` is ignored. For advanced usage only. upload_results (bool, default=True): Whether to upload the results to LangSmith. error_handling (str, default="log"): How to handle individual run errors. `'log'` will trace the runs with the error message as part of the experiment, `'ignore'` will not count the run as part of the experiment at all. **kwargs (Any): Additional keyword arguments to pass to the evaluator. Returns: An async iterator over the experiment results. Environment: - `LANGSMITH_TEST_CACHE`: If set, API calls will be cached to disk to save time and cost during testing. Recommended to commit the cache files to your repository for faster CI/CD runs. Requires the `'langsmith[vcr]'` package to be installed. Examples: Prepare the dataset: ```python import asyncio from langsmith import Client client = Client() dataset = client.clone_public_dataset( "https://smith.langchain.com/public/419dcab2-1d66-4b94-8901-0357ead390df/d" ) dataset_name = "Evaluate Examples" ``` Basic usage: ```python def accuracy(outputs: dict, reference_outputs: dict) -> dict: # Row-level evaluator for accuracy. pred = outputs["resposen"] expected = reference_outputs["answer"] return {"score": expected.lower() == pred.lower()} def precision(outputs: list[dict], reference_outputs: list[dict]) -> dict: # Experiment-level evaluator for precision. # TP / (TP + FP) predictions = [out["response"].lower() for out in outputs] expected = [ref["answer"].lower() for ref in reference_outputs] # yes and no are the only possible answers tp = sum([p == e for p, e in zip(predictions, expected) if p == "yes"]) fp = sum([p == "yes" and e == "no" for p, e in zip(predictions, expected)]) return {"score": tp / (tp + fp)} async def apredict(inputs: dict) -> dict: # This can be any async function or just an API call to your app. await asyncio.sleep(0.1) return {"response": "Yes"} results = asyncio.run( client.aevaluate( apredict, data=dataset_name, evaluators=[accuracy], summary_evaluators=[precision], experiment_prefix="My Experiment", description="Evaluate the accuracy of the model asynchronously.", metadata={ "my-prompt-version": "abcd-1234", }, ) ) ``` Evaluating over only a subset of the examples using an async generator: ```python async def example_generator(): examples = client.list_examples(dataset_name=dataset_name, limit=5) for example in examples: yield example results = asyncio.run( client.aevaluate( apredict, data=example_generator(), evaluators=[accuracy], summary_evaluators=[precision], experiment_prefix="My Subset Experiment", description="Evaluate a subset of examples asynchronously.", ) ) ``` Streaming each prediction to more easily + eagerly debug. ```python results = asyncio.run( client.aevaluate( apredict, data=dataset_name, evaluators=[accuracy], summary_evaluators=[precision], experiment_prefix="My Streaming Experiment", description="Streaming predictions for debugging.", blocking=False, ) ) async def aenumerate(iterable): async for elem in iterable: print(elem) asyncio.run(aenumerate(results)) ``` Running without concurrency: ```python results = asyncio.run( client.aevaluate( apredict, data=dataset_name, evaluators=[accuracy], summary_evaluators=[precision], experiment_prefix="My Experiment Without Concurrency", description="This was run without concurrency.", max_concurrency=0, ) ) ``` Using Async evaluators: ```python async def helpfulness(outputs: dict) -> dict: # Row-level evaluator for helpfulness. await asyncio.sleep(5) # Replace with your LLM API call return {"score": outputs["output"] == "Yes"} results = asyncio.run( client.aevaluate( apredict, data=dataset_name, evaluators=[helpfulness], summary_evaluators=[precision], experiment_prefix="My Helpful Experiment", description="Applying async evaluators example.", ) ) ``` Evaluate an existing experiment: ```python results = asyncio.run( client.aevaluate( # The target is the ID of the experiment we are evaluating target="419dcab2-1d66-4b94-8901-0357ead390df", evaluators=[accuracy, helpfulness], summary_evaluators=[precision], ) ) ``` !!! version-added "Added in `langsmith` 0.2.0" r) aevaluaterN)langsmith.evaluation._arunnerr)rr`rLrrrCrrrrrrrrr aevaluate_s rerzClient.aevaluateC*sN JIIIIIZ  !1/#++!))           ripreviewfilters!dict[uuid.UUID, list[str]] | NoneIterator[list[ExampleWithRuns]]c#Kd}d} |r||z nd} | rtd| nd} |g|| |||d} |dd|dd t| i } | } | sdSd | D}|V|t | z }t | | ks|r||krdS|t | z }) a%Paginate through examples with runs and yield batches. Args: dataset_id: Dataset UUID to fetch examples with runs session_id: Session UUID to filter runs by, same as project_id preview: Whether to return preview data only comparative_experiment_id: Optional comparative experiment UUID filters: Optional filters to apply limit: Maximum total number of results to return Yields: Batches of run results as lists of ExampleWithRuns instances rTNrj) session_idsrrrrPrrrkrrLrc0g|]}tjdi|Sr)r{rO)r|rs rerZz7Client._paginate_examples_with_runs..o+s)WWWvj8BB6BBWWWri)r0r/rrrp)rrgrrrPrrr results_countr batch_limitrr2rexamples_batchs re_paginate_examples_with_runsz#Client._paginate_examples_with_runs>+s,,  !38B..dI1:C#c9---K!+| $"-F" D00.Z... & D(9(9:1H MMOOE XWQVWWWN SZZ 'M5zzK''E'mu6L6L c%jj F? !rizget_experiment_results() is deprecated and will be removed after Jan 31, 2027. Use client.datasets.experiment_runs.query() instead. See https://docs.langchain.com/langsmith/smithdb-sdk-migration#dataset-experiment-runs-query for the migration guide.ls_schemas.ExperimentResultsc ||d std|d fd} j j j j j j j j j j j j j jd}i jpi jpi} t%j| ||S)aGet results for an experiment, including experiment session aggregated stats and experiment runs for each dataset example. .. admonition:: Deprecated Use :meth:`langsmith.Client.datasets.experiment_runs.query` instead. See https://docs.langchain.com/langsmith/smithdb-sdk-migration#dataset-experiment-runs-query for the migration guide. Will be removed after Jan 31, 2027. Experiment results may not be available immediately after the experiment is created. Args: name: The experiment name. project_id: Experiment's tracing project id, also called session_id, can be found in the url of the LS experiment page preview: Whether to return lightweight preview data only. When True, fetches inputs_preview/outputs_preview summaries instead of full inputs/outputs from S3 storage. Faster and less bandwidth. comparative_experiment_id: Optional comparative experiment UUID for pairwise comparison experiment results. filters: Optional filters to apply to results limit: Maximum number of results to return Returns: ExperimentResults with: - feedback_stats: Combined feedback statistics including session-level feedback - run_stats: Aggregated run statistics (latency, tokens, cost, etc.) - examples_with_runs: Iterator of ExampleWithRuns Raises: ValueError: If project not found for the given session_id Example: ```python client = Client() results = client.get_experiment_results( project_id="037ae90f-f297-4926-b93c-37d8abf6899f", ) for example_with_runs in results["examples_with_runs"]: print(example_with_runs.model_dump()) # Access aggregated experiment statistics print(f"Total runs: {results['run_stats']['run_count']}") print(f"Total cost: {results['run_stats']['total_cost']}") print(f"P50 latency: {results['run_stats']['latency_p50']}") # Access feedback statistics print(f"Feedback stats: {results['feedback_stats']}") ``` T)rrrz&No experiment found with project_id: 'rc3lKjjD] }|Ed{V dS)z2Yield examples with corresponding experiment runs.)rgrrrPrrN)rrr)rrPrrrrrs re _get_examples_with_runs_iteratorzGClient.get_experiment_results.._get_examples_with_runs_iterator+si::"7":*C ; ! !!         ! !ri) run_count latency_p50 latency_p99rrr last_run_start_time run_facetsrrr first_token_p50first_token_p99 error_rate)r  run_statsexamples_with_runs)rrrrrrrr rrrrr rrrr session_feedback_statsr{r) rrrrrPrrrrr rs ` ```` @reget_experiment_resultszClient.get_experiment_resultsx+s:|##*D$   USjSSSTT T ! ! ! ! ! ! ! ! ! !!*".".#0$2!(!:#*#>!,!,".&6&6&6!,4 4  %+ -3 +)??AA    ri) instructionsrmodelopenai_api_keyanthropic_api_keychat_historieslist[list[dict]]rr%Literal['openai', 'anthropic'] | Nonerrls_schemas.InsightsReportc||||}|||}|d|dd|d}|dd|jd| } t j| | } tjd i| |j| |j d } td | j | S) a Generate Insights over your agent chat histories. !!! note - Only available to Plus and higher tier LangSmith users. - Insights Agent uses user's model API key. The cost of the report grows linearly with the number of chat histories you upload and the size of each history. For more see [insights](https://docs.langchain.com/langsmith/insights). - This method will upload your chat histories as traces to LangSmith. - If you pass in a model API key this will be set as a workspace secret meaning it will be usedin for evaluators and the playground. Args: chat_histories: A list of chat histories. Each chat history should be a list of messages. We recommend formatting these as OpenAI messages with a "role" and "content" key. Max length 1000 items. instructions: Instructions for the Insights agent. Should focus on what your agent does and what types of insights you want to generate. name: Name for the generated Insights report. model: Whether to use OpenAI or Anthropic models. This will impact the cost of generating the Insights Report. openai_api_key: OpenAI API key to use. Only needed if you have not already stored this in LangSmith as a workspace secret. anthropic_api_key: Anthropic API key to use. Only needed if you have not already stored this in LangSmith as a workspace secret. Example: ```python import os from langsmith import Client client = client() chat_histories = [ [ {"role": "user", "content": "how are you"}, {"role": "assistant", "content": "good!"}, ], [ {"role": "user", "content": "do you like art"}, {"role": "assistant", "content": "only Tarkovsky"}, ], ] report = client.generate_insights( chat_histories=chat_histories, name="Conversation Topics", instructions="What are the high-level topics of conversations users are having with the assistant?", openai_api_key=os.environ["OPENAI_API_KEY"], ) # client.poll_insights(report=report) ``` rrrzxThe run.outputs.messages field contains a chat history between the user and the agent. This is all the context you need.)z%How are your agent traces structured?z.What would you like to learn about your agent?r[)r user_context last_n_hoursrrrz /insightsrrrhost_urlzThe Insights Agent is running! This can take up to 30 minutes to complete. Once the report is completed, you'll be able to see results here: rh) _ensure_insights_api_key_ingest_insights_runsr/rrr0rr{InsightsReportrJrprintr) rrrrrrrrrr2rJreports regenerate_insightszClient.generate_insights+s0D--)/.   ,,^TBB:tBN  ,, 6666V-   +H555mmoo*   z))++^       {      rir\i)r rrraterverboser ls_schemas.InsightsReport | Noner r c |r|s|std|s|r|rtd|r|j}|j}td||z}t |D]}|dd|d|} t j| | } | ddkrGtj di| || |j d } td | j| cS| dd krtd | d |rtd ||ztj|t#d)a"Poll the status of an Insights report. Args: report: THe InsightsReport. id: The Insights report ID. Should only specify if 'report' is not specified. project_id: The Tracing project ID. Should only specify if 'report' is not specified. 1Must specify ('id' and 'project_id') or 'report'.@Must specify exactly one of ('id' and 'project_id') or 'report'.r[r-r /insights/rsuccessrz1Insights report completed! View the results at %srzFailed to generate insights: zPolling time: zInsights still pendingNrh)rrrrtrcr/rr0rr{r rJrr rr^rk TimeoutError) rr rrr rr max_triesr r2 resp_jsonjobs re poll_insightszClient.poll_insightsF,s$ +  +v +PQQ Q +J +F +R  +B*J7d?++ y!!  A00>J>>">>H  / 9 9 9 I"i// /)"1133!^  GH 8$// !U7AS!U!UVVV 31q4x11222 Jt    3444ri)rr r include_runsr ls_schemas.InsightsReportResultc|"||td|j}|j}n||td|}|}|dd|d|}t j||}|s*tjd i|} | |||| Stj ||||d<tjd i|} | |||| S) aGFetch an Insights report by ID or from a prior report object. Args: id: The Insights report ID (aka clustering job ID). Provide with ``project_id`` if ``report`` is not provided. report: An ``InsightsReport`` object returned by ``generate_insights`` or ``poll_insights``. If provided, ``id`` and ``project_id`` must be omitted. project_id: The tracing project (session) ID associated with the report. Required if ``report`` is not provided. include_runs: Whether to include all runs for the report. Returns: An ``InsightsReportResult`` with job metadata, clusters, summary report, and optionally ``runs``. Raises: ValueError: If the required identifiers are not provided. Nr r r-rr )rrjob_idrrh) rrrr/rr0rr{InsightsReportResult_attach_client_fetch_insights_runs) rrr rr r rresp report_jsonrs reget_insights_reportzClient.get_insights_report|,s?6  ~!7 VYF*JJzZ/ !TUUUF#J(( > >>f>>   +D111iikk  4CC{CCF  ! !$ F ; ; ;M(=!   F 0??;??dJ777 ri)rpriorityr list[dict]cd|i}|||d<|||d<t|jd}|d||}|S)aList issues associated with a tracing project (forge issues board). Retrieves all issues from the forge issues board that are linked to the given tracing project (identified by its session name). Args: project_name (str): The name of the tracing project (session) whose issues you want to list. status (Optional[str]): Filter issues by status (e.g. ``"open"``, ``"resolved"``). If ``None``, issues of all statuses are returned. priority (Optional[str]): Filter issues by priority (e.g. ``"high"``, ``"medium"``, ``"low"``). If ``None``, issues of all priorities are returned. Returns: List[dict]: A list of issue objects. Each dict contains the following keys: - ``id`` (str): Issue UUID. - ``tenant_id`` (str): Workspace/tenant UUID. - ``issue_board_id`` (str): UUID of the issue board this issue belongs to. - ``title`` (str): Issue title. - ``description`` (str): Issue description. - ``priority`` (str): Issue priority. - ``status`` (str): Issue status. - ``category`` (str | None): Optional category label. - ``trace_ids`` (List[str]): Run/trace IDs associated with the issue. - ``github_issue_url`` (str | None): URL of the linked GitHub issue. - ``github_issue_number`` (int | None): Number of the linked GitHub issue. - ``created_at`` (str): ISO-8601 creation timestamp. - ``updated_at`` (str): ISO-8601 last-updated timestamp. - ``resolved_at`` (str | None): ISO-8601 resolution timestamp, or ``None``. Example: ```python from langsmith import Client client = Client() # List all issues for a project issues = client.list_project_issues("my-project") # Filter by status and priority open_high = client.list_project_issues( "my-project", status="open", priority="high" ) for issue in open_high: print(issue["id"], issue["title"]) ``` rNrr issuesr-r)rrkr/r)rrrr rr_r2s relist_project_issueszClient.list_project_issues,shr#1,!?  %F8   !)F: dlH55,,UD,HH}}rirLiteral['openai', 'anthropic']c||dd}tj|d|D}t }|dvr|t |dvr|t||x}r t |vrdndS|dkr |}t } n8|dkr |}t} n(|s|r|p|}|rt nt} ntd|d d| |d g }tj|| t krdndS) Nr-z/workspaces/current/secretsc8h|]}|dSrrz)r|ss rerz2Client._ensure_insights_api_key..-s"@@@1!%%,,@@@ri)Nopenai)N anthropicr$ r% z1Must specify openai_api_key or anthropic_api_key.r)r}r~r) r/rr0rrr_OPENAI_API_KEY_ANTHROPIC_API_KEY intersectionr) rrrrr2workspace_keys target_keys existing_keysrapi_vars rerzClient._ensure_insights_api_key,s|,,U4QRR+H555@@ @@@ee $ $ $ OOO , , , ' ' ' OO. / / /*77 DD D= R.-??88[ P h  $G%GG k ! !'G(GG  R0 R$9(9G)7Ooo=OGGPQQ Q,,  )!G445-   +H555"o55xx;Fric4t|dkrtjd|dd}tjtjj||p3dtjdzdtt|D}fdt||D}| || S)NrZzDCan only generate insights over 1000 data. Truncating to first 1000.z insights z%Y-%m-%d %H:%M:%ScNg|]"}ttj#Srh)rWr[rrs rerZz0Client._ingest_insights_runs..*-s&???3tz||$$???ric g|]]\}}d|ddid|i||dt|tjdz djdd ^S)r5Nr[z%Y%m%dT%H%M%S%fZ)rrmr) rrrrrrrrrr)strftimerWrr.r)r|rrrcrs rerZz0Client._ingest_insights_runs..+-s   &q!u-&?"#&<<0B#C#C RS[[ R R!H$6q$A$A$AA#%j     ri)rY) rprrrrcrrrr0 rcrrir)rrLrrrrcrs @@rerzClient._ingest_insights_runs-s# t99t   MV   ;D##H$5$9::%%  Uh/3355>>?RSSS  @?eCII.>.>???     !$//    d+++ riry)8rkrrrr rGr rHr6rrsrIr7rr8rJr9rKr:rKr;rKr<rr=rLr>rMr?rrrNr@rrArOrrrrrrrBrrCrMrrrDrPrErrFrQrXrY)rXr)rXr)rXr)rXr)rXr)rXr)rXr)rXr)rXr)rXr)rXr)rrrr rXrY) rrWrrrrWrr rrMrXrY)rXrW)rr@rXrY)rXr)rrWr~rrXrYrXrY)rXr)r~rrXrY)r~rrXrY)rXr)rXr3)r7rMrXr)rDrErFrWr>rGr?rr@rHrArHrBrIrCrWrrrXrJ)r_rWrrrXr) r_rWrrrrrrWrXr)rrrrWrrrrrrrrrXr)rrrrrrrrrrrrrXr)FF)rrrDrrrrXr )rrrXrYrXr)rrrrrXr)rXr)rr*rXrY)rrWrrurrrrrrrrrrrkrrrrXrY)rr rrrkrrrrrrrrrrXrY)rr rrrkrrrrrrrrr)rr )rr )rr)rCr rXr )rr/rXr0)rBrCrkrrrrrrrrrrrrXrYr)rYrZrDrZrXrY)rrrCrWrkrrrrrrrrrrr)rBrvrkrrrrrrrrrrrrXrY)rYrZrDrZrrrXrY)rr:rrrkrrrrrrrrrrr)rrrrrr),rrrrrrrrrrrrrrrrrr/rrrrrrrrrrrrrkrrrrrrrrrrrrXrY)rr rrrkrrrrrrrrr)rr rrrkrrrrrrrrrrXrY)rN)rrrrrXrY)rrrXrY)rrrXrr) rrrrrrrrrXr)rrWrrrrrrrrMrrrrrrrrrXr)$rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrMrrrXr)rrrrrrMrrrrrrrXr!)"rr;rrrrrrr8rrr;r9r;rrrrrrrrrrrrrrrrr:rrXru)rrErrrrrXrW)rrrLrrXrW)rrrXrY)rrrXr)rrrXr)rNr^rrrXr)rNr^rrrXr)rgrrfrrXrh)rgrrXrY)rNrWrXr)rNrWryr;rrMrXrz) rrWrr;rrrrrrMrXr)rrWrrrCrrrrrrrrrMrrMrrrXr)rrrrrrrCrrrrrrXr)rXr)rXr)rrrrrrrXr)rrWrrrXr)rrrrrXr) NNNNNNNNNN)rr;rrrrrrrrrrrrrrrrMrCrrXr)rrrrrXrY)rfrWrrrrrrrrrrrCrrXr)rfrrgrrXr)rfrrgrrXr) rgrrfrrrrrrXr)rgrrfrrXr)rr;rrrfrr rrCrrrMrXr )rgrrfrrXrY) rgrrfrrrrrWrXrY) rgrrfrrrrrMrXr) rgrrfrrrrrrXr)r rWrrrfrrXr)rgrrXr)NNNN) rLrWr-rrgrrfrr.rrXr/) r5r6r7r8rgrrfrr.rrXr/rM) rrrgrrfrr.rrXr/)rrIrJrrrrXrK)rgrr`rarrrXrb)rgrrnrorrrXrb)rrtrXr)r)rrzrCrrXr{)rrrrrXrb)rfrrgrrrrrrrrrrXr) NNNNNNNNNFNN)rrrgrrfrr.rrrrCrr`rrrrPrrNrrOrrrrXr/)rrrrrXr/)NNNNNT)rgrrfrryrrrrrrrrrrrMrCrrrrrrrrXrz)rrrrrrrCrr`rrgrrWrrrrXru) rfrrgrr`rrrrrrXru)rrrXrY)ryrrrrXrY)rgrrfrrrrXr) rgrrfrr rWryr rrrXrY) rrrrrrrrrXr)rrrrrXr)rrrrrXr)rrrr1rrrrr0rrr2rrrXr!) r9rrr;r0rrrr:r<rXr)NrN).rrr}rWrrr~rSrrrArTr$rUr0rrCrVrPrrOrrBrWr?rrrrPrrQrrrrrrrrrrRrrrrXrX) rOrrrr~rSrArTr$rUrXrY)rOrrXrX) rrrprrCrqrrMrrrXrr)rOrrXrY)r rTrrr~rSrArTr$rUrCrrXrY) rrrprWryrzrBrWrOrrXr{) rrrrryrzrrrXr)rrrrMrXr) rgrrrrrMrrrXr)rrrXr) rprWrrrrrgrrrrXr) rrrprWrrrrrXr)rrrXrY)rprWrBrrrrXr) rprrrrrMrrrXr)rprWrBrWrrrXr)rprWrXrY) rr;rrrrrrMrXr) rrWrrrrrrrrrXr)rrrXr) rrrrrrrrrrrXrY)rrrXrY)rrrr;rrrXrY)rrrrrXrY)rrrrrXr)rrrrrrMrXr)rrWrrrrrrr.rrCrrrrXr)rrWrXr)rrWrrWrXr)r[r)rrWrrrrrXr)rrWrrWrrrXrY)rrWrrrXr)rrWrXrW)rrWrXr)rrWrXr)rrrrr rr!rr"r$r#r%rrrXr&)rrWrXr,) rrWrrr1rrrr rrXr2) rrWrrr8rrrrrrXrW)rrWrrr1rrrr rr!rrXru)rrWrXrY)rrWrrrXrW)rrWrrrXrL) rrWrrrRrrrrXrL) rrWrrMrrrrrXrY)rrWrrr r!r"rrRrrrrXr)rrWrrcr8rWr rrrr1rrrrarrbrrXrW)rirWrrrXrj)rirWrMrrr<rrrr1rrrr rrXrW)rirWrrrXrz)rirWrXrY)rirWrXr) rrrrr rr!rrrrXr&)rirWrrrrrXru)rirWrrrMrur<rrrr1rrrr rrXrW)rrrrrrr rr!rrrrXr&)rrWrrWrXr)rrWrrrrr1rrrr rrXrY)rrWrrWrrr1rrrr rrXrY) NNNNNNrr[TNT)rLrrrrrrCrrrrrrrMrrrrrrrrr`rrrrXr)rLrrrrrrCrrrrrrrMrrrrrrrrr`rrrrXr) NNNNNNrr[TNTr )rLrrrrrrCrrrrrrrMrrrrrrrrrrr`rrrrXr)rLrrrrrrCrrrrrrrMrrrrrrrrrrr`rrrrXr)FNNN)rgrrrrrrPrrrrrMrXr)NNFNNN)rrrrrrrPrrrrrMrXr)rrrrWrrrrrrrrrXr)r r rrrrr rrrr rrXr) rrr r rrr rrXr )rrWrrr rrXr )rrrrrrrXr )rLrrr)rrrr __slots__rrrrrpropertyrrrrrrrrr staticmethodrrr rrrrrr"rsetterrBrrr6r=r/rrr{r&r'rrrrrrrrrrrrrrrrrrrXrirMrrryrrrrrrr _deprecatedrrrr7rDrHrGrTrYr[r]rbrdrprrrtrxrrrrrrJrxor_argsrrrrrrrrrmrrr rrrrr+rr4r@rHr_rfrersryrrrrr(rrr3rrrrrrr rrrr/r6r7rMrGr_r?rlrortrvrxrrrrrrrrrrrrrrrrrrrrrrrrrrrr rrrrrPromptSortField updated_atr+rr6rErGrIrKrPrWr^r`rgrqrxr~rrrrrrrrmrvrrrrrrrrrrrrr-DEFAULT_INSTRUCTIONSr r r r rrrhrirerrs$88666Ip&&&&####!!!!2222))))4444===="&d("&(,<@!%.2#'7;EIFJGK). -159@D-19=.2'+15&*.2,0HL%*48?d(d(d(d(d(d(Z    ####*((((*...X. ;;;X; 333X3 222X2 ;;;X; 111X1 000X0 000X0 4$( AAAAA\AFXXXX    3333(((X(BBBXB"FFFF FFFFX ^;;;^;"""X"@@@@$$$X$ ^HHH^HX@    .-1"#<@=AMQn n n n n n b6:!!!!!!H $17 '0'0'0'0'0'0`&*3=3F3I: : : : : : D#%)3=3F3I_ _ _ _ _ _ H 77777r@@@@ :::: ,,,,,,\"""X" '+%)-2!%!%ssssssr"&!%%)#''+ $ppppppl"&!%%)#''+ $E E E E E E N))))++++,----   \ ,(DDDD>"&!%%)#''+ $______J  L3L3L3L3L3f"&!%%)#''+ $D7D7D7D7D7D7T"&!%%)#''+ $8?8?8?8?8?8?|  `3.3`3`3`3`3`3`3L!%!%%)#''+ $jjjjjjb VVVVVVx#)-2604#!%"&+/ $$(8<-27;!%!%%)#''+ $-bbbbbbP"&!%%)#''+ $ddddddT"&!%%)#''+ $555555p=A7272727272r<<<<[ ?PTBBBB  B6)-& '+ & & & & & & T)-& '+ & & & & & & P6666     :04# D04"'+#J&*#'(,26&*)-.2BUBUBUBUBUBUP#%)#'(,04/U/U/U/U/U/Ub( X566%)&*# ' ' ' ' ' 76' X%)&*#       :AE.)-&* R R R R R R l04"'+2604)-(,)-#-1IIIIIVX566/3QU66666766@&*)3)<)?2637LP#'G G G G G G X'+(, .X566'+(, - - - - - 76- b)-C='+ C=C=C=C=C=C=N)- '+      J04#'&*/3-1#333333jX566)-&* 66666766>)-&* :6:6:6:6:6:6~)-&* $# !!!!!!L)-&*-1! /</</</</</[ [ [ [ [ 76[ z    D(((TX566/3(,&*26/304+/(,+/"':>8<jjjj76jZLP! ! ! ! ! ! J)-&*379=*.#{##' $$){{{{{{B,0/3#'+/(,MQ8<U#U#U#U#U#U#t$(%)GK-2 W#W#W#W#W#W#r6666$FK060606060606j)-&*9= ######P)-&*'6'6'6'6'6'6\)-26 ??????B""""L"& %%%%%%N[ ))-2604  %BBBBB  BRKO04(,96:999999v[ ))-2604  %BBBBB  BN%) O 04:>&*(,$(04  ) -+/)-?C"$(,7;/3 $ $(,26'+5OOOOOOj04:>(,$()6)6)6)6)6)6V6666&0404RV# ))))))V6666(0406 ;?(,$(#'060606060606nHL?C)-?A?A?A?A?A?ALHL ddddddT $ B F& Z F)-(,# JJJJJ J$Z F J J J  JZ F)-(,JJJJJ J*Z FJJJ J(Z F J J J  J$16 9B9B9B9B9B9B|15'+# ------f@D04 2B2B2B2B2B2Bh6666<.2"'+# ((((((\&*&*-1MQ, , , , , , \HHHH,#%)-1MQ&6&6&6&6&6&6P6666(,06: 666666666666p6666&HHHH> #L04%)26-1 $9>9>9>9>9>9>v ? ? ? ?    IJ>>>>>*     D.    . ) ) ) ) I I I I J J J J$(&+1;1K1V17#*A*A*A*A*A*AX8&* $(,3<3<3<3<3<3>>>>>H $# 777777z&+)-!& /4O O O O O O j!%"*$(%) $(,15,0KKKKKKb"& <<<<<<.(,%) $(,$(      6"& <<<<<<(,%) $(,$(      .////////2222 2222$(&+#      ,$(&+#      &,>P>P>P>P@    AAAA:6SSSS.#####(FFFFFB "&6:FJ#'+/%))* -1#    X $ "&BFFJ#'+/%))* -1#++++X+0"& FJ#'+/%))* -1#38'b b b b b b KOFJ#'+/%))* MQ#381y y y y y ~9=59#8!8!8!8!8!t[ B#*.9=59#c c c c  c J 17;%)(,`````Y`D48%)-13535353535Y35j&*37-1! 99999Y9~!%"& @@@@@@J&*(,7; #G#G#G#G#G#GJrirr5 model_kwargsrr c ddlm}n#t$rtdwxYw|}|pi}|dd} |j|fd|i|S#t $r}t jd|d}~wwxYw)a Convert a prompt to OpenAI format. Requires the `langchain_openai` package to be installed. Args: messages (Any): The messages to convert. model_kwargs (Optional[Dict[str, Any]]): Model configuration arguments including `stop` and any other required arguments. Returns: The prompt in OpenAI format. Raises: ImportError: If the `langchain_openai` package is not installed. ls_utils.LangSmithError: If there is an error during the conversion process. r) ChatOpenAIzThe convert_prompt_to_openai_format function requires the langchain_openaipackage to run. Install with `pip install langchain_openai`stopNz#Error converting to OpenAI format: )langchain_openair> rr_get_request_payloadr@rrB)r5r< r> r$ r? rMs reconvert_prompt_to_openai_formatrB ?-s( ///////     K    Z\\F%2L   FD ) )DQ*v*8OO$O,OOO QQQ%&OA&O&OPPPQs  # A B%A<<Bcr ddlm}n#t$rtdwxYw|pi}|dd}|dd}|dd}|d |||d |} ||| S#t $r}t jd |d}~wwxYw) a[Convert a prompt to Anthropic format. Requires the `langchain_anthropic` package to be installed. Args: messages (Any): The messages to convert. model_kwargs (Optional[Dict[str, Any]]): Model configuration arguments including `model_name` and `stop`. Returns: The prompt in Anthropic format. r) ChatAnthropiczThe convert_prompt_to_anthropic_format function requires the langchain_anthropic package to run. Install with `pip install langchain_anthropic` model_namezclaude-3-haiku-20240307r? Nr)rE rr? )r? z&Error converting to Anthropic format: rh)langchain_anthropicrD rrrA r@rrB)r5r< rD rE r? rr% rMs re"convert_prompt_to_anthropic_formatrG f-s   5555555     =     %2L!!,0IJJJ   FD ) )Dy$//G wT=IIT--hT-BBB TTT%&Rq&R&RSSSTs  #9B B6B11B6c.eZdZdZd fd Zd d d ZxZS)_FailedAttachmentReaderzHBytesIO that raises an error when read, for failed attachment downloads.rr@cVt||_dSry)rr_error)rrrs rerz _FailedAttachmentReader.__init__-s$  rirrxrMrXrcFtjd|j|j)NzFailed to download attachment: )rrBrK )rrxs rerz_FailedAttachmentReader.read-s)% ;dk ; ;  { ri)rr@)r)rxrMrXr)rrrrrrrrs@rerI rI -s\RRrirI rrLrdict[str, AttachmentInfo]c li}||vr,||r#||D]\}}|ds|t||d}n|d} tj|d}|t j|j}nF#t$r9} t d|d| t| }Yd} ~ nd} ~ wwxYwtd i|d||dd ||d< |S) zOConvert attachments from the backend database format to the user facing format.z attachment.N presigned_urlT)rOzError downloading attachment r.rQ)rO readerrQrh)r startswithrgrr{rrrcontentr@rarbrI rN removeprefix) rLrrkattachments_dictr}r~rr2rP rMs rerr-s|$4#8/5577  JC>>-00 ")'53IJJ 1 4#<>>>))+++H$455 4 4 4IsIIaIIJJJ033 4AOAA%*?%;$!&;!7!7AA S--m<< = = s)AB-- C07/C++C0rMlist[io.BufferedReader]c|D]G} |#t$r#td|jYDwxYwdS)z2Close all opened files used in multipart requests.zCould not close file: %sN)rr@rarr)rMrs rerr-sf  JJLLLL    LL3TY ? ? ? D s*AArgcj|ddrd|dSd|dS)Nrmrz/platform/datasets/r}z/v1/platform/datasets/rnro)rkrgs rerlrl-sF~~c##E**>:Z::::= ====rir_cf|ddrd|Sd|S)z=Construct a platform API path based on the API URL structure.rmrz /platform/z /v1/platform/rX )rkr_s rerr-s@~~c##E**&"D"""%t%%%rirFc|dr|S|drd}|tdd}nA|drd}|tdd}ntd||dd}|dd}|std|d||s|S|dd kr1|d d kr |dd }n|d dd d gkr |dd }||z}|dd |DzS)NrrdrczBapi_url must start with 'http://' or 'https://'. Received api_url=rmzPMust specify non-empty api_url or pathname must be a full url. Received api_url=z , pathname=rapirrSv1c3K|]}||V dSryrh)r|ps rer z!_construct_url..-s'11q11111111ri)rQ rprrnr`lstripr)rkrFr api_partsrdrzs rergrg-s6""*%%  #j//++,   I & & #i..**+ S S S   s##))#..I%%++C00J   0 0 0$, 0 0    !} R=E ! !!#2#II rss^t} , ,!#2#I  "E #((11e11111 11rirrct|dr||St|dr||Std)z)Dump model depending on pydantic version.rrr zUnsupported model type)rrr r)rrs rerr-s`ul##2\:::   2zz|z4440111riobjc ddlm}ddlm}ddlm}m}n#t$rtdwxYw|}t||rt|j |rt|j d|rdt|j cxkrdkrnn|j }|j d}|j }|j } t||s%d |j vr|j d } |d d |ji| }t||rF||zj dj fd | D|_ |||}|S) z/Format the object so its Prompt Hub compatible.r)ChatPromptTemplater(r*r:r[r6rls_structured_output_formatr5c$i|] \}}|v || Srhrh)r|rrstructured_kwargss rerz%prep_obj_for_push..%.s1"""ABS9S9S19S9S9Srirh)r:rd r;r)langchain_core.runnablesr+r,rr rArGrprIrr5r) rb rd r)r+r, chain_to_pushrL bound_modelrr< output_formatrg s @rer>r>-s ======FFFFFFNNNNNNNNN     I    M3())B sy"4 5 5B sy|_ 5 5B SY $ $ $ $1 $ $ $ $ $il !") 6#344 Q-1CCC'./LMM%%PPvP-PPF f. / / B!'% 6q 9 @ """"!-!3!3!5!5"""K  -,V[AAM s2rMapping[str, str]rrrrrci|}t|||g}|rt|d}| ||t<n|st||}|||d<|||d<|||d<|||d<|S)Nz X-Service-KeyrrCookie)r_apply_optional_api_keyrv)rrrrrrrhas_non_api_keys rerr/.skG; v>??O9)'488$  E)'3CDD#. !*  #0  " Nric|r ||t<nH|td|td|Sry)rvrupper)rrs rero ro L.sP-$  It$$$ IOO%%t,,, Nri)rVrWrXrYr1 )rkrWrXrW)rsrtrXru)rrrrrXrr2 ) rrrrrrrrrXr)rrWrrrXrY) rrrrr r!r"rrXr)r6rR) rSrTrkrWrUrrVrWrXrW)rgrWrXr)rXr!)rsrtrXrY)rrWrXr)rkrWrrrrrXrY)rr)rrrXrYry)rrrXr)rrrXr)r~rrYrrXr)rrrrrXr)r~rrXr)r~rrrrXr)r~rrr)rrrXr)rrrrrXrY)r5rr< rrXr )rLr rrWrkrrXrM )rMrU rXrY)rkrWrgrrXrW)rkrWr_rWrXrW)rkrWrFrWrXrW)rrrXru)rb rrXr)rrl rrrrrrrrrrrrrXr)rrrrrXr(r __future__rrrrJrCrrr>rrrr}importlib.metadatarrrloggingrrjrr^rqtypingr[rr}collections.abcrrrrrr r inspectr pathlibr rr rrrrrrrrrrrurllibrr]httpxrpackaging.versionrrpydanticrrrrequests_toolbeltrrtyping_extensionsrrrurllib3.poolmanagerr urllib3.utilr!rRr"rr#r{r$rlangsmith._internalr%rr&r'r($langsmith._internal._backend_versionr)r=r*r+r#langsmith._internal._beta_decoratorr7 r,rr-&langsmith._internal._compressed_tracesr.langsmith._internal._constantsr/r0r1r2r3langsmith._internal._hubr4r5r6r7r8langsmith._internal._multipartr9r:r;langsmith._internal._operationsr<r=r>r?r@rArBrClangsmith._internal._serderDrlangsmith._internal._uuidrE'langsmith._internal._v2_migration_utilsrFrGlangsmith._openapi_clientrHrrIr&langsmith._openapi_client._base_clientrJrrKrlangsmith.prompt_cacherLrMlangsmith.schemasrNrO getLoggerrrar`r]r_rr\r frozensetrUrrfrjrrrrr& r' rrzoneinforropentelemetry.sdk.tracerrrrh rGlangsmith._openapi_client.resources.annotation_queues.annotation_queuesr5langsmith._openapi_client.resources.datasets.datasetsr5langsmith._openapi_client.resources.online_evaluatorsrr1langsmith._openapi_client.resources.public.publicr(langsmith._openapi_client.resources.runsr7langsmith._openapi_client.resources.sandboxes.sandboxesr+langsmith._openapi_client.resources.threadsr*langsmith._openapi_client.resources.tracesr#langsmith._openapi_client.types.runrV2Runrrrrrr-rr'rrrrrrrrrrrrrrervrrrrrr; rrrrrrQrfrkr\rWrrrrrrrrrrrarrr HTTPAdapterrrrrrrB rG rrI rrrlrrgrr>rro rhrirer s   #"""""      PPPPPPPPPPPPPP((((((((%%%%%%%%                        )(((((222222>=========''''''######++++++''''''444444GGGGGGGGGGGGGGGGJIIIII:99999CCCCCC                    A@@@@@++++++SSSSSSSSNNNNNNMMMMMMGFFFFFFF========  8 $ $!"$in&&34 '0y+1F1F'G'GGGGG$++++0000f "( $(BBBBBBJ   $(!!!!!!!(((((((((((((( .6666666.............. 0111111!!!!!!KJJJJJQPPPPPNNNNNN@@@@@@555555HHHHHHH   ?>>>>>                      8 $ $#'#$<==   , '0B0B0MMS#&355(((( 1%%%%4,-rTV!!!!!!H     FFFFX :      3  H Q333 3>!, :    (.2. $ $ $ $ 2JJJJCG QVQ  %%%%%-9%%%P"""""i"""     (|e|e|e|e|e|e|e|eBL.2$Q$Q$Q$Q$QR.2%T%T%T%T%TP     bj   CG<>>>>&&&&2222D/4222222....b:s6HH-,H-1H88I  I  J&& J21J2