jU dZddlmZddlZddlZddlmZmZmZm Z ddl m Z ddl m Z ddlmZddlmZmZmZdd lmZerdd lmZdd lmZmZdd lmZej:eZd Z e!hdZ"ddZ#ddZ$ddZ%ddZ&d dZ'GddZ(GddZ)GddZ*de ddddeddd d!dZ+y)"uLangSmith tracing for the OpenAI Realtime API (raw WebSocket). OpenAI Realtime has no native telemetry — it's a raw WebSocket event stream of ``input_audio_buffer.*`` / ``response.*`` events. :func:`wrap_realtime` wraps the ``AsyncRealtimeConnection`` so the trace is built directly from that stream while the caller's ``async for event in connection`` loop is left untouched: each received event is observed, spanned where warranted, and grouped into turns; the running transcript and a stereo conversation WAV land on the root span. Tool nesting is preserved without wrapping the caller's loop body: an event's span is opened when the event is received and kept as the active LangSmith context until the *next* event arrives — so any ``@traceable`` work the caller does while handling the event (tool execution) nests under it. Trace shape — one conversation = one trace:: realtime_session (root; transcript + WAV) ├── session.created / session.updated (setup) ├── turn (latency_ms, was_interrupted) │ ├── conversation.item.input_audio_transcription.completed (user message) │ └── response.done (chain wrapper) │ ├── model (llm — message + tokens) │ └── (nested under response.done) └── turn … ) annotationsN) TYPE_CHECKINGAnyCallableOptional)get_package_version)_create_usage_metadata)observe_safely)DEFAULT_MAX_AUDIO_SECONDS EventSession start_session)tracing_context)Sequence)ClientRunTree) WriteReplicai]> response.createdconversation.item.doneconversation.item.addedresponse.output_item.doneresponse.content_part.doneresponse.output_audio.doneresponse.output_item.addedresponse.content_part.addedinput_audio_buffer.committed%response.function_call_arguments.donec0|jdxsd|vS)uDirection of an event relative to the model. Inbound = something the user sent toward the model (their speech buffer, their transcription) → goes in span ``inputs``. Everything else is the model/server talking back → span ``outputs``. input_audio_bufferinput_audio_transcription) startswith) event_types |/var/www/html/Fitness-lenito-AI-main/venv/lib/python3.12/site-packages/langsmith/integrations/openai_realtime/_connection.py is_inboundr#Is% 23 5 &* 4c g}g}t|ddxsgD]}t|dddk(rH|jt|dddt|t|ddt|ddxsd d d [t|d dxsgD]2}t|d dxs t|dd}|s"|j|4ddj|j d}|r||d<|S)u Curated assistant message from a ``response.done`` payload. The readable, AIMessage-shaped view of what the model returned this response: the spoken text plus any tool calls it requested. Tool calls use the OpenAI ChatCompletion shape (``type: "function"`` with a nested ``function`` holding the name and the raw JSON ``arguments`` string) — that's the format the LangSmith UI recognizes and renders inline on the model span; a flatter ``{name, args, id}`` shape would instead spill into "Additional Fields". outputNtype function_callcall_idfunctionname argumentsz{})r+r,)idr'indexr*content transcripttext assistant roler/ tool_calls)getattrappendlenjoinstrip)responsetextsr6itempartr1outs r"response_assistant_outputrAVsE')J(D17R7 4 &/ 9   !$ 48& _ 'fd ;&-T;%E%M !   D)T28b8D4t4Sfd8SD T"9!8($/388E?;P;P;RSC&L Jr$c^t|tr t|}nIt|dd} t|rt|ni}|sdD]}t||d}||||<dD]1\}}|j |}t|ts(||vs-|||<3|S#t$ri}YcwxYw)uGNormalize a Realtime ``usage`` object into the dict the shared mapper reads. Realtime spells its per-modality breakdowns ``input_token_details`` / ``output_token_details`` (singular "token"), whereas :func:`_create_usage_metadata` looks for the Chat-Completions ``*_tokens_details`` spelling — so we alias them here to let it pick up audio and cached-token detail (which dominate voice cost). Falls back to the top-level token counts if the object can't be dumped, omitting the detail blocks rather than risk passing non-dict values to the mapper. model_dumpN input_tokens output_tokens total_tokens))input_token_detailsinput_tokens_details)output_token_detailsoutput_tokens_details) isinstancedictr7callable Exceptionget)usagebaserCkeyval realtime_key mapper_keyblocks r"_usage_to_dictrX}s%E{UL$7  )1*)=4 %2DHeS$/? #DII% j& eT "z'=$D  % K D sB B,+B,cr|yt|tfddDsyttS)aMap a Realtime ``usage`` (object or dict) onto LangSmith ``usage_metadata``. Delegates to the shared OpenAI usage mapper so the Realtime integration and the Chat/Agents wrappers agree on the canonical shape (and so audio/cached token detail is captured, not dropped). Returns ``None`` when there is no usage or it carries no token counts (e.g. a cancelled turn). Nc3DK|]}j|duywN)rP).0rS usage_dicts r" z,usage_metadata_from_usage..s'DC s4'Ds rD)rXanyrMr )rQr]s @r"usage_metadata_from_usager`sB }&J D  &z2 33r$c.tt|ddS)zJMap a ``response.done`` payload's token ``usage`` onto ``usage_metadata``.rQN)r`r7)r<s r"response_usage_metadatarbs $WXw%E FFr$c`eZdZdZdd d dZd dZddZddZddZdd Z dd Z dd Z y)_RealtimeTraceru&Turns the Realtime event stream into a LangSmith trace, one event at a time. Decides — privately — whether to open a span, open/close a turn, append to the transcript rollup, set the trace title, time first-audio latency, flag a barge-in, and record the ``llm`` span. All trace writes go through the injected :class:`EventSession`. An event's span is held open (as the active LangSmith parent context) from the moment the event is received until the next event arrives, so the caller's tool calls nest under it. Nis_agent_speakingcJ||_||_d|_d|_d|_yr[)_session_is_agent_speaking_await_audio_since_open_model)selfsessionrfs r"__init__z_RealtimeTracer.__init__s/   #415.2 "& r$c|j|jj}t|dd}|y|dvr*tt|dddd}|rt ||_|dk(r|j |y|jds|tvry|dk(r,|jjd t|d d xsd y|j|||jj||fd |i|j|}|j}|d k(rdt|dd|_n&|dk(r!t|dd}||j||t!|}|j||f|_y)a"Observe one received event; span it where warranted. Closes the previous event's span first (the caller has finished handling it), then applies turn/transcript/latency side-state and, for meaningful events, opens a new span kept active until the next call. r'N)zsession.createdzsession.updatedrnmodelzresponse.output_audio.deltaz.deltaz%response.output_audio_transcript.doner2r0r+2conversation.item.input_audio_transcription.failedztranscription_failed: error response.doner<)parent) _close_openrhnowr7strrl_record_first_audioendswith NOISE_EVENTS add_message _before_span event_span _span_kwargs __enter__rt_record_response_llmrrk) rmevent received_atetyperqcm_spanrunr<cm_ctxs r"observez_RealtimeTracer.observes mm'') vt, =  : :GE9d;WdKE!%j  1 1  $ $[ 1  >>( #u '<  ; ; MM % %WUL"=C   %-*$--** ; %* .2.?.?.F ! H H01N0OPCI o %uj$7H#))#x8 ,v& r$c$|jy)z5Close any still-open event span (called at teardown).N)rwrms r" finalize_openz_RealtimeTracer.finalize_opens r$c|jy|j\}}d|_|jddd|jdddyr[)rk__exit__)rmrrs r"rwz_RealtimeTracer._close_open sE ::  ** dD)tT*r$c|j}|dk(rZ|j,|jr|jjd|jj d|_y|dk(r||_y|dk(rat |ddxsdj}|r8|jjd ||jj|yd|_y|d k(rd|_yy) zDTurn/transcript/title side-state applied as a span is about to open.z!input_audio_buffer.speech_startedNT)was_interruptedz!input_audio_buffer.speech_stopped5conversation.item.input_audio_transcription.completedr0rruserrs) r'rirhadd_turn_metadata start_turnrjr7r;r} set_title)rmrrrr1s r"r~z_RealtimeTracer._before_spans  7 7&&2t7N7N7P ///E MM $ $ &&*D # 9 9&1D # M ME<4:AACD ))&$7 ''-*.' J J&*D #Kr$c|j}dt|i}|dk(r+t|ddxsdj}|rd|d|d<|S|dk(ri|d <|S) z9Curated, conversation-shaped I/O for the readable events.inboundrr0rrrr4inputsruoutputs)r'r#r7r;)rmrrkwargsr1s r"rz_RealtimeTracer._span_kwargs+ss "+Z->!? K KE<4:AACD,2t#Dx  o %!#F9  r$c|j=|jjt||jz dzd|_yy)Ni)latency_to_first_audio_ms)rjrhrround)rmrs r"rzz#_RealtimeTracer._record_first_audio9sO  " " . MM + +*/ 4#:#::dB+ ,  '+D # /r$ct|ddd|jd}|jj|t |t ||y)Nstatusopenai)r ls_provider ls_model_name)rusage_metadatametadata)r7rlrh record_llmrArb)rmrr<rs r"rz$_RealtimeTracer._record_response_llmBsNh$7#![[$    -h728< ! r$)rnr rfOptional[Callable[[], bool]]returnNone)rrrr)rr)rrrfloatrr)rrrdict[str, Any])rrrr)rrr<rrr) __name__ __module__ __qualname____doc__rorrrwr~rrzrr$r"rdrdsQ  ;? ''8 '  '&/'b++. +  r$rdcXeZdZdZ d dZd dZd dZd dZd dZddZ ddZ y )_TracedRealtimeConnectionaTransparent proxy over an ``AsyncRealtimeConnection`` that traces events. Delegates every attribute to the wrapped connection, so the caller uses it exactly like the original (``connection.session.update(...)``, ``connection.response.create()``, ``async for event in connection``). Each received event is passed to the tracer. Also exposes :meth:`record_user_audio` / :meth:`record_agent_audio` so the app can (optionally) feed PCM for the stereo conversation WAV. c<||_||_||_d|_yr[) _connection_tracerrh_aiter)rm connectiontracerrns r"roz"_TracedRealtimeConnection.__init__[s! &   r$cBttj|d|S)Nr)r7object__getattribute__)rmr+s r" __getattr__z%_TracedRealtimeConnection.__getattr__esv..t]CTJJr$cD|jj|_|Sr[)r __aiter__rrs r"rz#_TracedRealtimeConnection.__aiter__hs&&002  r$cK|j}|!|jj}||_|jd{}t |j j ||S7&wr[)rrr __anext__r rr)rmaiterrs r"rz#_TracedRealtimeConnection.__anext__ls\  =$$..0EDKoo''t||++U3 (sAA.A,'A.cK|jjd{}t|jj||S7&wr[)rrecvr rr)rmrs r"rz_TracedRealtimeConnection.recvus;&&++--t||++U3 .sA A'A cl|jj|jj|y)zCRecord a chunk of user (mic) PCM16 for the stereo conversation WAV.N)rh record_userrxrmpcms r"record_user_audioz+_TracedRealtimeConnection.record_user_audiozs# !!$--"3"3"5s;r$cl|jj|jj|y)zGRecord a chunk of agent (played) PCM16 for the stereo conversation WAV.N)rh record_agentrxrs r"record_agent_audioz,_TracedRealtimeConnection.record_agent_audio~s# ""4==#4#4#6r s94# 99D=< 2()0   8 $  " $ND4&G V V r0=0=fNNh $*"& $)-6:)B#15555 5  5  5'545'5 5/55r$