tjFdZddlmZddlmZddlmZmZmZm Z ddl m Z m Z m Z mZmZddlmZddlmZddlmZmZmZmZmZmZmZmZmZmZdd lm Z m!Z!m"Z"m#Z#m$Z$erdd l%m&Z&dd l'm(Z(dd l)m*Z*d Z+ GddeZ,Gdde e!e#e"e#fZ-gdZ.dS)z1PII detection and handling middleware for agents.) annotations)partial) TYPE_CHECKINGAnyClassVarLiteral) AIMessage AnyMessage BaseMessage HumanMessage ToolMessage)StreamTransformer)override) PIIDetectionErrorPIIMatch RedactionRuleResolvedRedactionRuleapply_strategydetect_credit_card detect_email detect_ipdetect_mac_address detect_url)AgentMiddleware AgentStateContextT ResponseT hook_config)Callable)Runtime) ProtocolEventceZdZUdZdZded<dZded< d9ed d:fdZd;dZ dd#Zd?d&Zd@d(ZdAd+ZdBd.ZdCd0ZdAd1ZdDd3ZdEd4ZdFd5ZdGd8ZxZS)H_PIIStreamTransformeruMutates `content-block-delta` text on `messages` events in flight. Runs before built-in stream transformers so the redacted text is what every downstream consumer sees — both the main protocol event log and the `run.messages` projection that `MessagesTransformer` snapshots into. Holds a sliding buffer of the most recent text per (run_id, content block index) so PII patterns that straddle delta boundaries are caught. Anything older than `lookback` characters is redacted with the resolved rule's strategy and emitted as the new delta text; the trailing tail stays in the buffer until a later delta extends it past the cap or the block's finish event flushes the snapshot. TzClassVar[bool]before_builtins)messagestoolsvalueszClassVar[tuple[str, ...]]required_stream_modes)lookbackscopetuple[str, ...]rulerr+intreturnNonect|||_||_i|_i|_dSN)super__init___rule _lookback_buffers _tool_buffers)selfr,r.r+ __class__s ]/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/langchain/agents/middleware/pii.pyr5z_PIIStreamTransformer.__init__DsB  !46 .0dict[str, Any]ciSr3r*r:s r<initz_PIIStreamTransformer.initUs  r=eventr!boolc|d}|dkr||S|dkr||S|dkr||SdS)Nmethodr&r'r(T)_process_messages_event_process_tools_event_process_values_event)r:rBrEs r<processz_PIIStreamTransformer.processZsmx Z  //66 6 W  ,,U33 3 X  --e44 4tr=c|dd}|dS|||dd<dS)uRedact the state snapshot on the `values` channel. State snapshots emitted between nodes carry the full state dict, which typically includes the messages list. Walking the snapshot with `_redact_value` returns a fresh structure where every message has a redacted copy of its content — the original objects in graph state remain intact for the state-level enforcer (`apply_to_tool_results` via `before_model`) to act on independently when the agent loops back. paramsdataNT)get _redact_value)r:rBrLs r<rHz+_PIIStreamTransformer._process_values_eventdsFX""6** <4"&"4"4T":":htr=cZ|d}|d}t|trt|dkrdS|\}}t|tr"||}||ur||f|d<dSt|t sdS|d}|r$t|dpdnd}|dkr|||n6|d kr| ||n|d vr| |dS) NrKrLTrBrun_idzcontent-block-deltazcontent-block-finish>message-finisherror) rM isinstancetuplelenr _redact_base_messagedictstr _mutate_delta_finalize_block _drop_run) r:rBrKrLpayloadmetadataredactedkindrQs r<rFz-_PIIStreamTransformer._process_messages_eventusMxzz&!!$&& #d))q..4  g{ + + 0099Hw&&"*H!5v4'4(( 4{{7##6>FX\\(++1r222B ( ( (   w / / / / + + +  & 1 1 1 1 0 0 0 NN6 " " "tr=cd|dd}t|tsdS|d}|d}|dkr$d|vr||d|d<n)|dkr.||t|t r|nd n|d krSd |vr||d |d <t|t r|j|dn|d kr|d }t|t r<|r:|j |}|rt|||jj |d <t|t r|j|ddS)NrKrLTrB tool_call_idz tool-startedinputztool-output-deltarRz tool-finishedoutputz tool-errormessage) rMrUrYrN_mutate_tool_output_deltarZr9popr6detectorrstrategy)r:rBrLrarcmsgmatchess r<rGz*_PIIStreamTransformer._process_tools_eventsX""6**$%% 4xx  xx// > ! ! $ $ 2 24= A AW ( ( (  * *js&C&CKll    _ $ $4!%!3!3DN!C!CX,,, ;"&&|T::: \ ! !((9%%C#s## X X*--c22X&4S'4:CV&W&WDO,,, ;"&&|T:::tr=rLrcrZc|d}t|tr|j|d}||z}|j|}|rt |||jj}tdt||j z }||d|j|<|d||d<dSt|ttfr| ||d<dSdS)uRedact a `tool-output-delta` payload. String deltas go through the same lookback machinery as text-deltas, keyed by `tool_call_id` in the disjoint `_tool_buffers` dict so `_drop_run` on the messages channel can't sweep active tool-output state. Structured deltas (dict/list) walk recursively without buffering — they don't have a position-stable shape across deltas to buffer against. deltarRrN)rMrUrZr9r6rirrjmaxrWr7rYlistrN)r:rLrcrnheldcombinedrlemit_ends r<rgz/_PIIStreamTransformer._mutate_tool_output_deltas!! eS ! ! 6%)),;;De|Hj))(33G R *(GTZ=PQQ1c(mmdn<==H/7 /BD | ,$YhY/DMMM d| , , 6 ..u55DMMM 6 6r=callslist[Any] | Nonetuple[list[Any], bool]c>|s|pgdfSg}d}|D]}t|tr_d|vr[|dS||d}||dkr,t|}||d<||d}v||||fS)uWalk a list of tool-call (or invalid-tool-call) dicts. Returns `(new_list, changed)`. Each element's `args` is run through `_redact_value` regardless of its type — `tool_call.args` is a dict, `invalid_tool_call.args` is a raw JSON string, and `_redact_value` handles both shapes uniformly. If nothing changed, returns the input list and `changed=False`. FargsNT)rUrYrNappend)r:rt new_callschangedtcr`new_tcs r<_redact_tool_call_listz,_PIIStreamTransformer._redact_tool_call_lists &;B% %!  ! !B"d## " F9O--bj99r&z))!"XXF%-F6N$$V,,,"G   R '!!r=valuerct|tr=|s|Sj|}|s|St ||jjSt|t r|St|tr fd| DSt|trfd|DSt|trtfd|DS|S)uzRecursively redact PII in string leaves of a nested structure. Returns a new value where every `str` leaf that contains PII has been replaced (or emptied under `block`). Non-string leaves and the structure itself are preserved. `BaseMessage` payloads (typically `ToolMessage` from `tool-finished.output`, or any message reached via the `values` channel) return a fresh copy with `.content` redacted plus `AIMessage.tool_calls[*].args` / `invalid_tool_calls[*].args` walked. The original object stays intact for state-level enforcers (`after_model`, `before_model` with `apply_to_tool_results`) to act on independently. Scope mirrors the pre-streaming state-level surfaces: `.content` (string or list-of-content-blocks) and `tool_calls` args. Other message attributes (`additional_kwargs`, `response_metadata`, `ToolMessage.artifact`) are intentionally not walked here — they aren't scrubbed in graph state by the existing hooks, so scrubbing them on the wire would create a wire/state divergence. cBi|]\}}||Sr*rN).0kvr:s r< z7_PIIStreamTransformer._redact_value..s-GGGAAt))!,,GGGr=c:g|]}|Sr*rrrr:s r< z7_PIIStreamTransformer._redact_value..s'999aD&&q))999r=c3BK|]}|VdSr3rrs r< z6_PIIStreamTransformer._redact_value..s1>>1++A..>>>>>>r=) rUrZr6rirrjr rXrYitemsrprV)r:rrls` r<rNz#_PIIStreamTransformer._redact_values!. eS ! ! G  j))%00G  "%$*2EFF F e[ ) ) 4,,U33 3 eT " " HGGGGGGG G eT " " :99995999 9 eU # # ?>>>>>>>>> > r=r ci}|j}t|tr=|r;|j|}|rt |||jj|d<n7t|tr"|r ||}||kr||d<t|trH| |j \}}|r||d<| |j \}} | r||d<|s|S| |S)zCReturn a fresh copy of `value` with PII-carrying surfaces redacted.content tool_callsinvalid_tool_calls)update)rrUrZr6rirrjrprNr r~rr model_copy) r:rrrrlredacted_content new_tc_list tc_changed new_inv_list inv_changeds r<rXz*_PIIStreamTransformer._redact_base_message s8!#- gs # # 5 5j))'22G Z$27GTZEX$Y$Yy!  & & 57 5 $11':: 7**$4y! eY ' ' <&*&A&A%BR&S&S #K 3'2|$(,(C(CED\(](] %L+ tool_call_chunkserver_tool_call_chunk)rMrUrY_mutate_string_field_delta_mutate_tool_call_chunk_delta)r:r^rQrn delta_typers r<r[z#_PIIStreamTransformer._mutate_delta@s G$$%&&  FYYv&&  % %  + +E7FF K K K F * * *  + +E7FK P P P F  & &YYx((F&$'' ;FJJv,>,>C--226::::: ' & ; ;--r=rnfieldc||}t|tr|sdS|d}t|tsdS||f}|j|d}||z} |j| } | rt| | |jj} tdt| |j z } | | d|j|<| d| ||<dS)a[Apply the lookback-buffer redaction to a string field on a delta. Shared by `text-delta` (`field="text"`) and `reasoning-delta` (`field="reasoning"`). Buffer is keyed by `(run_id, block_index)`; block indices are unique within a message so different block types share the same key space without collision. NindexrRr) rMrUrZr/r8r6rirrjrorWr7) r:rnr^rQrrrkeyrqrrrlrss r<rz0_PIIStreamTransformer._mutate_string_field_delta^syy$$$ D  F G$$%%%  Fuo}  b))$;*%%h//  N%h9LMMHq#h--$.899%hii0 c  *e r=rc4|d}t|tr|sdS|j|}|rt |||jj}tdt||j z }|d||d<dS)uRedact cumulative tool-call args with lookback withholding. Each `tool_call_chunk` `block-delta` event carries the full accumulated args string (verified against `_compat_bridge.py` — `delta_source = current` for these block types — and against the consumer-side `_merge_block_delta_into_store`, which replaces wholesale rather than appends). Detection runs on the full cumulative args so any complete PII anywhere in the string is redacted before emission. Lookback withholding then trims the trailing the lookback window characters from what reaches the consumer — those characters might be the start of a partial PII match that completes in a future cumulative delta. The trimmed tail surfaces at `content-block- finish` where `_finalize_block` redacts the parsed args dict. For args that fit within the lookback window (the typical case), this withholds the entire args string during streaming — the redacted args dict appears only at finalize. For args that exceed the lookback window, the safe prefix streams incrementally as the cumulative state grows. PII that appears more than the lookback window characters from the cumulative tail in a delta where it hasn't yet completed can still surface in the emit prefix — same residual exposure as PII longer than the lookback window on the text path. The `content-block-finish` snapshot redaction is the backstop. rxNr) rMrUrZr6rirrjrorWr7)r:rrxrlrss r<rz3_PIIStreamTransformer._mutate_tool_call_chunk_deltas8zz&!!$$$ D  F*%%d++  F"$1DEEDq#d))dn455ixivr=c|d}t|tsdS||f}|d}t|tr}|d}|dkr||dnK|dkr||dn.|dvr*d|vr&|d||d|d<|j|ddS)Nrrrrr> tool_callserver_tool_callinvalid_tool_callrx)rMrUr/rY_finalize_string_fieldrNr8rh)r:r^rQrrrctypes r<r\z%_PIIStreamTransformer._finalize_blocks  G$$%%%  Fuo++i(( gt $ $ FKK''E++GV<<<<+%%++G[AAAAOOOg%%FO/ #'"4"4WV_"E"E #t$$$$$r=rc||}t|tr|sdS|j|}|sdSt |||jj||<dS)zRe-redact a string content-block field on `content-block-finish`. Used for `text` and `reasoning` content blocks. Under `strategy="block"` `apply_strategy` raises `PIIDetectionError`, failing the run immediately. N)rMrUrZr6rirrj)r:rrrrls r<rz,_PIIStreamTransformer._finalize_string_fieldss{{5!!$$$ D  F*%%d++  F'gtz7JKKr=cHfd|jD}|D] }|j|= dS)Nc,g|]}|dk|S)rr*)rrrQs r<rz3_PIIStreamTransformer._drop_run..s'BBBQ61A1A1A1A1Ar=)r8)r:rQstalers ` r<r]z_PIIStreamTransformer._drop_runsG CBBB BBB # #C c"" # #r=cj|j|jdSr3r8clearr9r@s r<finalizez_PIIStreamTransformer.finalize1    """""r=err BaseExceptioncj|j|jdSr3r)r:rs r<failz_PIIStreamTransformer.failrr=)r*)r,r-r.rr+r/r0r1)r0r>)rBr!r0rC)rLr>rcrZr0r1)rtrur0rv)rrr0r)rr r0r )r^r>rQrZr0r1) rnr>r^r>rQrZrrZr0r1)rr>r0r1)rr>rrZr0r1)rQrZr0r1)r0r1)rrr0r1)__name__ __module__ __qualname____doc__r%__annotations__r)_DEFAULT_STREAM_LOOKBACKr5rArIrHrFrGrgr~rNrXr[rrr\rr]rr __classcell__r;s@r<r$r$2s  '+O****7VVVVV"$0 1 00000000" "!!!!F####J6666>""""2))))V////@;;;;<'+'+'+'+R()()()()T%%%%4 L L L L################r=r$ceZdZdZddddddd$fdZed%dZd&dZedge d'd Z edgd'd!Z e d'd"Z d'd#Z xZS)( PIIMiddlewarea Detect and handle Personally Identifiable Information (PII) in conversations. This middleware detects common PII types and applies configurable strategies to handle them. It can detect emails, credit cards, IP addresses, MAC addresses, and URLs in both user input and agent output. Built-in PII types: - `email`: Email addresses - `credit_card`: Credit card numbers (validated with Luhn algorithm) - `ip`: IP addresses (validated with stdlib) - `mac_address`: MAC addresses - `url`: URLs (both `http`/`https` and bare URLs) Strategies: - `block`: Raise an exception when PII is detected - `redact`: Replace PII with `[REDACTED_TYPE]` placeholders - `mask`: Partially mask PII (e.g., `****-****-****-1234` for credit card) - `hash`: Replace PII with deterministic hash (e.g., ``) Strategy Selection Guide: | Strategy | Preserves Identity? | Best For | | -------- | ------------------- | --------------------------------------- | | `block` | N/A | Avoid PII completely | | `redact` | No | General compliance, log sanitization | | `mask` | No | Human readability, customer service UIs | | `hash` | Yes (pseudonymous) | Analytics, debugging | Example: ```python from langchain.agents.middleware import PIIMiddleware from langchain.agents import create_agent # Redact all emails in user input agent = create_agent( "openai:gpt-5.5", middleware=[ PIIMiddleware("email", strategy="redact"), ], ) # Use different strategies for different PII types agent = create_agent( "openai:gpt-5.5", middleware=[ PIIMiddleware("credit_card", strategy="mask"), PIIMiddleware("url", strategy="redact"), PIIMiddleware("ip", strategy="hash"), ], ) # Custom PII type with regex agent = create_agent( "openai:gpt-5.5", middleware=[ PIIMiddleware("api_key", detector=r"sk-[a-zA-Z0-9]{32}", strategy="block"), ], ) ``` redactNTF)rjriapply_to_inputapply_to_outputapply_to_tool_resultspii_typeALiteral['email', 'credit_card', 'ip', 'mac_address', 'url'] | strrj*Literal['block', 'redact', 'mask', 'hash']ri,Callable[[str], list[PIIMatch]] | str | NonerrCrrr0r1ct||_||_||_t ||||_|jj|_|jj |_ |jj |_ |js|jr#tt|jf|_ dSdS)u Initialize the PII detection middleware. Args: pii_type: Type of PII to detect. Can be a built-in type (`email`, `credit_card`, `ip`, `mac_address`, `url`) or a custom type name. strategy: How to handle detected PII. Options: * `block`: Raise `PIIDetectionError` when PII is detected * `redact`: Replace with `[REDACTED_TYPE]` placeholders * `mask`: Partially mask PII (show last few characters) * `hash`: Replace with deterministic hash (format: ``) detector: Custom detector function or regex pattern. * If `Callable`: Function that takes content string and returns list of `PIIMatch` objects * If `str`: Regex pattern to match PII * If `None`: Uses built-in detector for the `pii_type` apply_to_input: Whether to check user messages before model call. apply_to_output: Whether to check AI messages after model call. When `True`, a stream transformer is also installed so that every wire surface of an agent run is redacted in flight: * Streamed AI text deltas (`content-block-delta` of type `text-delta`) * Streamed tool-call arguments (`content-block-delta` with `tool_call_chunk` / `server_tool_call_chunk` fields, plus the finalized `tool_call` content block on `content-block-finish`) * Tool execution events on the `tools` channel (`tool-started.input`, `tool-output-delta`, `tool-finished.output`, `tool-error.message`) * State snapshots on the `values` channel — message lists are walked and each message's `.content` is redacted on a fresh copy (state itself stays intact for `before_model` / `after_model` to act on independently) State-level redaction via `after_model` (and `before_model` with `apply_to_tool_results`) remains the canonical enforcer; the streaming transformer ensures consumers reading `astream_events(version="v3")` or `run.messages` / `run.tool_calls` / `run.values` never see PII on the wire. apply_to_tool_results: Whether to check tool result messages after tool execution. Raises: ValueError: If `pii_type` is not built-in and no detector is provided. )rrjri)r.N)r4r5rrrrresolve_resolved_rulerrjrirr$ transformers)r:rrjrirrrr;s r<r5zPIIMiddleware.__init__,sF ,.%:"5B6 6 6  '))  +4 +4 +4    4#= ),!D     r=rZc0|jjd|jdS)zName of the middleware.[])r;rrr@s r<namezPIIMiddleware.names".)< +(+*/333O 3BL/#'L  % ,K3x==1,b"55  hqk955"#KE&{QH >>,,A"1+C!#{33,#&'/%$"%h&6"7"7/3/D/DW/M/M, W&%$+6$/'{!))1)> ++++: Q'+  . - -tr=c2K|||S)aAsync check user messages and tool results for PII before model invocation. Args: state: The current agent state. runtime: The langgraph runtime. Returns: Updated state with PII handled according to strategy, or `None` if no PII detected. Raises: PIIDetectionError: If PII is detected and strategy is `'block'`. )rr:rrs r< abefore_modelzPIIMiddleware.abefore_models&  000r=c|jsdS|d}|sdSd}d}tt|dz ddD]%}||}t|tr|}|}n&| |r|jsdSt |j}||\} } | sdSt | |j|j |j } t|} | | |<d| iS)aCheck AI messages for PII after model invocation. Args: state: The current agent state. runtime: The langgraph runtime. Returns: Updated state with PII handled according to strategy, or None if no PII detected. Raises: PIIDetectionError: If PII is detected and strategy is `'block'`. Nr&rr)rrrr) rrrWrUr rrZrrrrrp) r:rrr& last_ai_msgrrrkrrrlrrs r< after_modelzPIIMiddleware.after_models)&# 4$ 4  s8}}q("b11  A1+C#y)) !     k 9L 4k)**#44W== W 4$~!"-    H~~ $3 [!L))r=c2K|||S)aAsync check AI messages for PII after model invocation. Args: state: The current agent state. runtime: The langgraph runtime. Returns: Updated state with PII handled according to strategy, or None if no PII detected. Raises: PIIDetectionError: If PII is detected and strategy is `'block'`. )rrs r< aafter_modelzPIIMiddleware.aafter_modelPs$w///r=)rrrjrrirrrCrrCrrCr0r1)r0rZ)rrZr0r)rrrrr0r)rrrrr5propertyrrrrrrrrrrs@r<rrsF==J@HAE# %&+ddddddddL===X=""""[eW%%% \\\X&%\|[eW%%%111&%1(9*9*9*X9*v00000000r=r)rrrrrrrrN)/r __future__r functoolsrtypingrrrrlangchain_core.messagesr r r r r langgraph.streamrtyping_extensionsr&langchain.agents.middleware._redactionrrrrrrrrrr!langchain.agents.middleware.typesrrrrrcollections.abcrlanggraph.runtimer langgraph.stream._typesr!rr$r__all__r*r=r<rs977""""""888888888888aaaaaaaaaaaaaa......&&&&&&                        6(((((())))))555555w#w#w#w#w#-w#w#w#t v0v0v0v0v0OJy$98Y$NOv0v0v0r   r=