tj P U d Z ddlZddlZddlmZmZmZ ddlmZ ddl m
Z
m Z m Z m
Z
ddlmZmZmZmZmZ ddlmZ ddlmZmZmZ dd lmZ dd
l m Z dd l m Z dd l m!Z!m"Z"m#Z#m$Z$ dd
l%m&Z&m'Z' eee ge(f Z)dZ* dZ+dZ,dZ-d e.ddh iZ/e0e1e.e1 f e2d< de1de1dz de3fdZ4e5e d e6f Z7 e5e d e(f Z8 e5e d e(f Z9 e7e8z e9z Z: G d d e d Z;d!e&de)fd"Z< G d# d$e!e"e$ e#e$f Z=dS )%zSummarization middleware. N)CallableIterableMapping)partial)AnyLiteral TypedDictcast) AIMessage
AnyMessageMessageLikeRepresentation
RemoveMessage ToolMessage HumanMessage)count_tokens_approximatelyget_buffer_string
trim_messages)REMOVE_ALL_MESSAGES)Runtime)override)AgentMiddleware
AgentStateContextT ResponseT)
BaseChatModelinit_chat_modela
Context Extraction Assistant
Your sole objective in this task is to extract the highest quality/most relevant context from the conversation history below.
You're nearing the total number of input tokens you can accept, so you must extract the highest quality/most relevant pieces of information from your conversation history.
This context will then overwrite the conversation history presented below. Because of this, ensure the context you extract is only the most important information to continue working toward your overall goal.
The conversation history below will be replaced with the context you extract in this step.
You want to ensure that you don't repeat any actions you've already completed, so the context you extract from the conversation history should be focused on the most important information to your overall goal.
You should structure your summary using the following sections. Each section acts as a checklist - you must populate it with relevant information or explicitly state "None" if there is nothing to report for that section:
## SESSION INTENT
What is the user's primary goal or request? What overall task are you trying to accomplish? This should be concise but complete enough to understand the purpose of the entire session.
## SUMMARY
Extract and record all of the most important context from the conversation history. Include important choices, conclusions, or strategies determined during this conversation. Include the reasoning behind key decisions. Document any rejected options and why they were not pursued.
## ARTIFACTS
What artifacts, files, or resources were created, modified, or accessed during this conversation? For file modifications, list specific file paths and briefly describe the changes made to each. This section prevents silent loss of artifact information.
## NEXT STEPS
What specific tasks remain to be completed to achieve the session intent? What should you do next?
The user will message you with the full message history from which you'll extract context to create a replacement. Carefully read through it all and think deeply about what information is most important to your overall goal and should be saved:
With all of this in mind, please carefully read over the entire conversation history, and extract the most important and relevant context to replace it so that you can free up space in the conversation history.
Respond ONLY with the extracted context. Do not include any additional information, or text before or after the extracted context.
Messages to summarize:
{messages}
i amazon_bedrockbedrockbedrock_converse_LS_PROVIDER_ALIASESmessage_providermodel_ls_providerreturnc ^ |dS | |k rdS t | }|d uo| |v S )NFT)r# get)r$ r% aliasess g/var/www/html/CA-Chatbot/venv/lib/python3.11/site-packages/langchain/agents/middleware/summarization.py_provider_matchesr+ g sH u,,,t"&&'899G $ >#3w#>> fractiontokensmessagesc 6 e Zd ZU dZeed< eed< eed< dS )
TriggerClausea Dictionary-based trigger specification for AND conditions.
All specified thresholds in a single `TriggerClause` must be met for the clause to
trigger summarization (AND semantics). When multiple clauses are provided in a list,
summarization triggers if any clause is met (OR semantics).
Example:
```python
# AND: Trigger when tokens >= 4000 AND messages >= 10
trigger_clause: TriggerClause = {"tokens": 4000, "messages": 10}
# Use in a list for OR semantics:
trigger_list: list[TriggerClause] = [
{"tokens": 5000, "messages": 3},
{"tokens": 3000, "messages": 6},
]
```
r. r/ r- N)__name__
__module__ __qualname____doc__int__annotations__float r, r* r1 r1 sG &
KKK MMMCOOO r, r1 F)totalmodelc | j d rt t dd S t t d S )zATune parameters of approximate token counter based on model type.zanthropic-chatTgffffff
@)use_usage_metadata_scalingchars_per_tokenr= ) _llm_type
startswithr r )r; s r* _get_approximate_token_counterrB sR !!"233
&4Y\
-$ O O OOr, c & e Zd ZdZddefeeeddee z de
e z e e
e z z dz de
de
d ed
edz d ed df fd
Zedee dee d eeef dz fd Zedee dee d eeef dz fd Zede
e z e e
e z z dz d e
e z e e
e z z dz fd Zde
e z e e
e z z dz d e e
fdZde
e z e e
e z z dz d e e fdZde e de d e fdZ de e ded e fdZ de e d efdZ!de e d edz fdZ"d edz fdZ#ed e
d ed e
fd Z$ed ed e e% fd Z&ede e d dfd! Z'ed"e e d#ed e(e e e e f fd$ Z)de e d%ed efd&Z*ede e d#ed efd' Z+d(e e d efd)Z,d(e e d efd*Z-de e d e e fd+Z. xZ/S ),SummarizationMiddlewarea; Summarizes conversation history when token limits are approached.
This middleware monitors message token counts and automatically summarizes older
messages when a threshold is reached, preserving recent messages and maintaining
context continuity by ensuring AI/Tool message pairs remain together.
Nr/ )triggerkeep
token_countersummary_prompttrim_tokens_to_summarizer; rE rF rG rH rI deprecated_kwargsr& c d|v r,|d }t j dt d ||d|f}d|v r5|d }t j dt d |d t fk rd |f}t t
|t rt | }|| _ |
| | _ | | j | _
| | j | _ | |d
| _ |t$ u r5t' | j | _ t+ | j d | _ n|| _ || _ || _ || _ t3 d
| j
D } | j d dk rd} | r%| d}
t7 |
dS dS )a Initialize summarization middleware.
Args:
model: The language model to use for generating summaries.
trigger: One or more thresholds that trigger summarization.
Provide a single
[`ContextSize`][langchain.agents.middleware.summarization.ContextSize]
tuple, or a single
[`TriggerClause`][langchain.agents.middleware.summarization.TriggerClause]
dict, or a list mixing either form.
A `ContextSize` tuple expresses one threshold. A `TriggerClause` dict
expresses multiple thresholds that must *all* be met (AND). When a list is
provided, summarization runs if *any* item is met (OR).
!!! example
```python
# Trigger summarization when 50 messages is reached
("messages", 50)
# Trigger summarization when 3000 tokens is reached
("tokens", 3000)
# Trigger summarization either when 80% of model's max input tokens
# is reached or when 100 messages is reached (whichever comes first)
[("fraction", 0.8), ("messages", 100)]
# Trigger when tokens >= 4000 AND messages >= 10
{"tokens": 4000, "messages": 10}
# Trigger when (tokens >= 5000 AND messages >= 3) OR
# (tokens >= 3000 AND messages >= 6)
[{"tokens": 5000, "messages": 3}, {"tokens": 3000, "messages": 6}]
```
See [`ContextSize`][langchain.agents.middleware.summarization.ContextSize]
for more details.
keep: Context retention policy applied after summarization.
Provide a [`ContextSize`][langchain.agents.middleware.summarization.ContextSize]
tuple to specify how much history to preserve.
Defaults to keeping the most recent `20` messages.
Does not support multiple values like `trigger`.
!!! example
```python
# Keep the most recent 20 messages
("messages", 20)
# Keep the most recent 3000 tokens
("tokens", 3000)
# Keep the most recent 30% of the model's max input tokens
("fraction", 0.3)
```
token_counter: Function to count tokens in messages.
summary_prompt: Prompt template for generating summaries.
trim_tokens_to_summarize: Maximum tokens to keep when preparing messages for
the summarization call.
Pass `None` to skip trimming entirely.
max_tokens_before_summaryzOmax_tokens_before_summary is deprecated. Use trigger=('tokens', value) instead. )
stacklevelNr. messages_to_keepzEmessages_to_keep is deprecated. Use keep=('messages', value) instead.r/ rF Fr? c 3 K | ]}d |v V dS )r- Nr9 ).0clauses r* z3SummarizationMiddleware.__init__..d s' X XzV 3 X X X X X Xr, r r- Ta Model profile information is required to use fractional token limits, and is unavailable for the specified model. Please use absolute token counts instead, or pass `
ChatModel(..., profile={"max_input_tokens": ...})`.
with a desired integer value of the model's maximum input tokens.) warningswarnDeprecationWarning_DEFAULT_MESSAGES_TO_KEEPsuper__init__
isinstancestrr r;
_copy_triggerrE _normalize_trigger_trigger_clauses_legacy_trigger_conditions_trigger_conditions_validate_context_sizerF r rB rG r _partial_token_counterrH rI any_get_profile_limits
ValueError) selfr; rE rF rG rH rI rJ valuerequires_profilemsg __class__s r* rY z SummarizationMiddleware.__init__ s' ^ '*; ; ;%&ABE Ma"
5#4#U+ !2 2 2%&89E MW"
$=>>>"E*
eS ! ! +#E**E
w ' '
!% 7 7 E E $(#B#B4<#P#P //f== 6 6 6!?
!K!KD 8?"u9 9 9D ' ' "/D *7D ' ,(@% X X$BW X X XXX 9Q<: % % # " 8 8 : : BT
S// ! " " B Br, stateruntimec |d }| | | | }| || sdS | | }|dk rdS | || \ }}| | }| | } dt t g| |iS a! Process messages before model invocation, potentially triggering summarization.
Args:
state: The agent state.
runtime: The runtime environment.
Returns:
An updated state with summarized messages if summarization was performed.
r/ Nr id) _ensure_message_idsrG _should_summarize_determine_cutoff_index_partition_messages_create_summary_build_new_messagesr r
rf rk rl r/ total_tokens cutoff_indexmessages_to_summarizepreserved_messagessummary new_messagess
r* before_modelz$SummarizationMiddleware.before_modelq s $ ***))(33 %%h == 433H== 1 4484L4LXWc4d4d1 1&&'<==//88
!4555 $
r, c K |d }| | | | }| || sdS | | }|dk rdS | || \ }}| | d{V }| | } dt t g| |iS rn ) rq rG rr rs rt _acreate_summaryrv r r rw s
r*
abefore_modelz%SummarizationMiddleware.abefore_model s $ ***))(33 %%h == 433H== 1 4484L4LXWc4d4d1 1 --.CDDDDDDDD//88
!4555 $
r, c t | t r t dt | S t | t r d | D S | S )zPCopy mutable trigger containers so caller mutations do not affect this instance.r1 c v g | ]6}t |t r t d t | n|7S )r1 )rZ r r
dict)rQ items r*
z9SummarizationMiddleware._copy_trigger.. sN 6@g5N5NX_d4jj111TX r, )rZ r r
r list)rE s r* r\ z%SummarizationMiddleware._copy_trigger sf
gw ' ' 8g 77 7 gt $ $ #
r, c
|g S t |t r| |d gS t |t rmt | dk rg S t t
| \ }}| t d||f d gS g }|D ]}t |t r*| | |d At |t r}t | dk rjt t
| \ }}| | t d||f d |S )zHProject tuple-expressible triggers to the legacy private representation.NrE ContextSize)
rZ tuplera r lennextiteritemsr
append)rf rE kindrg
conditionsr s r* r_ z2SummarizationMiddleware._legacy_trigger_conditions s}
?I gu % % E//CCD D gw ' ' `7||q tGMMOO 4 455KD%//]T5M0R0RT]^^_ _(*
D$&&
!!$"="=dI"N"NOOOOD'**
s4yyA~~ "4
#5#5 6 6 e!!//]T5M0R0RT]^^ r, c |g S dt dt f fd }dt t t f dt f fd }g }|}t |t r | || nt |t r-| |t d| nt |t r|D ]}t |t r | || 6t |t r-| |t d| xdt | }t | n!d t | }t | |S )
a Normalize supported trigger inputs into list of Trigger clauses.
- tuple ("tokens", 3000) -> [{"tokens": 3000}]
- dict {"tokens": 4000, "messages": 10} -> [{"tokens": 4000, "messages": 10}]
- list of either -> OR across items
Ntr& c Z | d \ }}t d||i S )NrE r1 )ra r
)r r rg rf s r* _validate_and_convert_tuplezOSummarizationMiddleware._normalize_trigger.._validate_and_convert_tuple s0 55aCCKD%$ 77 7r, mc " | sd}t | i }| D ]\ }}|dvrd|}t | t |t r| d|}t | |dk r1t |t t
f sd|}t | n+t |t s| d|}t | t d||f d |||< t d
| S ) a Validate and convert a mapping to a TriggerClause.
Type checks reject silent coercion (booleans, numeric strings, and
fractional floats for integer metrics) so a misconfigured clause fails loudly
at construction. Range and positivity checks are delegated to
`_validate_context_size`, keeping a single source of truth for the rules and
error messages shared with the tuple form.
zLtrigger clause must specify at least one of 'tokens', 'messages', 'fraction'> r. r- r/ z Unsupported trigger metric: z$ trigger value must be numeric, got r- z-Fraction trigger values must be numeric, got z& trigger values must be integers, got r rE r1 )re r rZ boolr6 r8 ra r
)r ri outkvrf s r* _validate_mappingzESummarizationMiddleware._normalize_trigger.._validate_mapping sG
&d oo%*,C
1 >>>>>>C$S//) a&& * IIAIIC$S//)
??%a#u 66 . Sa S S (oo-. $As++ * KKaKKC$S//) ++D A,G,GSSSA-- -r, r z Unsupported trigger item type: zUnsupported trigger type: ) r r1 r r[ r rZ r r r
r type TypeError)rf rE r r clausessubjectr ri s ` r* r] z*SummarizationMiddleware._normalize_trigger s ?I 8; 8= 8 8 8 8 8 8# .c!2 # .} # . # . # . # . # . # .J (* gw ' ' ! NN,,W55 6 6 6 6
'
'
! NN66tM77S7STT U U U U
&
& !
)
) dG,, )NN#4#4T#:#:;;;; e,, )NN#>#>tMSW?X?X#Y#YZZZZHDJJHHC#C..(
) ?tG}}>>CC.. r, thresholdc t d t | D d }t |t r|j {|j dd x}r^||k rX|j d x}r.6 s1 M MS*S)2L2L MS M M M M M Mr, Nrx model_provider ls_providerTF)
r reversedrZ r usage_metadatar( response_metadatar+ r; _get_ls_params)rf r/ r last_ai_messagereported_tokensr$ s r* *_should_summarize_based_on_reported_tokenszBSummarizationMiddleware._should_summarize_based_on_reported_tokens1 s M MHX . . M M M
2 2 .:$3$B$F$F~WY$Z$ZZ ; 9,,%4%F%J%JK[%\%\\! -!
))++//
>> - 4ur, rx c 8 | j sdS | j D ]}d}| D ]\ }}|dk r%t | t d| k rd} n|dk r=t d| }||k r'| |t | sd} nv|dk ro| }|d} nVt |t d| z } | d k rd
} || k r'| |t | sd} n|r dS dS ) zGDetermine whether summarization should run for the current token usage.FTr/ r6 r. r- Nr8 r r )r^ r r r
r r8 rd r6 )
rf r/ rx rR
clause_metr rg threshold_tokensmax_input_tokensr s
r* rr z)SummarizationMiddleware._should_summarizeG s| $ 5+ " " F J %||~~
e:%%#h--$ue:L:L*L*L!&JE8##'+E5'9'9$ %'777 $ O O $e,<&=&=! ! 8
&+
:%%'+'?'?'A'A$'/%*
#$4tGU7K7K$K L LI A~~$% $y00 $ O O $eI&6&6! ! 1
&+
tt
ur, c | j \ }}|dv r4| | }||S | |t S | |t d| S )z7Choose cutoff index respecting retention configuration.> r. r- Nr6 )rF _find_token_based_cutoff_find_safe_cutoffrW r
)rf r/ r rg token_based_cutoffs r* rs z/SummarizationMiddleware._determine_cutoff_indexq sr i e ) ) )!%!>!>x!H!H !-)) ))(4MNN N%%hUE0B0BCCCr, c |sdS | j \ }}|dk r+| }|dS t ||z }n|dk rt | }ndS |dk rd}| | |k rdS dt | }}t | }t | dz } t
| D ]=}
||k r n4||z dz } | || d |k r| }| }8| dz }>|t | k r|}|t | k r't | dk rdS t | dz
}| || S )z2Find cutoff index based on target token retention.r r- Nr. r rM ) rF rd r6 rG r
bit_lengthrangerb _find_safe_cutoff_point) rf r/ r rg r target_token_countleftrightcutoff_candidatemax_iterations_mids r* r z0SummarizationMiddleware._find_token_based_cutoff} s 1i e : # 7 7 9 9 't!$%5%=!>!>
X
!$U 4 " "!" h ' '+= = =1 X e x== X 1133a7~&& Au}}%::>PPP#& Qw s8}} , , # s8}} , ,8}}!!q "8}}q 0 ++H6FGGGr, c | j j }n# t $ r Y dS w xY wt |t sdS | d }t |t sdS |S )z6Retrieve max input token limit from the model profile.Nr )r; profileAttributeErrorrZ r r( r6 )rf r r s r* rd z+SummarizationMiddleware._get_profile_limits s j(GG 44 '7++ 4";;'9::*C00 4 s
contextparameter_namec | \ }}|dk r(d|cxk rdk sn d| d| d}t | n:|dv r |dk r| d| d}t | nd | d
| d}t | | S ) z&Validate context configuration tuples.r- r r z Fractional z% values must be between 0 and 1, got .> r. r/ z( thresholds must be greater than 0, got z Unsupported context size type z for )re )r r r rg ri s r* ra z.SummarizationMiddleware._validate_context_size s e : u>>>>>>>>aNaaY^aaa oo% " +
+
+zz'YYQVYYY oo% P4OOnOOOC S// !r, r| c 0 t d| ddi gS )Nz0Here is a summary of the conversation to date:
lc_source
summarization)contentadditional_kwargsr )r| s r* rv z+SummarizationMiddleware._build_new_messages s3
VWVV#."@
r, c h | D ].}|j %t t j |_ /dS )zAEnsure all messages have unique IDs for the add_messages reducer.N)rp r[ uuiduuid4)r/ ri s r* rq z+SummarizationMiddleware._ensure_message_ids s: + +Cv~ TZ\\** + +r, conversation_messagesry c 2 | d| }| |d }||fS )zAPartition messages into those to summarize and those to preserve.Nr9 )r ry rz r{ s r* rt z+SummarizationMiddleware._partition_messages s. !6m|m D 2<== A$&888r, rO c | t | |k rdS t | |z
}| || S )a Find safe cutoff point that preserves AI/Tool message pairs.
Returns the index where messages can be safely cut without separating
related AI and Tool messages. Returns `0` if no safe cutoff is found.
This is aggressive with summarization - if the target cutoff lands in the
middle of tool messages, we advance past all of them (summarizing more).
r )r r )rf r/ rO
target_cutoffs r* r z)SummarizationMiddleware._find_safe_cutoff sB x== , , ,1H
(88
++HmDDDr, c ` |t | k st | | t s|S t }|}|t | k rt | | t rjt d| | }|j r| |j |dz
}|t | k rt | | t jt |dz
dd D ]@}| | }t |t r!|j rd |j D }||z r|c S A|S )a Find a safe cutoff point that doesn't split AI/Tool message pairs.
If the message at `cutoff_index` is a `ToolMessage`, search backward for the
`AIMessage` containing the corresponding `tool_calls` and adjust the cutoff to
include it. This ensures tool call requests and responses stay together.
Falls back to advancing forward past `ToolMessage` objects only if no matching
`AIMessage` is found (edge case).
r r r c b h | ],}| d | d -S ro )r( )rQ tcs r* zBSummarizationMiddleware._find_safe_cutoff_point.. s2 #Z#Z#ZRRVVTX\\#ZBFF4LL#Z#Z#Zr, )
r rZ r setr
tool_call_idaddr r
tool_calls)r/ ry
tool_call_idsidxtool_msgiri ai_tool_call_idss r* r z/SummarizationMiddleware._find_safe_cutoff_point sC 3x== ( (
8L;QS^0_0_ ( #&%%
CMM!!j# &L&L!M8C=99H$
9 !!("7888 1HC CMM!!j# &L&L! |a'R00 A1+C#y))
cn
#Z#Z3>#Z#Z#Z #33 HHH
r, rz c ` |sdS | | }|sdS t |d } | j | j | dddii }|j S # t $ r}d
|cY d }~S d }~ww xY w) zGenerate summary for the given messages.
Args:
messages_to_summarize: Messages to summarize.
!No previous conversation history.0Previous conversation was too long to summarize.xmlformatr/ metadatar r configError generating summary: N)
_trim_messages_for_summaryr r; invokerH r rstriptextstrip Exceptionrf rz trimmed_messagesformatted_messagesresponsees r* ru z'SummarizationMiddleware._create_summary s % 766 ::;PQQ FEE //? N N N 6z((#**4F*GGNNPP"[/$BC ) H =&&(( ( 6 6 6555 5 5 5 5 5 5 6s A#B
B- B("B-(B-c p K |sdS | | }|sdS t |d } | j | j | dddii d
{V }|j S # t $ r}d |cY d
}~S d
}~ww xY w) r r r r r r r r r r Nr )
r r r; ainvokerH r r r r r r s r* r z(SummarizationMiddleware._acreate_summary8 s % 766 ::;PQQ FEE //? N N N 6 !Z / /#**4F*GGNNPP"[/$BC 0 H =&&(( ( 6 6 6555 5 5 5 5 5 5 6s A)B
B5&B0*B50B5c | j |S t dt || j | j dddd S # t $ r |t
d cY S w xY w)z6Trim messages to fit within summary generation limits.Nzlist[AnyMessage]humanlastT)
max_tokensrG start_onstrategy
allow_partialinclude_system)rI r
r rG r _DEFAULT_FALLBACK_MESSAGE_COUNT)rf r/ s r* r z2SummarizationMiddleware._trim_messages_for_summaryR s ?,4 " # <"&"4 $ #"&#'
? ? ? < < = => > > > ?s 9 -9 AA)0r2 r3 r4 r5 rW r DEFAULT_SUMMARY_PROMPT_DEFAULT_TRIM_TOKEN_LIMITr[ r r r1 r TokenCounterr6 r rY r r r r r r~ r staticmethodr\ r_ r] r r8 r r rr rs r rd ra r rv rq r rt r r ru r r
__classcell__)rj s @r* rD rD s8 ]a ')B C&@ 4/HL" L" L"]"L"
-[=5P0QQTXX L"
L" $
L" L" #&*L" !L"
L" L" L" L" L" L"\ #
_#
/6x/@#
c3h $ #
#
#
X#
J #
_#
/6x/@#
c3h $ #
#
#
X#
J },tK-4O/PPSWW } $tK-,G'H H4 O \ },tK-4O/PPSWW
k 4J
-[=5P0QQTXXJ
m J J J JX Z(5:
,($z*: (# (RV ( ( ( (T
DZ0@
DS
D
D
D
D/Hj1A /HcDj /H /H /H /Hb S4Z " S [ \
S
T,-?
\
+d:&6 +4 + + + \+ 9 #J /99
tJ j!11 29 9 9 \9
E$z*:
Ec
EVY
E
E
E
E !$z*: !# !RU ! ! ! \!F6T*5E 6# 6 6 6 646Dr5 r rT collections.abcr r r functoolsr typingr r r r
langchain_core.messagesr r r
r r langchain_core.messages.humanr langchain_core.messages.utilsr r r langgraph.graph.messager langgraph.runtimer typing_extensionsr !langchain.agents.middleware.typesr r r r langchain.chat_modelsr r r6 r r rW r r frozensetr# r r[ r7 r r+ r r8 ContextFraction
ContextTokensContextMessagesr r1 rB rD r9 r, r* r s5 7 7 7 7 7 7 7 7 7 7 0 0 0 0 0 0 0 0 0 0 0 0 7 6 6 6 6 6
& % % % % % & & & & & & ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ @ @ @ @ @ @ @ @";<=sBC - \
"$ ii,> ?@@3 d3 #./
? ?d
?t ? ? ? ?
+U23 gh',-
+S01
-? 8 IU DP- PL P P P PI
? I
? I
? I
? I
?oj.CXy.X Y I
? I
? I
? I
? I
?r,