Skip to content

Fix/truncation strategy#257

Open
philipph-askui wants to merge 7 commits intomainfrom
fix/truncation_strategy
Open

Fix/truncation strategy#257
philipph-askui wants to merge 7 commits intomainfrom
fix/truncation_strategy

Conversation

@philipph-askui
Copy link
Copy Markdown
Contributor

@philipph-askui philipph-askui commented Apr 2, 2026

Refactors truncation strategies to align with prompt caching

  • makes truncation strategy configurable as parameter of the agent
  • removes obsolete factory logic for truncation strategies
  • replaces the old SimpleTruncationStrategy (which deleted messages to stay under limits) with a new summarization-based strategy SummarizingTruncationStrategy that use the VLM to compress history instead of discarding it
  • adds a new experimental SlidingImageWindowSummarizingTruncationStrategy, that additionally strips old base64 images beyond a configurable window and uses dual cache breakpoints (image-removal boundary + last user message).

For reviewing: I think you don't need to review the experimental SlidingImageWindowSummarizingTruncationStrategy in detail at this point.

@philipph-askui philipph-askui marked this pull request as ready for review April 2, 2026 05:53
Copy link
Copy Markdown
Collaborator

@programminx-askui programminx-askui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This are my first comments.

I need to continue the review.

# see https://docs.anthropic.com/en/api/messages#body-messages
MAX_MESSAGES = 100_000

IMAGE_REMOVED_PLACEHOLDER = "[Screenshot removed to reduce message history length]"
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume the description of the image is partly in the messages history or? So the LLM can assume what the App has done.

e.g. I see that an Error Message popped up. ...

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the rest of the content in the message history, e.g. the interpretation of the LLM remains unchanged. we just remove the image here. Generating a dedicated description for each image would create additional traffic and cost, hence I decided against that option

not _is_tool_result_user_message(message)
and previous_message is not None
and previous_message.role == "assistant"
response = vlm_provider.create_message(
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this Token Consumption tracked in our Report? Please say yes. :-D

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MessageParam(
role="user",
content=(
"Please provide a concise summary of the "
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How did you came up this the message?

Can me make the message configurable?

I would add something like. You are an CUA agent how should continue the conversation.. But

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed in b23dd88

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for the exact content: lets keep it simple in the beginning. This simple prompt worked quite well in my test. If we see that the agent loses track after summarizing, we can improve it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants