Skip to content

feat: optimize Gemini context caching boundary#5522

Open
Itish2003 wants to merge 1 commit into
google:mainfrom
Itish2003:fix-context-caching-boundary
Open

feat: optimize Gemini context caching boundary#5522
Itish2003 wants to merge 1 commit into
google:mainfrom
Itish2003:fix-context-caching-boundary

Conversation

@Itish2003
Copy link
Copy Markdown

This PR optimizes the GeminiContextCacheManager by implementing a 'last user batch' boundary strategy instead of naively caching the entire history. This cleanly isolates the stable prefix (system instructions, tools, prior conversation) from the transient suffix (the latest user message), preventing cache misses on multi-turn conversations and drastically improving token efficiency.

Changes:

  • Modified _find_count_of_contents_to_cache to dynamically determine the boundary before the last continuous batch of user messages.
  • Cleaned up manual cache stripping and caching logic to respect this dynamic boundary.
  • Removed the hardcoded 4096 token limit in favor of the cache's minimum token limits.

@google-cla
Copy link
Copy Markdown

google-cla Bot commented Apr 28, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@adk-bot adk-bot added the core [Component] This issue is related to the core interface and implementation label Apr 28, 2026
@adk-bot
Copy link
Copy Markdown
Collaborator

adk-bot commented Apr 28, 2026

Response from ADK Triaging Agent

Hello @Itish2003, thank you for creating this PR!

In order to get this PR reviewed and merged, could you please:

  • Sign our Contributor License Agreement (CLA).
  • Associate a GitHub issue with this PR. If one does not exist, please create one.
  • Include a testing plan section in your PR to describe how you tested your changes.

This information will help reviewers to review your PR more efficiently. Thanks!

@rohityan rohityan self-assigned this Apr 30, 2026
@rohityan rohityan added the request clarification [Status] The maintainer need clarification or more information from the author label May 8, 2026
@rohityan
Copy link
Copy Markdown
Collaborator

rohityan commented May 8, 2026

Hi @Itish2003 , Thank you for your contribution! It appears you haven't yet signed the Contributor License Agreement (CLA). Please visit https://cla.developers.google.com/ to complete the signing process. Once the CLA is signed, we'll be able to proceed with the review of your PR. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core [Component] This issue is related to the core interface and implementation request clarification [Status] The maintainer need clarification or more information from the author

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants