fix(models): Port Gemini context cache fingerprint scoping to v1 - #6804
Open
GWeale wants to merge 1 commit into
Open
fix(models): Port Gemini context cache fingerprint scoping to v1#6804GWeale wants to merge 1 commit into
GWeale wants to merge 1 commit into
Conversation
The explicit-cache fingerprint was computed from the system instruction, tools, tool config and content prefix alone. It did not include the model, and it did not include the backend the cache lives in. An identical prompt prefix under a different model, or under a different Vertex project or location, produced the same fingerprint, so a stored cache_name belonging to another context was treated as reusable and attached to a request that had no relationship to it. The fingerprint is now seeded with the model id and a cache scope. The scope records whether the client talks to Vertex or the Developer API, and for Vertex it records the project, the location and the base URL. Model dumps run in JSON mode and the hash input is canonical JSON with sorted keys, rather than the repr of a Python dict. A system instruction the SDK accepts but cannot serialize to JSON falls back to its string form. Behaviour change: every fingerprint value changes, so on upgrade a user with an active explicit context cache takes one cache miss and one cache re-creation. That cost is unavoidable while fixing the identity, because the old fingerprints did not identify what they were caching. In the other direction, argument mappings that differ only in key insertion order now hash the same, which removes cache misses that were happening for no reason.
GWeale
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR ports one fix to the
v1branch.fix: scope Gemini cache identity(8de4237e)cache_scoperecording the backend, the client base URL when one is set, and, for Vertex, the project and location.mode="json"and hashed as canonical JSON with sorted keys.Behaviour change. Existing explicit context caches are invalidated and recreated once on upgrade, with no opt-out. A stored
cache_namematches only for the same model, backend, base URL, and Vertex project and location.No public API change.
_generate_cache_fingerprintkeeps its signature and its 16-character hexadecimal return value.