fix(cache): compact newest tool results first to preserve prompt cache prefix#1
Open
fix(cache): compact newest tool results first to preserve prompt cache prefix#1
Conversation
…#60452) * refactor: move bundled replay policy ownership into plugins * test: preserve replay fallback until providers adopt hooks * test: cover response replay branches for ollama and zai --------- Co-authored-by: Shakker <shakkerdroid@gmail.com>
Merged via squash. Prepared head SHA: 28ae50b Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com> Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com> Reviewed-by: @gumadeiras
…or [AI] (openclaw#59822) * fix: address issue * fix: address PR review feedback * docs: add changelog entry for PR merge * docs: normalize changelog entry placement --------- Co-authored-by: Devin Robison <drobison@nvidia.com>
…a exceeds size limit (openclaw#60289) Merged via squash. Prepared head SHA: f33dd49 Co-authored-by: efe-arv <259833796+efe-arv@users.noreply.github.com> Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com> Reviewed-by: @gumadeiras
* fix(sandbox): block home credential binds * fix(sandbox): harden blocked credential bind checks
…e prefix compactExistingToolResultsInPlace iterated front-to-back, replacing the oldest tool results with placeholders when context exceeded 75%. This rewrote messages[k] for small k, invalidating the provider prompt cache from that point onward on every subsequent turn. Reverse the loop to compact newest-first. The cached prefix stays intact; the tradeoff is the model loses recent tool output instead of old, which is acceptable since this guard only fires as an emergency measure past the 75% threshold.
adcf7eb to
1358cba
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
compactExistingToolResultsInPlace(src/agents/pi-embedded-runner/tool-result-context-guard.ts:111) iterated front-to-back. When context exceeded 75%, it replaced the oldest tool results with[compacted: ...]placeholders first. This rewrotemessages[k]for small k, invalidating the provider prompt cache from that point onward on every turn past the threshold.Fix
Reverse the loop to compact newest-first:
for (let i = messages.length - 1; i >= 0; i--). The cached prefix stays byte-identical; only the tail diverges.Tradeoff
The model loses recent tool output instead of old. Acceptable because:
Verification
tool-result-context-guard.test.tsenforceToolResultContextBudgetInPlaceonly reads the return value (a sum) and the in-place mutationsreduced >= charsNeeded) is direction-agnostic — the accumulator is a simple sumapplyMessageMutationInPlace+ cache invalidation are per-message, no ordering assumptionNotes
origin/maindue to unrelatedgetEditorKeybindingsimport breakage introduced in c6d8318 — this change does not touch those files🤖 Generated with Claude Code