Skip to content

fix: don't subtract cached tokens from input_tokens in OpenAI→Claude translation#489

Closed
skrabe wants to merge 1 commit intorouter-for-me:mainfrom
skrabe:fix/byok-compaction-cached-tokens
Closed

fix: don't subtract cached tokens from input_tokens in OpenAI→Claude translation#489
skrabe wants to merge 1 commit intorouter-for-me:mainfrom
skrabe:fix/byok-compaction-cached-tokens

Conversation

@skrabe
Copy link
Copy Markdown

@skrabe skrabe commented Apr 5, 2026

Summary

extractOpenAIUsage in the OpenAI→Claude response translator subtracts cached_tokens from prompt_tokens before reporting input_tokens to downstream clients. This causes clients that rely on input_tokens for context window tracking (Claude Code, Factory Droid, etc.) to see near-zero values on cache hits, breaking compaction triggers.

Before: prompt_tokens=150000, cached_tokens=149900input_tokens=100
After: prompt_tokens=150000, cached_tokens=149900input_tokens=150000

The cache_read_input_tokens field is still set separately for clients that need the breakdown.

Context

Test plan

  • Verify input_tokens in Anthropic-format responses includes cached tokens
  • Verify cache_read_input_tokens is still reported correctly
  • Confirm Claude Code compaction triggers at expected thresholds with BYOK

🤖 Generated with Claude Code

…translation

When translating OpenAI responses to Anthropic format, extractOpenAIUsage
subtracted cached_tokens from prompt_tokens before reporting input_tokens.
This caused downstream clients (Claude Code, Factory Droid) to see near-zero
input_tokens on cache hits, breaking context tracking and compaction triggers.

Example: prompt_tokens=150000, cached_tokens=149900 → reported input_tokens=100
instead of the correct 150000.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@skrabe skrabe closed this Apr 5, 2026
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the .gitignore file, simplifies token usage extraction in the Claude translator by removing manual subtraction of cached tokens, and introduces an update-and-build.sh script for automated deployment. Feedback was provided to enhance the automation script by dynamically detecting the current branch and ensuring the installation directory exists before copying the binary.

BINARY_NAME="cli-proxy-api-plus"
INSTALL_PATH="$HOME/.local/bin/$BINARY_NAME"
BACKUP_PATH="$INSTALL_PATH.bak"
FIX_BRANCH="fix/byok-compaction-cached-tokens"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The branch name is hardcoded to a specific feature branch (fix/byok-compaction-cached-tokens). This makes the script unusable for future updates or other branches. It is better to dynamically detect the current branch so the script remains functional across different tasks.

Suggested change
FIX_BRANCH="fix/byok-compaction-cached-tokens"
FIX_BRANCH=$(git rev-parse --abbrev-ref HEAD)

cp "$INSTALL_PATH" "$BACKUP_PATH" 2>/dev/null || true

echo "==> Installing..."
cp "$BINARY_NAME" "$INSTALL_PATH"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The script assumes that the destination directory ~/.local/bin/ already exists. If it does not, the cp command will fail. It is safer to ensure the directory exists before attempting to install the binary.

Suggested change
cp "$BINARY_NAME" "$INSTALL_PATH"
mkdir -p "$(dirname "$INSTALL_PATH")"
cp "$BINARY_NAME" "$INSTALL_PATH"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant