fix: don't subtract cached tokens from input_tokens in OpenAI→Claude translation#489
fix: don't subtract cached tokens from input_tokens in OpenAI→Claude translation#489skrabe wants to merge 1 commit intorouter-for-me:mainfrom
Conversation
…translation When translating OpenAI responses to Anthropic format, extractOpenAIUsage subtracted cached_tokens from prompt_tokens before reporting input_tokens. This caused downstream clients (Claude Code, Factory Droid) to see near-zero input_tokens on cache hits, breaking context tracking and compaction triggers. Example: prompt_tokens=150000, cached_tokens=149900 → reported input_tokens=100 instead of the correct 150000. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request updates the .gitignore file, simplifies token usage extraction in the Claude translator by removing manual subtraction of cached tokens, and introduces an update-and-build.sh script for automated deployment. Feedback was provided to enhance the automation script by dynamically detecting the current branch and ensuring the installation directory exists before copying the binary.
| BINARY_NAME="cli-proxy-api-plus" | ||
| INSTALL_PATH="$HOME/.local/bin/$BINARY_NAME" | ||
| BACKUP_PATH="$INSTALL_PATH.bak" | ||
| FIX_BRANCH="fix/byok-compaction-cached-tokens" |
There was a problem hiding this comment.
The branch name is hardcoded to a specific feature branch (fix/byok-compaction-cached-tokens). This makes the script unusable for future updates or other branches. It is better to dynamically detect the current branch so the script remains functional across different tasks.
| FIX_BRANCH="fix/byok-compaction-cached-tokens" | |
| FIX_BRANCH=$(git rev-parse --abbrev-ref HEAD) |
| cp "$INSTALL_PATH" "$BACKUP_PATH" 2>/dev/null || true | ||
|
|
||
| echo "==> Installing..." | ||
| cp "$BINARY_NAME" "$INSTALL_PATH" |
There was a problem hiding this comment.
The script assumes that the destination directory ~/.local/bin/ already exists. If it does not, the cp command will fail. It is safer to ensure the directory exists before attempting to install the binary.
| cp "$BINARY_NAME" "$INSTALL_PATH" | |
| mkdir -p "$(dirname "$INSTALL_PATH")" | |
| cp "$BINARY_NAME" "$INSTALL_PATH" |
Summary
extractOpenAIUsagein the OpenAI→Claude response translator subtractscached_tokensfromprompt_tokensbefore reportinginput_tokensto downstream clients. This causes clients that rely oninput_tokensfor context window tracking (Claude Code, Factory Droid, etc.) to see near-zero values on cache hits, breaking compaction triggers.Before:
prompt_tokens=150000, cached_tokens=149900→input_tokens=100After:
prompt_tokens=150000, cached_tokens=149900→input_tokens=150000The
cache_read_input_tokensfield is still set separately for clients that need the breakdown.Context
internal/translator/openai/claude/openai_claude_response.go)Test plan
input_tokensin Anthropic-format responses includes cached tokenscache_read_input_tokensis still reported correctly🤖 Generated with Claude Code