Mac base/Pro 16 GB support: Qwen 2.5 14B, ChatML stop markers, <tools> parser, offline leak fix

[tadrianonet]

Why

The README targets Mac Max/Ultra (64-128 GB). On a Mac base/Pro 16 GB
five reproducible failures stop the project from working out of the box,
the worst of which silently breaks the "100% offline / 0$/mo" promise.

This branch documents and fixes all five.

What's broken on 16 GB (and how this branch fixes each)

1. macOS keychain bug — login screen on every launch even with
   ANTHROPIC_API_KEY set. The launcher's --bare flag doesn't
   exist on Claude Code 2.1.76. → Workaround: set hasCompletedOnboarding
   in ~/.claude.json, export ANTHROPIC_AUTH_TOKEN alongside
   ANTHROPIC_API_KEY, set DISABLE_LOGIN_COMMAND=1. Documented +
   automated in the new launchers.

2. clean_response only handled Gemma 4 stop markers (<turn|>,
   <|turn>). On any ChatML/Llama 3.x model the special tokens leaked
   into the visible answer (<|im_end|>, <|endoftext|>, <|eot_id|>).
   → Extended the marker list (commit 1).

3. Claude Code 2.1's "extended thinking" exhausts small models
   on the first call (the thinking pass), leaving the answer pass with
   no budget → (No output). → Force --effort low in the launchers
   (disables the two-step flow). Documented in MAC-BASE-SETUP.md §3.

4. parse_tool_calls didn't recognize Qwen 2.5's <tools> wrapper.
   Qwen 2.5 Coder fine-tunes emit tool calls inside <tools>...</tools>
   instead of <tool_call> (Qwen 3.5 / generic ChatML). Without the
   parser knowing this, tool calls were returned as plain text and
   Claude Code never executed them. → Added Format 3.5 to the parser
   (commit 4).

5. (Most serious) Claude Code calls api.anthropic.com on startup
   even with ANTHROPIC_BASE_URL set. Confirmed via lsof:

   claude TCP mac:63057->160.79.104.10:https (ESTABLISHED)

whois 160.79.104.10 → OrgName: Anthropic, PBC.
dig +short api.anthropic.com → 160.79.104.10.

The CLI fires telemetry, statsig feature flags, marketplace
auto-install and autoupdater checks against api.anthropic.com
directly, bypassing the configured base URL. That means setups
following the current README are not actually offline.

→ Export 4 env vars in the launchers (and document):

CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 DISABLE_AUTOUPDATER=1 CLAUDE_CODE_DISABLE_OFFICIAL_MARKETPLACE_AUTOINSTALL=1 CLAUDE_CODE_DISABLE_BACKGROUND_TASKS=1

With these set, lsof shows the claude process only connects to
localhost:4000. Validated end-to-end.

Recommended model for 16 GB

mlx-community/Qwen2.5-Coder-14B-Instruct-4bit:

• 7.8 GB weights → fits in 16 GB unified memory with no swap
• Tool calls validated end-to-end after Format 3.5 fix
• ~10–15 tok/s generation, ~0.2–0.4 tok/s with cold prefill of the
  full Claude Code system prompt (~3000 tokens)

Gemma 4 31B Abliterated (the upstream default) was tested first and
crashes with kIOGPUCommandBufferCallbackErrorOutOfMemory on the very
first inference on 16 GB — 18 GB of weights + KV cache simply don't
fit alongside macOS. The MAC-BASE-SETUP.md table has been updated to
reflect this.

Files

• proxy/server.py: stop-marker fix + Format 3.5 parser
• launchers/Claude Chat.command: chat-only (--tools "")
• launchers/Claude Agentico.command: tools enabled (--permission-mode auto)
• docs/MAC-BASE-SETUP.md: full PT-BR diagnosis of all 5 failures with
  reproducible commands and validation logs

Test plan

• Server loads Qwen2.5-Coder-14B-Instruct-4bit cleanly (~30s)
• Plain /v1/messages PT-BR call: clean text, no token leakage,
  stop_reason=end_turn
• /v1/messages with tools=[Bash]: returns stop_reason=tool_use
  with Bash({"command":"pwd"})
• End-to-end with Claude Code CLI 2.1.76: --print "Hi" returns
  Hello! How can I assist you today? from the local model
• lsof -p $CLAUDE_PID confirms zero connections to
  api.anthropic.com, only localhost:4000

[nicedreamzapp]

Thanks @tadrianonet — this is a really substantial contribution and I appreciate the depth here.

The standout for me is #5 — the api.anthropic.com leak on startup. The repo's whole pitch is "100% offline / $0/mo" and your lsof evidence shows that wasn't actually true without CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 and friends. That's the kind of finding I'd never have caught on Max/Ultra hardware where the auth path "just works" and nobody packet-watches the startup. Glad you did, and validating with lsof after the fix is exactly the right receipts. Merging that bit alone is worth the PR.

The ChatML stop-marker extension, the Qwen 2.5 <tools> Format 3.5 parser (nicely gated behind if not tool_calls: so it can't fight existing paths), and the --effort low workaround for the small-model two-pass thinking issue are all clean and well-motivated. The keychain workaround is the known upstream pattern, so no surprises there.

Two small things before merge:

1. MAC-BASE-SETUP.md is PT-BR only — could you add an English version, or at minimum an English summary at the top? Most of the repo's audience is English-speaking, and the diagnostics in this doc (especially the offline-leak section) are valuable enough that I want them to land for everyone. Keeping the PT-BR alongside as MAC-BASE-SETUP.pt-BR.md would be totally welcome.

2. Tiny ordering issue in the clean_response stop-marker loop:

for stop_marker in ['<turn|>', '<|turn>', '<|im_end|>', '<|endoftext|>',
                    '<|im_start|>', '<|end_of_text|>', '<|eot_id|>']:
    if stop_marker in text:
        text = text[:text.index(stop_marker)].strip()
        break

This breaks on the first marker that appears in list order, not the one with the earliest position in the text. If two markers both show up, list order decides where we truncate instead of position — so output past the actual earliest marker can leak through. Suggested change:

markers = ['<turn|>', '<|turn>', '<|im_end|>', '<|endoftext|>',
           '<|im_start|>', '<|end_of_text|>', '<|eot_id|>']
positions = [text.index(m) for m in markers if m in text]
if positions:
    text = text[:min(positions)].strip()

Edge case in practice, but worth fixing while you're here.

Once those two are in, this is good to go. Thanks again — this is the kind of PR that makes me glad I open-sourced this thing.