Skip to content

feat(schema-drift): envelope validation for openai/groq/cerebras (#39 slice 2)#49

Merged
stackbilt-admin merged 2 commits intomainfrom
feat/schema-drift-slice-2-openai-compat
Apr 21, 2026
Merged

feat(schema-drift): envelope validation for openai/groq/cerebras (#39 slice 2)#49
stackbilt-admin merged 2 commits intomainfrom
feat/schema-drift-slice-2-openai-compat

Conversation

@stackbilt-admin
Copy link
Copy Markdown
Member

Summary

Follow-up to PR #40 (Anthropic slice 1 of #39). Wires response envelope schema validation into the three OpenAI-compatible providers so silent upstream contract drift routes through the existing SchemaDriftError → fallback → onSchemaDrift machinery instead of corrupting downstream parsers.

  • openaiOPENAI_RESPONSE_SCHEMA validates id / model / choices[].message.{content,tool_calls} / usage.*
  • groq — same envelope (OpenAI-compat)
  • cerebras — same envelope (OpenAI-compat)
  • Cloudflare is deferred — Workers AI returns heterogeneous shapes across model families (response / choices / output / result wrappers, all optional). Needs model-family-aware schema selection rather than a copy-paste template. Will file as its own PR.

Design decisions

Per-provider schema constants, not a shared import. Each provider's envelope is an independent API surface. If OpenAI ships a breaking change and Groq doesn't, we want both drift signals distinct, not hidden behind a single `OPENAI_COMPATIBLE_SCHEMA`.

Discriminated-union for `tool_calls`, even with one variant today. Matches the Anthropic template and gives forward-compat if/when OpenAI adds a new tool type (e.g. `code_interpreter`) — additive upstream change doesn't trip drift.

`No choices returned from X` → `SchemaDriftError('x', 'choices[0]', 'object', 'undefined')`. Empty choices was previously an uncaught generic `Error` that bypassed the fallback chain. Now it's drift-eligible like every other envelope failure, per PR #40 conventions.

Behavioral upgrade (called out explicitly)

One test from PR #23 was updated, not broken:

  • Before: OpenAI returning `tool_calls[].function.arguments: 42` (a number) was silently dropped by `validateToolCalls`.
  • After: It's routed as `SchemaDriftError` at path `choices[0].message.tool_calls[0].function.arguments`.

Rationale: OpenAI's contract is "stringified JSON". A non-string is an envelope violation, not a within-envelope semantic issue. Division of responsibility:

  • Envelope shape violations → schema drift → fallback (observable)
  • Within-envelope semantic issues (empty id/name, missing function object, etc.) → `validateToolCalls` → silent drop (unchanged)

Three existing PR #23 tests still assert silent drop (empty id, empty name, non-`function` type) because those are within-envelope semantic issues — schema validation accepts the shape, `validateToolCalls` filters the contents.

Test coverage

  • Driven via `describe.each` over `[openai, groq, cerebras]` so schema-parity is enforced by construction — one provider's schema diverging breaks its tests loudly.
  • 6 tests per provider × 3 providers = 18 new tests:
    • well-formed response passes through
    • `usage.prompt_tokens` rename → drift
    • `choices` removed → drift
    • `choices: []` → drift at `choices[0]` path
    • non-string `tool_calls[].function.arguments` → drift
    • unknown `tool_calls[].type` variant → forward-compat accept (no drift)
  • 229 → 247 tests passing on Node 18/20/22 (expected via CI).

Test plan

  • CI green on Node 18, 20, 22
  • Existing Response envelope schema validation + schema drift canary #39 factory-fallback + onSchemaDrift-hook tests still pass (unchanged; slice 2 only adds new drift sites)
  • Downstream consumers that touch OpenAI/Groq/Cerebras paths continue to work (no public API change; schema validation is transparent when envelopes are well-formed)

Follow-ups (tracked in #39)

🤖 Generated with Claude Code

stackbilt-admin and others added 2 commits April 21, 2026 16:54
…ras (#39 slice 2)

Follow-up to PR #40 (Anthropic slice 1 of #39). Each OpenAI-compat
provider now validates its /chat/completions response envelope at the
provider boundary and throws SchemaDriftError on mismatch — routing
through the factory's fallback chain and firing onSchemaDrift instead
of corrupting downstream consumers silently.

Per-provider schema constants (not a shared import) — each provider's
envelope is an independent API surface, and correlated drift across
providers is a signal worth detecting, not hiding behind DRY.

The "No choices returned" bare throws in openai/groq/cerebras are
replaced with `SchemaDriftError(<provider>, 'choices[0]', 'object',
'undefined')` so empty-choices is fallback-eligible like every other
envelope failure mode, rather than bubbling up as an uncaught generic
Error.

Also upgrades one tool-call-validation test from PR #23: non-string
`function.arguments` is an envelope contract violation (OpenAI spec
says stringified JSON), so it now routes through drift rather than
silent drop. Division of responsibility:
- Envelope shape violations → schema drift → fallback
- Within-envelope semantic issues (empty id/name) → validateToolCalls
  → silent drop (unchanged)

Driven via describe.each over [openai, groq, cerebras] so schema-parity
is enforced by construction — if one provider's schema diverges, its
tests break loudly.

Tests: 229 → 247 (+18 for the 3 new providers, +0 net on
tool-call-validation — one test's assertion was updated).

Cloudflare deferred to its own PR: Workers AI returns heterogeneous
shapes across model families (response / choices / output / result
wrappers, all optional), needing model-family-aware schema selection
rather than a copy-paste template.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…boundary

Review on PR #49 flagged: groq/cerebras formatResponse unconditionally
dereferenced `tc.function.name` / `tc.function.arguments`, but the new
schema's discriminated-union intentionally skips unknown `type` values
for forward-compat. A future `code_interpreter`-style variant with no
`function` field would pass schema validation, then throw a bare
TypeError inside `.map()` — bypassing SchemaDriftError and the
onSchemaDrift hook.

Second failure mode: if the unknown variant happens to carry a
function-shaped payload, it gets mis-surfaced as a normal function
tool call.

Fix: filter `tool_calls` by `type === 'function'` before the map, so
the schema's forward-compat skip is matched by the provider code's
forward-compat drop. OpenAI's version already passes `tc.function`
through untouched, and `validateToolCalls` drops null-function entries
downstream, so no change there.

Test upgrade: the original slice-2 test used a function-shaped mock
and only asserted res.content, so it missed both failure modes. New
test omits the `function` field (exercises the TypeError path) and
asserts `res.toolCalls === undefined` (exercises the mis-surface
path).

All 247 tests passing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@stackbilt-admin
Copy link
Copy Markdown
Member Author

Reviewer finding addressed in 5f084f0.

Root cause of the bug: schema discriminator says "unknown variant → skip (forward-compat)," but groq/cerebras `formatResponse` said "every tool_call has a `function` field." Two inconsistent forward-compat stances in the same code path.

Fix: filter `tool_calls` to `type === 'function'` before the map. Matches the schema's skip behavior explicitly at the map boundary. openai was already safe — passes `tc.function` through untouched, validateToolCalls drops null-function entries — so no change there.

Test upgrade: original test mocked a function-shaped payload and only asserted `res.content`, which hid both failure modes. New test:

  • Omits `function` entirely (exercises the TypeError-through-`.map()` path)
  • Asserts `res.toolCalls === undefined` (exercises the mis-surfaced-as-function-call path)

Two real bugs, one surfacing test. Thanks for the careful read.

All 247 tests still passing.

@stackbilt-admin stackbilt-admin merged commit 1aa6253 into main Apr 21, 2026
3 checks passed
@stackbilt-admin stackbilt-admin deleted the feat/schema-drift-slice-2-openai-compat branch April 21, 2026 22:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant