Skip to content

v1.6.0 — SSE validation, cache hints, schema canary

Latest

Choose a tag to compare

@stackbilt-admin stackbilt-admin released this 27 Apr 23:23
· 1 commit to main since this release

What's new

Streaming schema validation (#41)

All four providers now surface malformed SSE frames as SchemaDriftError and fire onSchemaDrift instead of swallowing silently. Anthropic additionally validates content_block_delta event shape and delta.text type; future tool-streaming delta types are skipped via forward-compat discriminator.

Cache-aware routing (#52)

  • New CacheHints type — LLMRequest.cache is a no-op for callers that don't set it
  • Anthropic: strategy: 'provider-prefix' wraps the system prompt as a content block with cache_control: { type: 'ephemeral' } and marks the last tool as a breakpoint
  • OpenAI / Groq / Cerebras: automatic caching with no request-side translation needed
  • Cached token counts normalized into TokenUsage: cachedInputTokens, cacheReadInputTokens, cacheCreationInputTokens
  • supportsPromptCache flag added to ModelCapabilities

Schema drift canary (#39 Part 2)

  • extractShape(obj) — flat path → type map from any response object
  • compareShapes(golden, live) — diffs two shape maps into { added, removed, changed }
  • runCanaryCheck(provider, golden, liveResponse) — one-shot canary returning a CanaryReport
  • Golden fixtures committed for all five providers under src/__tests__/fixtures/response-shapes/
  • All three utilities exported from the package root

Previously merged, now documented

  • Factory-level streaming with fallback (#26) — generateResponseStream uses the same circuit-breaker and fallback chain as generateResponse
  • Tool-use loop helper (#28) — generateResponseWithTools with ToolLoopLimitError, ToolLoopAbortedError, iteration/cost caps, and abort-signal support
  • Cloudflare AI Gateway metadata forwarding (#29) — cf-aig-* headers forwarded only when baseUrl matches the Gateway pattern
  • Cloudflare LoRA / fine-tune forwarding (#51) — LLMRequest.lora forwarded to Workers AI binding

Bug fixes

  • stop_sequence schema false positive — was typed as string; real Anthropic API returns null when no stop sequence triggers, causing SchemaDriftError on every normal response. Fixed to string-or-null.
  • AnthropicProvider.getProviderBalance() — was calling a non-existent endpoint (/v1/organizations/cost_report). Now returns unavailable with a message directing users to the Admin API, matching the Groq pattern.

Full changelog

See CHANGELOG.md for the complete entry.