refactor(tools): tighten platform tool schemas; storage-symmetric shape#127
refactor(tools): tighten platform tool schemas; storage-symmetric shape#127mgoldsborough merged 2 commits intomainfrom
Conversation
The LLM-facing JSON Schema is a contract. A loose `manifest: { type:
"object" }` invites the model to invent structure under-spec — saw this
in production conv_30076c3681ad4c91 where Sonnet serialized the manifest
as a JSON-string because the schema had no inner shape.
Apply four invariants across every platform tool that authors a
persistent thing:
1. MCP-native — built via `defineInProcessApp`, surfaces through
`tools/list` and `tools/call` byte-identical to a subprocess MCP
server. Same wire format external clients (Claude Code, Cursor)
consume.
2. Strict input schemas — every `object` declares `properties`; every
`array` declares `items`; identifiers use `pattern`; constrained
strings use `enum`; bounded numbers use `minimum`/`maximum`. Locked
by test/unit/tools/platform/schema-shape.test.ts.
3. Storage-symmetric shape — create/update tools use
`{ scope?, manifest, body }` where `manifest` mirrors the on-disk
metadata field-for-field and `body` is the content payload. No
`name` at root.
4. Minimum sufficient surface — operator/runtime-set fields
(`source`, `bundleName`, `allowedTools`, `requiresBundles`,
`overrides`, `derivedFrom`) live on the type, not in the LLM-
facing schema.
Per-tool changes:
- skills__create / update — full SkillManifest mirror; `name` moved
into manifest; metadata sub-schema for keywords/triggers/category/
tags. Dropped six advanced fields from the LLM input.
- instructions__write_instructions — renamed `text` → `body` for
cross-tool consistency.
- automations__create / update — restructured to {manifest, body};
`prompt` → `body`; manifest holds schedule + run-time policy. Dropped
`allowedTools`, `source`, `bundleName` from input.
- files__create — restructured to {manifest, body}; `base64_data` →
`body`; `mime_type` → `mimeType` (camelCase consistency).
No backward compat — clean break. Handlers cast directly to typed
`interface CreateInput` since the validator already enforced the schema
upstream. Deleted `src/skills/manifest-input.ts` (loose-shape coercion
is no longer needed; the schema is the contract).
Adds `src/tools/platform/CLAUDE.md` (with `AGENTS.md` symlink) — directory-
scoped authoring rules for new platform tools. Lint test
`schema-shape.test.ts` walks every source's tools/list and asserts the
shape invariants automatically.
Verify: bun run verify green (2129 unit, 213 web, 416 integration, 17
smoke).
eac07f9 to
295013a
Compare
QA review of #127 found three silent-breakage regressions sharing one root cause: the LLM-facing tool was treated as the only caller of the automation domain. The CLI (`nb automation pause/resume`) and bundle lifecycle were calling `automations__update` / `automations__create` with the old flat shape; AJV with strict:false accepted the call but the new handler reads `args.manifest`, never saw the fields, and silently no-op'd. Bundle install path lost `source: "bundle"` and `bundleName`, orphaning every bundle-contributed schedule on uninstall. Architecture fix — extract a domain layer: src/bundles/automations/src/domain.ts (NEW) createAutomation(input, ctx) updateAutomation(name, patch, ctx) deleteAutomation(name, ctx) Accepts the full Automation shape including operator-only fields (`source`, `bundleName`, `allowedTools`, `ownerId`, `workspaceId`). src/bundles/automations/src/server.ts Tool handlers become thin schema-translators that delegate to the domain. LLM-facing path stamps `source: "agent"`, derives ownership from request context, and never reaches operator fields. src/runtime/runtime.ts `registerAutomationsContext(getter)` / `getAutomationsContext()` — the source factory registers a workspace-scoped context getter during construction; internal callers read it back to bypass the LLM-facing surface. src/cli/commands/automation.ts (issue 1) pause/resume call updateAutomation directly — no schema gymnastics. src/bundles/lifecycle.ts (issues 2+3) syncBundleAutomations / removeBundleAutomations call domain. `source: "bundle"` and `bundleName` round-trip cleanly. Uninstall finds and cleans up bundle-contributed schedules. Plus follow-up issues from the same review: - test/unit/tools/platform/schema-shape.test.ts: register automations in SOURCES (issue 4) - executor.ts: recursion guard at the executor — the LLM can no longer set allowedTools, but operator file edits and bundle schedules still can; guard sees the merged Automation regardless of authoring path (issue 5) - validateAutomationFields(args: ValidatableAutomationFields) — typed signature replaces the synthetic-flat-record cast (issue 6) - skills__update + automations__update: separate *_UPDATE_MANIFEST_PROPERTIES omitting `name` so renames are rejected at schema validation rather than silently ignored (issue 7) - UPDATABLE_FIELDS now matches Automation type field order, comment explains why (issue 9) - CLAUDE.md § 1.4: "Internal callers use the domain API, not the tool handler" — codifies the convention so the next domain doesn't repeat this mistake New test coverage: - test/unit/bundles/automations/domain.test.ts — pause/resume regression - executor.test.ts — recursion-guard rejects, permits non-recursive Verify: bun run verify green (2151 unit, 213 web, 416 integration, 17 smoke).
QA review adjudication — fixed in 263fb05All 9 issues addressed. The three criticals shared one root cause: the LLM-facing tool was the only caller of the automation domain, so when we tightened its schema the CLI and lifecycle paths silently broke. Architectural fix — domain APIExtracted CLAUDE.md § 1.4 codifies the rule: internal callers use the domain API, not the tool handler. Per-issue resolution
Verify: 2151 unit (+9 new), 213 web, 416 integration, 17 smoke — all green. |
Summary
The LLM-facing JSON Schema is a contract. A loose
manifest: { type: "object" }invites the model to invent structure under-spec — saw this in productionconv_30076c3681ad4c91where Sonnet serialized the manifest as a JSON-string because the schema had no inner shape, then the validator rejected it, and the user saw "Couldn't create" with no signal what went wrong.This PR applies four invariants across every platform tool that authors a persistent thing, plus a directory-scoped CLAUDE.md (symlinked as AGENTS.md) so future contributors author against the convention rather than retrofit.
Four invariants
defineInProcessApp, surfaces throughtools/list/tools/callbyte-identical to a subprocess MCP server. Same wire format external clients (Claude Code, Cursor, Claude Desktop) consume.objectdeclaresproperties; everyarraydeclaresitems; identifiers usepattern; constrained strings useenum; bounded numbers useminimum/maximum. Locked bytest/unit/tools/platform/schema-shape.test.ts.{ scope?, manifest, body }wheremanifestmirrors the on-disk metadata field-for-field andbodyis the content payload. Nonameat root, no flat-config-at-root.source,bundleName,allowedTools,requiresBundles,overrides,derivedFrom) live on the type and the on-disk file, not in the LLM-facing schema.Per-tool changes
skills__create/updateSkillManifestmirror;namemoved into manifest; nestedmetadatafor keywords/triggers/category/tags; droppedallowedTools,requiresBundles,loadingStrategy,appliesToTools,overrides,derivedFromfrom LLM inputinstructions__write_instructionstext→bodyfor cross-tool consistencyautomations__create/update{manifest, body};prompt→body; manifest holds schedule + run-time policy; droppedallowedTools,source,bundleNamefrom inputfiles__create{manifest, body};base64_data→body;mime_type→mimeTypeDeleted
src/skills/manifest-input.ts— loose-shape coercion accepting kebab/snake/camel variants. The schema is the contract; one casing wins; the validator enforces it.Added
src/tools/platform/CLAUDE.md(withAGENTS.mdsymlink) — directive, ~200 lines, the four invariants and an "adding a new tool" checklist. Authoring rules for any future platform tool.test/unit/tools/platform/schema-shape.test.ts— walks every source'stools/list, recursively asserts everyobjecthaspropertiesand everyarrayhasitems. Locks invariants (1) and (2) against future regression. New sources register in the test'sSOURCESarray; new tools on existing sources are auto-detected.No backward compat
Clean break. Handlers cast directly to typed
interface CreateInput/UpdateInputsince the validator already enforced the schema upstream — no defensive re-validation, no kebab/snake/camel coercion, no flat-field plucking. The LLM rebuilds its tool understanding every conversation; there is no migration window to bridge.Test plan
bun run verifygreen: 2129 unit, 213 web, 416 integration, 17 smokeschema-shape.test.tspasses — locks (1) and (2) for skills, instructions, files, conversationsfiles__createfrom the agent — sameFiles
src/tools/platform/skills.ts— schema rewrite, handler simplification, types importsrc/tools/platform/instructions.ts— text → bodysrc/tools/platform/files.ts— schema rewrite, handler refactorsrc/bundles/automations/src/schemas.ts— full schema rewritesrc/bundles/automations/src/server.ts— handler rewrite, removedcontainsRecursiveTool(dead code;allowedToolsno longer reachable from input)src/skills/manifest-input.ts— DELETEDsrc/skills/index.ts— drop coercer exportsrc/tools/platform/CLAUDE.md— new convention docsrc/tools/platform/AGENTS.md— symlink to CLAUDE.mdtest/unit/tools/platform/schema-shape.test.ts— new linttest/**/*.test.ts— multiple test fixtures updated to new shapeweb/src/pages/settings/components/WorkspaceInstructions.tsx— body field rename