From cbc22a6b62d6e61806ecbba2a85e0faf5af13b02 Mon Sep 17 00:00:00 2001 From: Hassan Abdel-Rahman Date: Thu, 16 Apr 2026 12:08:56 -0400 Subject: [PATCH 01/20] Implement parse validation step (CS-10713) Replace the NoOpStepRunner('parse') placeholder with a real ParseValidationStep that validates .gts files via content-tag preprocessing + TypeScript syntax checking, and .json card instances via structural validation (JSON syntax + card document shape) against spec linkedExamples. - realm/parse-result.gts: ParseResult card definition with ParseFileResult and ParseError field defs - src/parse-result-cards.ts: CRUD helpers (create, complete, build) - src/validators/parse-step.ts: ParseValidationStep implementation - Wire into createDefaultPipeline() and factory-issue-loop-wiring - Unit tests (21 cases) and Playwright E2E tests (3 cases) - Update phase-2-plan.md with Parse Step Details section Co-Authored-By: Claude Opus 4.6 (1M context) --- .../software-factory/docs/phase-2-plan.md | 43 +- .../software-factory/realm/parse-result.gts | 504 ++++++++++++ .../scripts/smoke-tests/smoke-test-realm.ts | 5 + .../src/factory-issue-loop-wiring.ts | 5 + .../src/parse-result-cards.ts | 182 +++++ .../src/validators/parse-step.ts | 760 ++++++++++++++++++ .../src/validators/validation-pipeline.ts | 23 +- packages/software-factory/tests/index.ts | 1 + .../software-factory/tests/parse-step.test.ts | 663 +++++++++++++++ .../tests/parse-validation.spec.ts | 282 +++++++ .../tests/validation-pipeline.test.ts | 6 +- 11 files changed, 2458 insertions(+), 16 deletions(-) create mode 100644 packages/software-factory/realm/parse-result.gts create mode 100644 packages/software-factory/src/parse-result-cards.ts create mode 100644 packages/software-factory/src/validators/parse-step.ts create mode 100644 packages/software-factory/tests/parse-step.test.ts create mode 100644 packages/software-factory/tests/parse-validation.spec.ts diff --git a/packages/software-factory/docs/phase-2-plan.md b/packages/software-factory/docs/phase-2-plan.md index 40da7b20ef..bd7bbd1e6a 100644 --- a/packages/software-factory/docs/phase-2-plan.md +++ b/packages/software-factory/docs/phase-2-plan.md @@ -84,20 +84,21 @@ interface ValidationStepRunner { - **Lint step** (CS-10714): `{ lintResultId, filesChecked, filesWithErrors, totalViolations, violations: [{ rule, file, line, message }] }` — calls the realm's `_lint` endpoint (ESLint + Prettier + `@cardstack/boxel` rules) for each `.gts`, `.gjs`, `.ts`, `.js` file. Creates a `LintResult` card as a persistent artifact. - **Eval step** (CS-10715): `{ evalResultId, modulesChecked, modulesWithErrors, modules: [{ path, error, stackTrace? }] }` — evaluates each non-test `.gts` module via `_run-command` → `evaluate-module` host command → `/_prerender-module` (prerenderer sandbox). Creates an `EvalResult` card as a persistent artifact. Files matching `*.test.gts` are excluded. - **Instantiate step** (CS-10716): `{ instantiateResultId, cardsChecked, cardsWithErrors, cards: [{ specId, cardName, error, stackTrace? }] }` — discovers Spec cards in the realm, resolves each spec's `ref` to a card definition module, reads `linkedExamples` entries as instance data, and instantiates via `_run-command` → `instantiate-card` host command → `store.__dangerousCreateFromSerialized(...)` (prerenderer sandbox) so `Field.validate()` failures surface during instantiation. Creates an `InstantiateResult` card as a persistent artifact. Field specs (`specType: 'field'`) are excluded. -- **Future parse step**: defines its own `details` shape +- **Parse step** (CS-10713): `{ parseResultId, filesChecked, filesWithErrors, totalErrors, errors: [{ file, line, message }] }` — validates `.gts` files via two-phase parsing (content-tag preprocessing + TypeScript syntax checking) and `.json` card instances via structural validation (JSON syntax + card document shape). JSON validation runs against spec `linkedExamples` — same discovery as the instantiate step. Creates a `ParseResult` card as a persistent artifact. -**Adding a new validation step** = creating a new module file in `src/validators/` + replacing the `NoOpStepRunner` in `createDefaultPipeline()`. +**Adding a new validation step** = creating a new module file in `src/validators/` + wiring it into `createDefaultPipeline()`. ### Validation Artifacts: Naming and Storage All validation artifacts (test runs, lint results, future validation types) are stored in a shared `Validations/` directory in the target realm with type-prefixed names: +- Parse results: `Validations/parse_{issue-slug}-{seq}.json` (e.g., `Validations/parse_sticky-note-define-core-1.json`) - Test runs: `Validations/test_{issue-slug}-{seq}.json` (e.g., `Validations/test_sticky-note-define-core-1.json`) - Lint results: `Validations/lint_{issue-slug}-{seq}.json` (e.g., `Validations/lint_sticky-note-define-core-1.json`) - Eval results: `Validations/eval_{issue-slug}-{seq}.json` (e.g., `Validations/eval_sticky-note-define-core-1.json`) - Instantiate results: `Validations/instantiate_{issue-slug}-{seq}.json` (e.g., `Validations/instantiate_sticky-note-define-core-1.json`) -Each artifact is a card instance (`TestRun`, `LintResult`, `EvalResult`, or `InstantiateResult`) with `linksTo` relationships to the `Issue` and `Project` being validated. +Each artifact is a card instance (`ParseResult`, `TestRun`, `LintResult`, `EvalResult`, or `InstantiateResult`) with `linksTo` relationships to the `Issue` and `Project` being validated. ### Validation Context Flow @@ -110,6 +111,28 @@ Each artifact is a card instance (`TestRun`, `LintResult`, `EvalResult`, or `Ins The Phase 1 `testResults` field on `AgentContext` is deprecated. All validation flows through `validationResults` (for the loop) and `validationContext` (for the LLM prompt). +### Parse Step Details (CS-10713) + +The parse validation step (`src/validators/parse-step.ts`) verifies that `.gts` and `.json` files are syntactically valid. It replaces the `NoOpStepRunner('parse')` placeholder in the default pipeline. Unlike lint (which uses the realm's `_lint` endpoint) or eval/instantiate (which use `_run-command` through the prerenderer), the parse step runs entirely in the factory's Node process using `content-tag` and TypeScript's parser — no realm server calls beyond file reads. + +**GTS validation is two-phase:** + +1. **content-tag preprocessing** — `Preprocessor.process(source, { filename })` transforms GTS → TS by replacing `