Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
cbc22a6
Implement parse validation step (CS-10713)
habdelra Apr 16, 2026
8851c12
Switch parse step from content-tag to glint (ember-tsc) for full type…
habdelra Apr 16, 2026
c343102
Merge remote-tracking branch 'origin/main' into cs-10713-implement-pa…
habdelra Apr 16, 2026
54c0962
Address PR review feedback
habdelra Apr 16, 2026
92481b5
Treat unreadable linked examples as instantiation failures
habdelra Apr 16, 2026
2c89364
Detect ember-tsc non-diagnostic exits as failures
habdelra Apr 16, 2026
41bc861
Use host node_modules for Ember/Glimmer type resolution in parse step
habdelra Apr 16, 2026
57ce078
Use loop iteration as validation artifact sequence number
habdelra Apr 16, 2026
d778f2a
Add glint type checking patterns to boxel-development skill
habdelra Apr 16, 2026
41600d0
Add more glint patterns to boxel-development skill
habdelra Apr 17, 2026
0cb65cf
Add strict mode template import rule to boxel-development skill
habdelra Apr 17, 2026
fb6b863
Add debug logging to Playwright parse tests for CI failure diagnosis
habdelra Apr 17, 2026
29deec3
Reset loader before eval and instantiate host commands
habdelra Apr 17, 2026
e87b445
Fix prettier formatting in instantiate-card host command
habdelra Apr 17, 2026
f4ab358
Fix outdated cardInfo field names in boxel-development skill
habdelra Apr 17, 2026
6fd137d
Fix CI Playwright test failures by scoping to test-written files
habdelra Apr 17, 2026
e578898
Fix false positive when base package has pre-existing glint errors
habdelra Apr 17, 2026
d6a5a29
Simplify test card to avoid BaseDef constraint error in CI
habdelra Apr 17, 2026
ed288f0
Show .json extension on JSON file names in ParseResult card
habdelra Apr 17, 2026
e4d1b7c
Fix prettier formatting
habdelra Apr 17, 2026
dc9ab2f
Remove debug logging from Playwright parse tests
habdelra Apr 17, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions packages/host/app/commands/evaluate-module.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,15 @@ export default class EvaluateModuleCommand extends HostBaseCommand<
let commandModule = await this.loadCommandModule();

try {
// Reset the loader to clear cached modules from prior eval runs.
// Without this, the Loader's internal module Map retains the old
// compiled bytecode and `loader.import()` returns the cached
// (stale) result — so edits the factory agent makes between
// validation turns are invisible to the eval step.
this.loaderService.resetLoader({
clearFetchCache: true,
reason: 'evaluate-module: fresh eval requires uncached loader',
});
let loader = this.loaderService.loader;
await loader.import(moduleUrl);

Expand Down
10 changes: 10 additions & 0 deletions packages/host/app/commands/instantiate-card.ts
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,16 @@ export default class InstantiateCardCommand extends HostBaseCommand<
let commandModule = await this.loadCommandModule();

try {
// Reset the loader to clear cached modules from prior runs.
// Without this, the Loader's internal module Map retains stale
// compiled bytecode — edits the factory agent makes between
// validation turns are invisible to instantiation.
this.loaderService.resetLoader({
clearFetchCache: true,
reason:
'instantiate-card: fresh instantiation requires uncached loader',
});

// Build or parse the card document
let doc;
if (input.instanceData) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -94,45 +94,50 @@ For computed fields, ask: "Am I keeping this simple and unidirectional?"

**Every CardDef inherits:**

- `title`, `description`, `thumbnailURL`
- `cardTitle`, `cardDescription`, `cardThumbnailURL`

### Inherited Fields and CardInfo

**IMPORTANT:** Every CardDef automatically inherits these base fields from the CardDef base class:
**IMPORTANT:** Every CardDef automatically inherits these base fields from the CardDef base class (defined in `packages/base/card-api.gts`):

#### Direct Inherited Fields (Read-Only)
#### CardInfoField (FieldDef — user-editable)

- `title` (StringField) - Computed pass-through from `cardInfo.title`
- `description` (StringField) - Computed pass-through from `cardInfo.description`
- `thumbnailURL` (StringField) - Computed pass-through from `cardInfo.thumbnailURL`
`CardInfoField` is a `FieldDef` with these fields:

#### CardInfo Field (User-Editable)
- `cardInfo.name` (StringField) — user-editable card name
- `cardInfo.summary` (StringField) — user-editable card summary
- `cardInfo.cardThumbnail` (linksTo ImageDef) — linked thumbnail image card
- `cardInfo.cardThumbnailURL` (MaybeBase64Field) — thumbnail URL or base64 string
- `cardInfo.theme` (linksTo Theme) — optional theme card link
- `cardInfo.notes` (MarkdownField) — optional internal notes

Every card also inherits a `cardInfo` field which contains the actual user-editable values:
#### Computed pass-through fields on CardDef (read-only)

- `cardInfo.title` (StringField) - User-editable card title
- `cardInfo.description` (StringField) - User-editable card description
- `cardInfo.thumbnailURL` (StringField) - User-editable thumbnail image URL
- `cardInfo.theme` (linksTo ThemeCard) - Optional theme card link
- `cardInfo.notes` (MarkdownField) - Optional internal notes
These are computed fields that read from `cardInfo`:

- `cardTitle` (StringField) — computed from `cardInfo.name`, falls back to `Untitled {displayName}`
- `cardDescription` (StringField) — computed from `cardInfo.summary`
- `cardThumbnailURL` (MaybeBase64Field) — computed from `cardInfo.cardThumbnailURL`

**How It Works:**
The top-level `title`, `description`, and `thumbnailURL` fields are computed properties that automatically pass through the values from `cardInfo.title`, `cardInfo.description`, and `cardInfo.thumbnailURL` respectively. This means:
The top-level `cardTitle`, `cardDescription`, and `cardThumbnailURL` fields are computed properties that pass through values from `cardInfo`. Users edit values through the `cardInfo` field in edit mode.

- When you read `@model.title` in templates, you get the value from `cardInfo.title`
- Users edit values through the `cardInfo` field in edit mode
- Override to add custom logic that respects user input
- `@model.cardTitle` reads `cardInfo.name` (with fallback)
- `@model.cardDescription` reads `cardInfo.summary`
- Override these computed fields to add custom logic

**Best Practice:** Define your own primary field and compute `title` to respect user's `cardInfo.title` choice:
**Best Practice:** Define your own primary field and compute `cardTitle` to respect user's `cardInfo.name` choice:

```gts
export class BlogPost extends CardDef {
@field headline = contains(StringField); // Your primary field

// Override inherited title - respects user's cardInfo.title if set
@field title = contains(StringField, {
computeVia: function () {
return this.cardInfo?.title ?? this.headline ?? 'Untitled';
// Override inherited cardTitle - respects user's cardInfo.name if set
@field cardTitle = contains(StringField, {
computeVia: function (this: BlogPost): string {
return this.cardInfo?.name?.trim()?.length
? this.cardInfo.name
: (this.headline ?? 'Untitled');
},
});
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,86 @@ Before generating ANY code:
- [ ] No JavaScript operations in templates
- [ ] ALL THREE FORMATS: isolated, embedded, fitted

### Glint (ember-tsc) Type Checking Patterns

The factory runs `ember-tsc` (glint) on all `.gts` and `.ts` files to catch type errors. These patterns avoid common glint failures:

#### Decorators inside inline class assignments

Glint does not support decorators (`@tracked`, etc.) on fields inside an inline class expression assigned to a static property. Declare the component class separately:

```gts
// ❌ WRONG — "Decorators are not valid here"
export class StickyNote extends CardDef {
static isolated = class Isolated extends Component<typeof StickyNote> {
@tracked editMode = false; // glint error!
<template>...</template>
};
}

// ✅ CORRECT — declare the class outside the assignment
class Isolated extends Component<typeof StickyNote> {
@tracked editMode = false;
<template>...</template>
}

export class StickyNote extends CardDef {
static isolated = Isolated;
}
```

Note: `@field` decorators on `CardDef`/`FieldDef` classes work fine — this restriction only applies to component classes using `@tracked` or similar decorators.

#### Typing dynamic imports in test files

When test files use `loader.import()`, the return type is `{}` by default. Destructuring a named export from it causes "Property does not exist on type '{}'":

```gts
// ❌ WRONG — "Property 'StickyNote' does not exist on type '{}'"
let { StickyNote } = await loader.import(cardModuleUrl);

// ✅ CORRECT — cast the import result
let { StickyNote } = (await loader.import(cardModuleUrl)) as Record<string, any>;
```

#### Accessing cardInfo properties in computeVia

`CardDef.cardInfo` is a `CardInfoField` (FieldDef) with these fields: `name`, `summary`, `cardThumbnailURL`, `cardThumbnail`, `theme`, `notes`. Access them directly — they are properly typed:

```gts
// ✅ CORRECT — access cardInfo fields directly (they are typed)
@field cardTitle = contains(StringField, {
computeVia: function (this: MyCard): string {
return this.cardInfo?.name?.trim()?.length
? this.cardInfo.name
: this.headline ?? 'Untitled';
},
});

// ❌ WRONG — these fields don't exist on CardInfoField
this.cardInfo.title // use .name instead
this.cardInfo.description // use .summary instead
this.cardInfo.thumbnailURL // use .cardThumbnailURL instead
```

**Note:** The computed pass-through fields on CardDef are named `cardTitle` (not `title`), `cardDescription` (not `description`), and `cardThumbnailURL`. Override these — not fields named `title`/`description`.

#### Explicit types for function parameters

Glint enforces strict mode. Always type function parameters and return values:

```gts
// ❌ WRONG — implicit any
greet = (name) => `Hello, ${name}!`;

// ✅ CORRECT
greet = (name: string): string => `Hello, ${name}!`;
```

#### Unused imports from Ember shims

If you import from `@ember/helper`, `@ember/modifier`, or `@glimmer/tracking`, only import what you actually use. Glint enforces `noUnusedLocals` in the factory's type checking configuration. Remove unused imports rather than suppressing the error.

### Common Mistakes

#### Using contains with CardDef
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,34 @@
### Template Essentials

#### ⚠️ CRITICAL: Strict Mode — Import Every Helper and Modifier

Boxel `.gts` templates run in **strict mode**. Every helper, modifier, and component used in a `<template>` tag must be explicitly imported at the top of the file. Unlike loose-mode Ember templates, there are no globals — if it's not imported, you get "Attempted to resolve a helper in a strict mode template, but that value was not in scope."

```gts
// ❌ WRONG — "eq" is not in scope
<template>{{#if (eq @model.status 'active')}}Active{{/if}}</template>

// ✅ CORRECT — import every helper you use
import { eq } from '@cardstack/boxel-ui/helpers';
import { fn } from '@ember/helper';
import { on } from '@ember/modifier';

// Now these are in scope for the template:
<template>
{{#if (eq @model.status 'active')}}Active{{/if}}
<button {{on 'click' (fn this.handleClick @model.id)}}>Click</button>
</template>
```

**Common helper imports:**

| Helper/Modifier | Import from |
| ------------------------------------------------------- | ----------------------------- |
| `eq`, `not`, `or`, `and`, `gt`, `lt`, `subtract`, `add` | `@cardstack/boxel-ui/helpers` |
| `fn`, `get`, `concat`, `array`, `hash` | `@ember/helper` |
| `on` | `@ember/modifier` |
| `tracked` | `@glimmer/tracking` |

**Field access patterns:**

```hbs
Expand Down
50 changes: 40 additions & 10 deletions packages/software-factory/docs/phase-2-plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,20 +84,21 @@ interface ValidationStepRunner {
- **Lint step** (CS-10714): `{ lintResultId, filesChecked, filesWithErrors, totalViolations, violations: [{ rule, file, line, message }] }` — calls the realm's `_lint` endpoint (ESLint + Prettier + `@cardstack/boxel` rules) for each `.gts`, `.gjs`, `.ts`, `.js` file. Creates a `LintResult` card as a persistent artifact.
- **Eval step** (CS-10715): `{ evalResultId, modulesChecked, modulesWithErrors, modules: [{ path, error, stackTrace? }] }` — evaluates each non-test `.gts` module via `_run-command` → `evaluate-module` host command → `/_prerender-module` (prerenderer sandbox). Creates an `EvalResult` card as a persistent artifact. Files matching `*.test.gts` are excluded.
- **Instantiate step** (CS-10716): `{ instantiateResultId, cardsChecked, cardsWithErrors, cards: [{ specId, cardName, error, stackTrace? }] }` — discovers Spec cards in the realm, resolves each spec's `ref` to a card definition module, reads `linkedExamples` entries as instance data, and instantiates via `_run-command` → `instantiate-card` host command → `store.__dangerousCreateFromSerialized(...)` (prerenderer sandbox) so `Field.validate()` failures surface during instantiation. Creates an `InstantiateResult` card as a persistent artifact. Field specs (`specType: 'field'`) are excluded.
- **Future parse step**: defines its own `details` shape
- **Parse step** (CS-10713): `{ parseResultId, filesChecked, filesWithErrors, totalErrors, errors: [{ file, line, message }] }` — validates `.gts`/`.ts` files by running `ember-tsc --noEmit` (glint) for template-aware TypeScript type checking, and validates `.json` card instances via structural validation (JSON syntax + card document shape). JSON validation runs against spec `linkedExamples` — same discovery as the instantiate step. Creates a `ParseResult` card as a persistent artifact.

**Adding a new validation step** = creating a new module file in `src/validators/` + replacing the `NoOpStepRunner` in `createDefaultPipeline()`.
**Adding a new validation step** = creating a new module file in `src/validators/` + wiring it into `createDefaultPipeline()`.

### Validation Artifacts: Naming and Storage

All validation artifacts (test runs, lint results, future validation types) are stored in a shared `Validations/` directory in the target realm with type-prefixed names:

- Parse results: `Validations/parse_{issue-slug}-{seq}.json` (e.g., `Validations/parse_sticky-note-define-core-1.json`)
- Test runs: `Validations/test_{issue-slug}-{seq}.json` (e.g., `Validations/test_sticky-note-define-core-1.json`)
- Lint results: `Validations/lint_{issue-slug}-{seq}.json` (e.g., `Validations/lint_sticky-note-define-core-1.json`)
- Eval results: `Validations/eval_{issue-slug}-{seq}.json` (e.g., `Validations/eval_sticky-note-define-core-1.json`)
- Instantiate results: `Validations/instantiate_{issue-slug}-{seq}.json` (e.g., `Validations/instantiate_sticky-note-define-core-1.json`)

Each artifact is a card instance (`TestRun`, `LintResult`, `EvalResult`, or `InstantiateResult`) with `linksTo` relationships to the `Issue` and `Project` being validated.
Each artifact is a card instance (`ParseResult`, `TestRun`, `LintResult`, `EvalResult`, or `InstantiateResult`) with `linksTo` relationships to the `Issue` and `Project` being validated.

### Validation Context Flow

Expand All @@ -110,6 +111,35 @@ Each artifact is a card instance (`TestRun`, `LintResult`, `EvalResult`, or `Ins

The Phase 1 `testResults` field on `AgentContext` is deprecated. All validation flows through `validationResults` (for the loop) and `validationContext` (for the LLM prompt).

### Parse Step Details (CS-10713)

The parse validation step (`src/validators/parse-step.ts`) verifies that `.gts`, `.ts`, and `.json` files are valid. It replaces the `NoOpStepRunner('parse')` placeholder in the default pipeline. For `.gts`/`.ts` files it uses glint (`ember-tsc`) for full template-aware TypeScript type checking. For `.json` files it validates card document structure.

**GTS/TS validation uses glint (ember-tsc):**

The step downloads realm `.gts` and `.ts` files to a temp directory, writes a tsconfig.json (mirroring `realm/tsconfig.json` with absolute paths to `packages/base`), symlinks the software-factory `node_modules` (so glint's internal `@glint/ember-tsc/-private/dsl` module resolves), and runs `ember-tsc --noEmit`. This catches:

- TypeScript type errors (type mismatches, missing properties, bad assignments)
- Template errors (invalid component args, missing helpers, malformed template expressions)
- Syntax errors (missing brackets, unterminated strings, malformed type annotations)

**Filtering:** The base package has pre-existing type errors (e.g., `<style scoped>` not in HTML type definitions, missing `@ember/*` module declarations). The step filters output to only errors from the temp directory and suppresses known false positives: TS2353 for `'scoped'` on `<style>` elements (Ember's `<style scoped>` is valid but not in the HTML type definitions).

**Test files excluded:** Files matching `*.test.gts` or `*.test.ts` are excluded — test files require QUnit and `@universal-ember/test-support` type declarations that aren't available in the parse step's isolated temp directory. Test file correctness is the test validation step's responsibility. `.js` files are also excluded because lint (ESLint) already validates JavaScript syntax and the factory agent does not generate `.js` files.

**JSON validation uses spec-based discovery** — the same mechanism as the instantiate step. The step searches the realm for Spec cards and extracts their `linkedExamples` URLs, then reads each example instance and validates:

1. JSON syntax via `JSON.parse()` (when reading raw content from mocks or non-realm sources)
2. Card document structure: presence of `data` object, `data.type` string, `data.meta.adoptsFrom` with `module` and `name`

When `readFile` returns a parsed `document` (as the realm API does for `.json` files), JSON syntax is already validated — only the structural check runs. When the realm enriches the document during indexing, the structural check validates the enriched version, which may pass even if the raw source was incomplete. This is intentional: if the realm accepted and indexed the card, it is valid from the realm's perspective.

**Bootstrap behavior:** When no `.gts`/`.ts` files exist and no Spec cards are found (bootstrap scenario), the step returns `passed: true` with no files checked and no artifact created. This matches the design principle: "nothing to validate is a pass."

**Performance:** The tsconfig content is cached in memory (it never changes between runs). The `node_modules` symlink avoids copying hundreds of megabytes of dependencies.

The `ParseResult` card definition (`realm/parse-result.gts`) and CRUD (`src/parse-result-cards.ts`) follow the same patterns as `LintResult` and `EvalResult` — fitted/embedded/isolated templates, a running state, `ParseFileResult` field def with nested `ParseError` entries, and links to Issue/Project.

### Lint Step Details (CS-10714)

The lint validation step (`src/validators/lint-step.ts`) uses the realm's existing `_lint` endpoint — the same one the Monaco editor uses in code mode. For each lintable file discovered in the realm:
Expand Down Expand Up @@ -190,13 +220,13 @@ The agent does **not** need to create "run tests" issues. Test execution happens

Bootstrap issues (the seed issue that creates Project, KnowledgeArticles, and implementation issues) produce no testable code artifacts — only JSON card instances. Validation still runs after every inner-loop iteration, but each step gracefully handles "nothing to validate":

| Step | Bootstrap behavior |
| ---------------------- | ----------------------------------------------------------- |
| **Parse** | Checks created `.json` files are valid — useful |
| **Lint** | No-op for JSON card instances — pass |
| **Module evaluation** | No `.gts` modules created — no-op, pass |
| **Card instantiation** | No `.gts` modules or Spec cards — pass, no artifact created |
| **Run tests** | No test files exist yet — vacuous pass |
| Step | Bootstrap behavior |
| ---------------------- | ------------------------------------------------------------------------------- |
| **Parse** | Checks `.gts` syntax + `.json` spec examples — pass if no specs or `.gts` files |
| **Lint** | No-op for JSON card instances — pass |
| **Module evaluation** | No `.gts` modules created — no-op, pass |
| **Card instantiation** | No `.gts` modules or Spec cards — pass, no artifact created |
| **Run tests** | No test files exist yet — vacuous pass |

**Design principle**: No special-casing per issue type. Each validation step returns `passed: true` with an empty errors array when there is nothing to validate. "Nothing to validate" is a pass, not an error.

Expand Down
Loading
Loading