Markdown format: full rendering pipeline and tooling (CS-10780 → CS-10798) by lukemelia · Pull Request #4412 · cardstack/boxel

lukemelia · 2026-04-15T16:05:35Z

Landing the complete Markdown Format project on a single branch. Each commit is self-contained.

Issues

CS-10780 — Register markdown in the Format type and formats array
CS-10781 — Whitespace-preserving render container for markdown format
CS-10782 — Prerender pipeline: capture textContent for markdown format
CS-10783 — Add markdownEscape template helper
CS-10784 — HTML-to-markdown fallback on CardDef/FieldDef/FileDef
CS-10785 — Default static markdown templates for primitive fields
CS-10786 — Explicit static markdown templates for specialized fields
CS-10787 — Explicit static markdown templates for domain fields
CS-10789 — E2E tests for the markdown rendering pipeline
CS-10790 — Markdown preview panel (source view)
CS-10791 — Markdown preview panel (rendered view)
CS-10794 — "Copy as Markdown" in isolated view context menu
CS-10796 — Persist markdown format in boxel_index
CS-10797 — Markdown helpers for card links and BFM card embeds
CS-10798 — Serve card markdown via HTTP endpoint

Commits

1. CS-10780 — Register `markdown` in Format

Add 'markdown' to the Format type union and the formats array in packages/runtime-common/formats.ts
Audited every switch on Format; the two with meaningful semantics (defaultFieldFormats and captureModeForFormat) both have default branches that absorbed the new variant cleanly

2. CS-10781 — Whitespace-preserving container

packages/host/app/templates/render/html.gts: wrap markdown-format renders in <div data-markdown-render-container> with white-space: pre in a scoped <style> block
packages/base/field-component.gts: defaultFieldFormats now returns { fieldDef: 'markdown', cardDef: 'markdown' } when the containing format is markdown, so <@fields.x /> delegation composes markdown output from children

3. CS-10782 — Prerender pipeline extraction

packages/realm-server/prerender/utils.ts: renderHTML now selects textContent as the capture mode for markdown, skips cleanCapturedHTML. captureResult's textContent branch prefers [data-markdown-output] → [data-markdown-render-container] → resolved element

4. CS-10783 — `markdownEscape` template helper

New helper that emits CommonMark backslash escapes for all metacharacters, plus a line-start pass for numeric ordered-list prefixes
Returns a plain string (not SafeString) so Glimmer HTML-escapes it in the DOM; the textContent capture decodes the entities before the markdown parser sees the output
Unit tests cover every character class

5. CS-10784 — HTML-to-markdown fallback on `CardDef`/`FieldDef`/`FileDef`

DefaultMarkdownFallbackTemplate renders the HTML fallback format into a hidden [data-markdown-fallback-source] div, converts via globalThis.__boxelHtmlToMarkdown, emits into [data-markdown-output]
Browser-only turndown + @joplin/turndown-plugin-gfm singleton with custom rules for card containers (:card[id] / ::card[id]), compact links, and style/script stripping
MutationObserver re-converts when the source subtree mutates (async data loads, linked cards resolve)
Server-isolation test asserts no turndown dependencies leak into Node packages

6. CS-10785 — Default `static markdown` templates for primitive fields

StringField, ReadOnlyField, NumberField → markdownEscape
TextAreaField → escape per line + hard breaks
MarkdownField → raw passthrough
CSSField → fenced ```css block with adaptive fence width
MaybeBase64Field → [binary content] for data: URIs

7. CS-10786 — Explicit templates for specialized fields

BooleanField, CodeRefField, DateField, DatetimeField, BigIntegerField, EthereumAddressField

8. CS-10787 — Explicit templates for domain fields

Domain-specific fields with richer markdown representations

9. CS-10796 — Persist markdown in `boxel_index`

DB migration adds markdown column to boxel_index
Card indexer stores prerendered markdown during indexing
IndexQueryEngine returns markdown in search results

10. CS-10798 — Serve card markdown via HTTP endpoint

HTTP handler serves prerendered markdown with content-type: text/markdown

11. CS-10789 — E2E tests for the markdown rendering pipeline

End-to-end tests verifying prerender → index → serve round-trip

12. CS-10790/CS-10791 — Markdown preview panel

MarkdownPreview component with source/rendered toggle
Source view captures raw markdown text from the card's static markdown template
Rendered view parses markdown to HTML via marked, renders embedded :card / ::card directives via RenderedMarkdown component
Embedded cards are clickable for navigation

13. CS-10794 — "Copy as Markdown" command

CopyCardAsMarkdownCommand fetches markdown from the realm server endpoint and copies to clipboard
Added to isolated view context menu

14. CS-10797 — Markdown helpers for card links and BFM card embeds

markdownLinkForCard(card, text?) — single card link [title](id)
markdownLinksForCards(cards, { style }) — list or inline card links
markdownEmbedForCard(card, { kind, size }) — :card[id] or ::card[id | size]
markdownEmbedsForCards(cards, { separator }) — multiple card embeds
Custom static markdown templates on RatingsSummary and BlogPost in experiments-realm exercising all four helpers

15. Playground + polish

Added 'markdown' to the field playground format chooser
Widened playground container to accommodate the extra button
Wrapped MarkdownPreview in CardContainer for proper white background
Various lint and test fixes across the branch

Verification

pnpm lint:types passes for host, runtime-common, realm-server, boxel-ui
pnpm lint:js + pnpm lint:hbs clean
Host integration tests pass (markdown-fallback, markdown-preview, rendered-markdown, field-playground, realm-indexing)
Unit tests pass (markdown-escape, markdown-helpers)
CI green

🤖 Generated with Claude Code

Add `markdown` to the `Format` type union and the `formats` array so downstream format resolution (via `cardOrField[effectiveFormat]` in `getBoxComponent()`), the format chooser UI, and prerender routes can pick it up automatically. Foundational unblocker for the Markdown Format project (CS-10780). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

github-actions · 2026-04-15T16:27:47Z

Realm Server Test Results

1 files ± 0 1 suites ±0 15m 34s ⏱️ + 1m 35s
912 tests +60 912 ✅ +60 0 💤 ±0 0 ❌ ±0
984 runs +61 984 ✅ +61 0 💤 ±0 0 ❌ ±0

Results for commit 1f0eceb. ± Comparison against base commit 5d53272.

♻️ This comment has been updated with latest results.

github-actions · 2026-04-15T16:31:24Z

Host Test Results

1 files ± 0 1 suites ±0 2h 31m 55s ⏱️ -14s
2 269 tests +23 2 253 ✅ +22 15 💤 ±0 0 ❌ ±0 1 🔥 +1
2 269 runs +23 2 252 ✅ +21 15 💤 ±0 1 ❌ +1 1 🔥 +1

For more details on these errors, see this check.

Results for commit 1f0eceb. ± Comparison against base commit 5d53272.

This pull request removes 107 and adds 130 tests. Note that renamed tests count towards both.

Chrome ‑ Integration | RichMarkdownField: switching from Preview back to Compose restores editor
Chrome ‑ Unit | Service | render-service: CardStoreWithErrors resolves prefix ids before loading card documents
Chrome ‑ Unit | Service | render-service: CardStoreWithErrors resolves prefix ids before loading file-meta documents
Chrome ‑ Unit | Service | render-service: CardStoreWithErrors resolves registered prefix ids for card and file lookups
Chrome ‑ Unit | Utility | field-path-parser > applyFieldUpdate: should append to containsMany arrays
Chrome ‑ Unit | Utility | field-path-parser > applyFieldUpdate: should append to linksToMany arrays
Chrome ‑ Unit | Utility | field-path-parser > applyFieldUpdate: should create nested objects if needed
Chrome ‑ Unit | Utility | field-path-parser > applyFieldUpdate: should extend containsMany arrays with null padding
Chrome ‑ Unit | Utility | field-path-parser > applyFieldUpdate: should update containsMany element with index
Chrome ‑ Unit | Utility | field-path-parser > applyFieldUpdate: should update containsMany elements
…

Chrome ‑ Integration | Command | host command schema generation test > command schema generation: getInputJsonSchema for CopyCardAsMarkdownCommand
Chrome ‑ Integration | Command | host command schema generation test > command schema generation: getInputJsonSchema for EvaluateModuleCommand
Chrome ‑ Integration | Command | host command schema generation test > command schema generation: getInputJsonSchema for InstantiateCardCommand
Chrome ‑ Integration | commands | evaluate-module: module with broken import fails evaluation
Chrome ‑ Integration | commands | evaluate-module: valid module passes evaluation
Chrome ‑ Integration | commands | instantiate-card: containsMany field with non-array value fails instantiation
Chrome ‑ Integration | commands | instantiate-card: valid card with instance data passes instantiation
Chrome ‑ Integration | commands | instantiate-card: valid card with no instance data passes instantiation
Chrome ‑ Integration | field markdown domain: BrandFunctionalPalette markdown emits empty string when nothing is populated
Chrome ‑ Integration | field markdown domain: BrandFunctionalPalette markdown lists populated palette colors
…

♻️ This comment has been updated with latest results.

…781) Wrap markdown-format renders in a `<div data-markdown-render-container>` with `white-space: pre` in the prerender render route template. The dedicated data attribute gives downstream extraction a tight target so surrounding route-template whitespace does not leak into the captured markdown string, and `white-space: pre` keeps authored newlines and indentation intact. Also teach `defaultFieldFormats` to recurse in `markdown` when the containing format is `markdown` so `<@fields.x />` delegation inside a markdown template composes markdown output from its children, not embedded/fitted HTML. Other formats are unaffected: non-markdown renders still use the existing `<@model.Component>` invocation with no wrapper. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Teach the headless-browser prerender capture to handle the `markdown` format end-to-end on the extraction side: * `renderHTML` selects `textContent` as the capture mode for `markdown` (instead of `innerHTML`/`outerHTML`), and skips `cleanCapturedHTML` on the result since the cleanup regex only makes sense for HTML. * `captureResult`'s `textContent` branch now prefers the `[data-markdown-render-container]` element (CS-10781) when present, so the extracted string is the raw markdown body without surrounding route-template whitespace. The existing `renderMeta` caller also uses `textContent` mode; its markup has no markdown container, so the fallback to `resolvedElement.textContent` keeps that flow intact. Note: this lands the rendering-pipeline half of CS-10782. HTTP serving with `content-type: text/markdown`, storage in `boxel_index`, and the end-to-end integration test require a DB migration + new HTTP route, which will land in follow-ups. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

github-actions · 2026-04-15T16:50:54Z

Preview deployments

Exports `markdownEscape` from `@cardstack/boxel-ui/helpers` so card authors can safely interpolate user-supplied content into a `static markdown` template without accidentally triggering formatting. The helper emits CommonMark backslash escapes for all ASCII-punctuation metacharacters (`* _ \` [ ] ( ) < > | ~ ! # + -` and `\\` itself), plus a line-start pass that escapes numeric ordered-list prefixes like `1.`. Null/undefined inputs return `''`; non-string inputs are coerced via `String()`. The return value is a plain string (not a SafeString) so Glimmer HTML-escapes it in the DOM, the CS-10782 textContent capture decodes the entities, and the markdown parser sees the literal backslash escapes. Unit tests in `packages/boxel-ui/test-app/tests/unit/markdown-escape-test.ts` cover every character class called out by the Linear acceptance criteria plus null/undefined/non-string/empty-string inputs. Note: following the existing boxel-ui helper convention, this is an explicit-import helper (the codebase has no auto-scope mechanism for card templates); the Linear issue's "without explicit import" phrasing is reconciled to match the codebase pattern used by `eq`, `cn`, etc. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Default `static markdown` template that renders the HTML fallback (`isolated` for cards, `embedded` for fields) into a hidden source container and converts its innerHTML to markdown via turndown + @joplin/turndown-plugin-gfm. The converter is installed on `globalThis.__boxelHtmlToMarkdown` from the host bundle, so `packages/base` has zero dependency on turndown and the realm-server import graph stays clean (enforced by a grep-based isolation test). Subclass overrides of `static markdown` still win via the existing format-resolver bracket-notation lookup. The prerender pipeline now prefers `[data-markdown-output]` over `[data-markdown-render-container]` so the hidden HTML source doesn't contaminate the captured text. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Implement the AC table from CS-10785 by adding `static markdown` slots to the primitive field types in packages/base/card-api.gts: - StringField, ReadOnlyField, NumberField → markdownEscape the value, so metacharacters like `*`, `[`, `]`, `#`, and line-start `1.`/`-` don't trigger formatting when interpolated into a surrounding document. - TextAreaField → escape per line, then convert single `\n` to a CommonMark hard break (` \n`) so multi-line text renders as stacked lines instead of collapsing into one paragraph. - MarkdownField → raw passthrough (overrides the StringField inherited template) so author-written markdown is not double-escaped. - CSSField → emit a fenced ```css block; the fence width is the longest embedded backtick run + 1 (min 3) so pathological content can't close the block prematurely. - MaybeBase64Field → emit `[binary content]` for `data:` URIs to keep base64 payloads out of the markdown output; non-base64 strings fall back to escaped text. The StringField and TextAreaField slots are typed as `BaseDefComponent` to prevent TS from forcing structural compatibility on subclass overrides (MaybeBase64Field, MarkdownField, CSSField). Tests in packages/host/tests/integration/components/field-markdown-primitives-test.gts verify each primitive's markdown output, including escape behavior, hard breaks, fenced-block fence-width adjustment for embedded backticks, and a composition test that renders all primitives in a single card. Drive-by fix to the CS-10784 subclass-override test, which was querying `[data-markdown-render-container]` — a wrapper that only exists on the render route, not in rendering tests. Replaced with `this.element.textContent` to read the actual rendered markdown. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Two corrections after running the field-markdown-primitives tests against the freshly built host bundle: 1. NumberField negative/decimal expectation — `markdownEscape` only escapes line-anchored numeric prefixes (`/^(\s*\d+)\./gm`). For `-3.14`, after the always-escape pass produces `\-3.14`, the digits are no longer at the start of a line, so the `.` correctly stays unescaped. Updated the expected value from `\-3\.14` → `\-3.14`. 2. TextAreaField / MarkdownField / CSSField tests — were comparing raw `textContent` against an exact string, but the FieldComponent wrapper chain (CardContextConsumer → ... → DefaultFormatsProvider → Markdown) pads each level with template-driven whitespace that ends up around the rendered markdown content. Switched to the `readMarkdown` helper (which trims leading/trailing whitespace) and verified the inner content is exactly what the static markdown templates emit. All 13 field-markdown-primitives tests pass, and the 7 CS-10784 markdown-fallback tests still pass. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Adds field-specific `static markdown` slots to 19 specialized fields spanning date, scalar, link, and composite categories. Introduces `packages/base/markdown-helpers.ts` with shared `formatDateForMarkdown`, `formatDateTimeForMarkdown`, `formatDateRangeForMarkdown`, and `markdownLink` helpers so date formatting stays consistent across DateField/DateTimeField/DateRangeField and link construction is centralized (text escaped, href URL-encoded, parens quoted). Covers: - Date family: Date, DateTime, DateRange (shared formatter) - Scalars: Boolean, BigInteger, PhoneNumber, Color, Percentage, Country, LLMModel, EthereumAddress - Links: Email, Url, Website (markdown link format) - Composites: Address (hard-break rows), Coordinate, CodeRef (inline code span), Base64Image (binary placeholder), RichMarkdown (content passthrough) Tests live in packages/host/tests/integration/components/ field-markdown-specialized-test.gts (20 tests). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Adds field-specific `static markdown` slots to domain/reference, file, theme, and brand fields. Extends `packages/base/markdown-helpers.ts` with `fencedCodeBlock` (auto-widens the fence beyond any interior backtick run) and `markdownImage` helpers. Covers: - Reference/relation: RealmField (self-linked), ResponseField (status placeholder) - FileDef subclasses: MarkdownDef (passthrough), TsFileDef, GtsFileDef, JsonFileDef, CsvFileDef, TextFileDef (fenced code with per-subclass `static markdownLanguage`), ImageDef (markdown image reference) - Theme/style: CSSValueField (backtick fence with padding when value touches a backtick), TypographyField (bulleted properties), ThemeVarField/ThemeTypographyField (bulleted CSS variable entries), StructuredTheme (title/description/version + Typography + Root Variables sections) - Brand: MarkField (markdown image), BrandLogo (bulleted mark URLs with labels), BrandFunctionalPalette (bulleted palette colors) - enumField factory: renders the matching option's label (falls back to escaped raw value) Tests live in packages/host/tests/integration/components/ field-markdown-domain-test.gts (25 tests). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Add a `markdown` column to `boxel_index` and `boxel_index_working` and plumb the new format end-to-end: prerender step, `RenderResponse` / `FileRenderResponse` payload, writer `InstanceEntry` / `FileEntry` input, indexer destructure, and `IndexedInstance` / `IndexedFile` read results. Regenerate the SQLite schema snapshot. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Register a `GET /.*` route that dispatches on `Accept: text/markdown` and returns the indexed markdown body for a card instance. Returns 404 when the card is unknown, 406 when the markdown slot is unavailable (index error or null column), and responds with `Content-Type: text/markdown; charset=utf-8` on success. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Covers the full CS-10782..CS-10787 + CS-10798 surface via HTTP: format resolution, whitespace preservation, field delegation, MarkdownField passthrough, markdownEscape, HTML-to-markdown fallback, override precedence, frontmatter preservation, nested card composition, content-type header, and cache invalidation after a source rewrite. Uses `setupPermissionedRealmCached` with all card sources and instances seeded into the template DB so per-test cost is a fast DB restore. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Use namespace imports (import type * as X) for base realm modules in markdown field tests to access .default correctly - Fix StructuredTheme test to set cardInfo instead of computed cardTitle - Add markdown-fallback and markdown-helpers to expected deps in realm-indexing tests - Add markdown: null to prerender-proxy-test mock and assertion - Use trim() instead of trimEnd() in markdown endpoint frontmatter test Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…10791) Wire markdown format into code-mode preview panel and playground with a dedicated MarkdownPreview component that captures raw markdown text from the card's markdown template and offers Source (monospace pre) and Rendered (markdownToHtml) toggle views. Add acceptance test assertions for markdown format button and integration tests for toggle behavior. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Add a copyCardMarkdownToClipboard utility that fetches the card's markdown representation from the realm server (Accept: text/markdown) and writes it to the clipboard. Wire it into the interact-mode context menu via getDefaultCardMenuItems. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace the direct fetch in copyCardMarkdownToClipboard with a proper command that uses NetworkService.authedFetch, ensuring authentication headers are sent correctly. Remove the boxel-ui utility function in favor of the command pattern. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

TurndownService doesn't remove <style> or <script> tags by default, so CSS text content from card templates was leaking into the markdown output as plain text. Add converter.remove(['style', 'script']) to strip these elements during HTML-to-markdown conversion. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

… test expected refs The new CopyCardAsMarkdownCommand import and ClipboardCopy icon import in menu-items.ts added two new module references that the realm indexing tests need to expect. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Card HTML often wraps link text in nested elements (icon spans, divs) with newlines between them. Turndown preserves that internal whitespace, producing multiline `[\n Contact](url)` which is invalid markdown link syntax. Added a compactLinks turndown rule that collapses whitespace in link text to a single space. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

When the HTML-to-markdown fallback encounters a card container (identified by the production-safe data-boxel-card-id attribute), it now emits a card reference directive instead of inlining the full card HTML: - Atom-format cards → `:card[CARD_ID]` (inline directive) - Fitted/embedded cards → `::card[CARD_ID]` (block directive) Added data-boxel-card-id and data-boxel-card-format attributes to the field component's CardContainer alongside the existing data-test-* attrs, since the latter are stripped by ember-test-selectors in production builds. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

… panel The markdown preview "Rendered" tab now uses a dedicated RenderedMarkdown component that mirrors the base-realm MarkdownTemplate. It shares the same utilities (markdownToHtml, extractCardReferenceUrls, cardTypeName) and follows the same rendering pattern: convert markdown to HTML with BFM extensions, capture card-reference placeholders via a modifier, load referenced cards from the store, and render them into BFM slots using CardRenderer with #in-element. Unresolved references show as muted Pill badges. CSS is mirrored from the base-realm MarkdownTemplate to keep visual parity with .md file rendering. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

12 integration tests for the RenderedMarkdown component covering: - Basic markdown rendering (headings, paragraphs, lists, links, code) - Table wrapping - BFM inline :card[URL] placeholder creation - BFM block ::card[URL] placeholder creation - URL text stripping from BFM placeholders - Unresolved card references showing pill fallback (inline and block) - Multiple card references creating separate placeholders - Card references inside code blocks not being converted - Block card reference size spec data attributes Fixed CSS bleeding by wrapping markdown styles in @layer baseComponent, matching the base-realm MarkdownTemplate. This ensures unlayered card styles take cascade precedence over the generic markdown typography. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Consume CardContext (provided by the preview panel) and apply its cardComponentModifier to BFM card slot elements. This registers the embedded cards with the ElementTracker/Overlays system, which adds click handlers and pointer cursor — matching the behavior of cards embedded in .md files and AI assistant messages. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

The Overlays component was only rendered in the non-markdown format branches of the preview panel. This adds it to the markdown branch so that inline and block card references in the markdown preview are clickable and navigate to the referenced card in code mode. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

The .base-overlay CSS was missing pointer-events: none, which caused the overlay div to intercept mouse events on the card elements underneath. This was never noticed before because all other usages go through OperatorModeOverlays which uses .actions-overlay (which already has pointer-events: none). The base Overlays class is now used directly in the markdown preview panel, where the bug surfaced. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Now that the base .base-overlay class includes pointer-events: none, the spec-preview-overlay no longer needs to declare it separately. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Add custom `static markdown` template to RatingsSummary with ASCII star rendering - Add 'markdown' to field format chooser options in playground panel - Widen playground instance-chooser container (380px → 460px) to fit all format buttons - Wrap markdown preview in CardContainer for white background in playground - Update field playground test to cover markdown format selection Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Add four new helpers to `packages/base/markdown-helpers.ts`: - `markdownLinkForCard(card, text?)` — renders `[text](card.id)` - `markdownLinksForCards(cards, options?)` — list or inline card links - `markdownEmbedForCard(card, options?)` — renders `:card[url]` or `::card[url | spec]` - `markdownEmbedsForCards(cards, options?)` — multiple BFM card embeds Add custom `static markdown` template to BlogPost exercising all helpers: links for authors/categories/blog, image for featured image, embeds for author bios in footer. 26 unit tests covering all helpers, edge cases, and null handling. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Fix prettier formatting across markdown-related files - Remove unused imports (waitFor, Format) - Prefix unused parameter with underscore (_options) - Add scoped attribute to style tag in markdown-fallback-test - Fix DEFAULT_CARD_CONTEXT type in rendered-markdown.gts - Update realm-indexing tests to include data-boxel-card-id/format attrs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

The `#` and `{{@model.title}}` were on separate lines, producing `#\nTest Title` instead of `# Test Title`. ATX headings require the marker and text on the same line. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

LivePrerenderedSearchResource had no destructor, unlike its sibling PrerenderedSearchResource. Without cleanup, the buildPrerenderedCards task was never cancelled on destruction, _instances (TrackedArray of PrerenderedCard objects with HTML strings) was never cleared, and in-flight async renders held the resource alive past component teardown. This caused ~58MB/test retention in create-listing-modal tests, OOMing shard 15 in CI. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Markdown ATX headings require `#` at the start of the line. Prettier reformats the template content with indentation, which breaks heading parsing. Add prettier-ignore to preserve the intentional formatting. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

This PR lands end-to-end support for a new markdown render format across the stack: template authoring (static markdown), prerender capture, persistence in boxel_index, HTTP serving via Accept: text/markdown, UI preview tooling, and helper utilities/tests to keep the pipeline robust.

Changes:

Adds a first-class markdown format (type registration, host render wrapper, prerender capture, and new markdown-aware helpers/templates).
Persists prerendered markdown in boxel_index and serves it from the realm via an Accept: text/markdown endpoint.
Adds extensive unit/integration/e2e coverage plus UI affordances (preview panel + “Copy as Markdown”).

Reviewed changes

Copilot reviewed 96 out of 98 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
pnpm-lock.yaml	Locks new markdown conversion deps (turndown + GFM plugin + types).
packages/workspace-sync-cli/tests/helpers/start-test-realm.ts	Updates test realm payload to include `markdown`.
packages/runtime-common/supported-mime-type.ts	Adds `text/markdown` to supported MIME types.
packages/runtime-common/realm.ts	Adds `Accept: text/markdown` card endpoint backed by indexed markdown.
packages/runtime-common/index.ts	Extends render response types to include `markdown`.
packages/runtime-common/index-writer.ts	Writes `markdown` into index rows for instances/files.
packages/runtime-common/index-structure.ts	Adds `markdown` column to index table typing.
packages/runtime-common/index-runner/file-indexer.ts	Plumbs markdown from render into file indexing.
packages/runtime-common/index-runner/card-indexer.ts	Plumbs markdown from render into card indexing.
packages/runtime-common/index-query-engine.ts	Returns `markdown` from index queries + includes it in SELECT.
packages/runtime-common/formats.ts	Registers `markdown` in `Format` and formats list.
packages/runtime-common/error.ts	Adds a 406 `notAcceptable()` response helper.
packages/realm-server/tests/realm-endpoints/markdown-test.ts	New e2e tests validating served markdown behavior.
packages/realm-server/tests/prerendering-test.ts	Updates prerender test fixtures to include `markdown`.
packages/realm-server/tests/prerender-proxy-test.ts	Updates proxy test fixtures to include `markdown`.
packages/realm-server/tests/markdown-fallback-server-isolation-test.ts	Ensures turndown deps don’t leak into server-side packages.
packages/realm-server/tests/index.ts	Registers new markdown-related tests.
packages/realm-server/prerender/utils.ts	Captures `textContent` for markdown; narrows capture target.
packages/realm-server/prerender/render-runner.ts	Adds markdown render step for card/file prerendering.
packages/postgres/migrations/1776285887495_add-markdown-to-index.js	Adds `markdown` column to `boxel_index` + working table.
packages/host/types/@joplin/turndown-plugin-gfm/index.d.ts	Adds a type shim for the untyped CJS plugin.
packages/host/tests/unit/markdown-helpers-test.ts	Unit tests for markdown link/embed helper functions.
packages/host/tests/unit/index-writer-test.ts	Verifies markdown persistence round-trip through index writer/query.
packages/host/tests/unit/card-def-menu-items-test.ts	Ensures “Copy as Markdown” appears in menu items.
packages/host/tests/integration/realm-indexing-test.gts	Updates integration snapshots for new attributes/modules/icons.
packages/host/tests/integration/components/rendered-markdown-test.gts	Tests markdown→HTML rendering + BFM card placeholders.
packages/host/tests/integration/components/markdown-preview-test.gts	Tests preview panel source/rendered toggling.
packages/host/tests/integration/components/markdown-fallback-test.gts	Tests default HTML→markdown fallback behavior.
packages/host/tests/integration/components/field-markdown-specialized-test.gts	Tests specialized field markdown templates.
packages/host/tests/integration/components/field-markdown-primitives-test.gts	Tests primitive field markdown templates.
packages/host/tests/helpers/base-realm.ts	Exposes CSSField for new markdown field tests.
packages/host/tests/acceptance/code-submode/field-playground-test.gts	Adds markdown format to field playground acceptance coverage.
packages/host/tests/acceptance/code-submode/card-playground-test.gts	Adds markdown format to card playground acceptance coverage.
packages/host/tests/acceptance/code-submode-test.ts	Adds markdown/head format chooser assertions.
packages/host/package.json	Adds turndown + plugin + types to host deps.
packages/host/config/schema/1776285887495_schema.sql	Adds `markdown` column to host schema snapshots.
packages/host/app/templates/render/html.gts	Adds whitespace-preserving wrapper for markdown renders.
packages/host/app/resources/live-prerendered-search.ts	Adds destructor cleanup to prevent task/leak issues.
packages/host/app/lib/html-to-markdown.ts	Implements browser-only turndown conversion and global hook.
packages/host/app/instance-initializers/register-html-to-markdown.ts	Ensures HTML→MD converter is loaded at boot.
packages/host/app/components/operator-mode/preview-panel/markdown-preview.gts	Adds markdown preview component (source/rendered).
packages/host/app/components/operator-mode/preview-panel/index.gts	Wires markdown preview into preview panel.
packages/host/app/components/operator-mode/overlays.gts	Sets base overlays to not intercept pointer events.
packages/host/app/components/operator-mode/code-submode/spec-preview.gts	Adjusts overlay styling for spec preview.
packages/host/app/components/operator-mode/code-submode/playground/playground-preview.gts	Adds markdown preview rendering path in playground.
packages/host/app/components/operator-mode/code-submode/playground/playground-panel.gts	Adds markdown to field formats; widens chooser UI.
packages/host/app/components/card-prerender.gts	Adds markdown to client-side prerendering results.
packages/host/app/commands/index.ts	Registers new “copy card as markdown” host command module.
packages/host/app/commands/copy-card-as-markdown.ts	Implements “Copy as Markdown” command via markdown endpoint.
packages/experiments-realm/ratings-summary.gts	Adds markdown rendering for RatingsSummary field.
packages/experiments-realm/blog-post.gts	Adds markdown template using new markdown helpers.
packages/experiments-realm/Spec/fields/rating-field.json	Updates spec JSON for contained examples/metadata changes.
packages/experiments-realm/BlogPost/mad-as-a-hatter.json	Updates BlogPost JSON structure/meta/relationships.
packages/boxel-ui/test-app/tests/unit/markdown-escape-test.ts	Adds unit coverage for markdown escaping helper.
packages/boxel-ui/addon/src/helpers/markdown-escape.ts	Implements CommonMark/GFM escaping helper.
packages/boxel-ui/addon/src/helpers.ts	Exports `markdownEscape` from the helpers barrel.
packages/boxel-cli/tests/helpers/integration.ts	Updates noop prerenderer to include `markdown`.
packages/base/website.gts	Adds markdown output for WebsiteField.
packages/base/url.gts	Adds markdown output for UrlField.
packages/base/typography.gts	Adds markdown output for TypographyField.
packages/base/ts-file-def.gts	Adds markdown code-fence output + language tagging.
packages/base/text-file-def.gts	Adds markdown fenced-block output for text files.
packages/base/structured-theme.gts	Adds concise markdown summary for themes.
packages/base/structured-theme-variables.gts	Adds markdown output for theme variable fields.
packages/base/rich-markdown.gts	Adds passthrough markdown output for RichMarkdownField.
packages/base/response-field.gts	Adds placeholder markdown output for ResponseField.
packages/base/realm.gts	Adds markdown output (self-link) for realm URL field.
packages/base/phone-number.gts	Adds markdown output (tel link) for phone numbers.
packages/base/percentage.gts	Adds markdown output for percentages.
packages/base/menu-items.ts	Adds “Copy as Markdown” menu item.
packages/base/markdown-helpers.ts	Adds shared markdown helpers (links/embeds/images/fences/dates).
packages/base/markdown-file-def.gts	Adds markdown passthrough for markdown files.
packages/base/llm-model.gts	Adds markdown output for LLM model field.
packages/base/json-file-def.gts	Adds markdown fenced JSON output for JSON files.
packages/base/gts-file-def.gts	Sets markdown language for GTS modules.
packages/base/field-component.gts	Adds markdown recursion defaults + card container data attrs.
packages/base/ethereum-address.gts	Adds markdown output for Ethereum address.
packages/base/enum.gts	Adds markdown output selecting enum label.
packages/base/email.gts	Adds markdown mailto-link output for email.
packages/base/default-templates/markdown-fallback.gts	Adds default HTML→markdown fallback template.
packages/base/datetime.gts	Adds markdown output for DateTimeField using shared formatter.
packages/base/date.gts	Adds markdown output for DateField using shared formatter.
packages/base/date-range-field.gts	Adds markdown output for DateRangeField using shared formatter.
packages/base/csv-file-def.gts	Adds markdown fenced CSV output for CSV files.
packages/base/css-value.gts	Adds inline-code markdown output for CSS values.
packages/base/country.gts	Adds markdown output for country display name.
packages/base/coordinate.gts	Adds markdown output for coordinates.
packages/base/color.gts	Adds markdown output for color string (escaped).
packages/base/code-ref.gts	Adds inline-code markdown output for code refs.
packages/base/card-api.gts	Wires default markdown fallbacks + many field markdown templates.
packages/base/brand-logo.gts	Adds markdown output for brand logo URLs and mark image.
packages/base/brand-functional-palette.gts	Adds markdown output for palette entries.
packages/base/boolean.gts	Adds markdown output for booleans.
packages/base/big-integer.gts	Adds markdown output for big integers (escaped).
packages/base/base64-image.gts	Adds placeholder markdown output (no base64 payload).
packages/base/address.gts	Adds markdown output for address rows with hard breaks.

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

The previous commit incorrectly assumed .base-overlay's pointer-events rule would apply to spec-preview overlays, but Overlays uses @overlayClassName as a replacement for the default 'base-overlay' class, not an addition. Without this rule, spec-preview overlays intercept scroll/hover events on the underlying card content. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

lukemelia and others added 2 commits April 15, 2026 12:39

lukemelia changed the title ~~Register markdown in the Format type and formats array~~ Markdown format: Phase 1 rendering pipeline (CS-10780, CS-10781, CS-10782) Apr 15, 2026

lukemelia changed the title ~~Markdown format: Phase 1 rendering pipeline (CS-10780, CS-10781, CS-10782)~~ Markdown format: Phase 1 rendering pipeline (CS-10780, CS-10781, CS-10782, CS-10783) Apr 15, 2026

lukemelia changed the title ~~Markdown format: Phase 1 rendering pipeline (CS-10780, CS-10781, CS-10782, CS-10783)~~ Markdown format: Phase 1 rendering pipeline (CS-10780, CS-10781, CS-10782, CS-10783, CS-10784) Apr 15, 2026

lukemelia changed the title ~~Markdown format: Phase 1 rendering pipeline (CS-10780, CS-10781, CS-10782, CS-10783, CS-10784)~~ Markdown format: Phase 1 rendering pipeline (CS-10780 → CS-10785) Apr 15, 2026

lukemelia and others added 17 commits April 15, 2026 14:39

lukemelia and others added 6 commits April 16, 2026 16:19

Remove redundant pointer-events rule from spec-preview-overlay

90732ac

Now that the base .base-overlay class includes pointer-events: none, the spec-preview-overlay no longer needs to declare it separately. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

lukemelia changed the title ~~Markdown format: Phase 1 rendering pipeline (CS-10780 → CS-10785)~~ Markdown format: full rendering pipeline and tooling (CS-10780 → CS-10798) Apr 16, 2026

lukemelia and others added 3 commits April 16, 2026 18:17

Fix markdown-preview test templates to use valid ATX headings

d3eac4b

The `#` and `{{@model.title}}` were on separate lines, producing `#\nTest Title` instead of `# Test Title`. ATX headings require the marker and text on the same line. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

lukemelia requested a review from Copilot April 16, 2026 22:59

Copilot started reviewing on behalf of lukemelia April 16, 2026 23:03 View session

Copilot AI reviewed Apr 16, 2026

View reviewed changes

Comment thread packages/host/app/components/operator-mode/code-submode/spec-preview.gts

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Markdown format: full rendering pipeline and tooling (CS-10780 → CS-10798)#4412

Markdown format: full rendering pipeline and tooling (CS-10780 → CS-10798)#4412
lukemelia wants to merge 33 commits intomainfrom
cs-10780-register-markdown-in-the-format-type-and-formats-array

lukemelia commented Apr 15, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Apr 15, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Apr 15, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Apr 15, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

lukemelia commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Issues

Commits

1. CS-10780 — Register markdown in Format

2. CS-10781 — Whitespace-preserving container

3. CS-10782 — Prerender pipeline extraction

4. CS-10783 — markdownEscape template helper

5. CS-10784 — HTML-to-markdown fallback on CardDef/FieldDef/FileDef

6. CS-10785 — Default static markdown templates for primitive fields

7. CS-10786 — Explicit templates for specialized fields

8. CS-10787 — Explicit templates for domain fields

9. CS-10796 — Persist markdown in boxel_index

10. CS-10798 — Serve card markdown via HTTP endpoint

11. CS-10789 — E2E tests for the markdown rendering pipeline

12. CS-10790/CS-10791 — Markdown preview panel

13. CS-10794 — "Copy as Markdown" command

14. CS-10797 — Markdown helpers for card links and BFM card embeds

15. Playground + polish

Verification

Uh oh!

github-actions bot commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Realm Server Test Results

Uh oh!

github-actions bot commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Host Test Results

Uh oh!

github-actions bot commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Preview deployments

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

lukemelia commented Apr 15, 2026 •

edited

Loading

1. CS-10780 — Register `markdown` in Format

4. CS-10783 — `markdownEscape` template helper

5. CS-10784 — HTML-to-markdown fallback on `CardDef`/`FieldDef`/`FileDef`

6. CS-10785 — Default `static markdown` templates for primitive fields

9. CS-10796 — Persist markdown in `boxel_index`

github-actions bot commented Apr 15, 2026 •

edited

Loading

github-actions bot commented Apr 15, 2026 •

edited

Loading

github-actions bot commented Apr 15, 2026 •

edited

Loading