diff --git a/IDEAS.md b/IDEAS.md new file mode 100644 index 000000000..c47f101d4 --- /dev/null +++ b/IDEAS.md @@ -0,0 +1,224 @@ +# Improvement Ideas + +Generated: 2026-02-20 | Branch: `post_mirgation_cleanup` + +--- + +## 1. Remove ~201 Unused Files Detected by Knip + +**Severity**: High +**Effort**: Low (mechanical deletion after verification) +**Impact**: Reduces codebase noise, speeds up IDE indexing, cuts build graph + +### Data + +`bunx knip` reports 201 unused files, 339 unused exports, and 307 unused exported types. Main clusters: + +| Cluster | ~Files | Notes | +|---------|--------|-------| +| `prompt-smith/prompt-parts/` | 170 | **False positive** — loaded at build time via `codegen:prompts` script (filesystem-based, invisible to static analysis) | +| `commanders/librarian/healer/` | 10 | Tree-action adapters, canonical-naming codec, duplicate healing | +| `commanders/textfresser/` | 8 | shard-path, apply-meta, propagation domain (index, intent-key, merge-policy, normalize, note-adapter, ports, types) | +| `types/` | 5 | `literals.ts`, `literals/commands.ts`, `literals/linguistic.ts`, `common-interface/maybe.ts` | +| misc | 8 | `sentence-splitter.ts`, `debounce-scheduler.ts`, `dom-waiter.ts`, VAM `events.ts`, `get-editor.ts` | + +Unused dependencies: `sbd`, `@types/sbd` (sentence boundary detection — replaced by block-marker pipeline), `ts-plugin-sort-import-suggestions`, `typescript-eslint`. + +### Next Steps + +1. Add knip entry point for the codegen script in `knip.json` so prompt-parts are no longer false positives +2. Delete the ~31 genuinely unused files (healer, textfresser, types, misc clusters) +3. Remove unused dependencies: `sbd`, `@types/sbd`, `ts-plugin-sort-import-suggestions` +4. Run `bun run build && bun test` after each batch to catch false negatives +5. Tackle the 339 unused exports in a follow-up pass + +--- + +## 2. Standardize Error Handling — 22 Files Still Use Raw `throw` + +**Severity**: Medium +**Effort**: Medium +**Impact**: Consistent Result-based error flow; fewer uncaught exceptions at runtime + +### Data + +The codebase convention is `neverthrow` Result types (used in 92 files). However, 22 files still use raw `throw` with 37 total throw statements. Categories: + +| Category | Files | Throws | Justification | +|----------|-------|--------|---------------| +| API/Network failures | `api-service.ts` | 5 | Caught by `withRetry()` wrapper — tolerable | +| Tree invariant violations | `healer.ts`, `tree.ts`, `compute-leaf-healing.ts` | 8 | "Should never happen" bugs — arguably correct | +| Background coordination | `background-generate-coordinator.ts` | 4 | Top-level bubbling — convert to Result | +| Retry exhaustion | `retry.ts` | 3 | Last-resort throws — wrap in ResultAsync | +| State validation | `global-state.ts`, `parsed-settings.ts`, `maybe.ts` | 3 | Defensive checks — convert to Result | +| Misc business logic | 11 other files | 14 | Mixed — triage individually | + +### Next Steps + +1. **Priority 1**: Convert `background-generate-coordinator.ts` (4 throws) — these are in the hot path of Generate command +2. **Priority 2**: Convert `retry.ts` to return `ResultAsync` — callers already expect neverthrow +3. **Priority 3**: Audit `api-service.ts` — throws inside `withRetry` are OK, but raw throws at top-level aren't +4. Leave tree invariant `throw`s as-is (they signal bugs, not recoverable errors) + +--- + +## 3. Enable Stricter tsconfig Options + +**Severity**: Medium +**Effort**: Low (fix warnings iteratively) +**Impact**: Catches dead variables/params at compile time; prevents new dead code + +### Data + +Currently disabled in `tsconfig.json`: +```json +"noUnusedLocals": false, +"noUnusedParameters": false +``` + +These flags would surface unused variables and function parameters as compile errors, complementing the knip-based unused-export analysis. + +### Next Steps + +1. Enable `noUnusedLocals: true` first — run `bun run typecheck:changed` to see scope of breakage +2. Fix violations (prefix unused params with `_`, remove dead locals) +3. Enable `noUnusedParameters: true` in a second pass +4. Both changes prevent future accumulation of dead code + +--- + +## 4. Split Large Monolithic Files + +**Severity**: Medium +**Effort**: High (refactoring with test verification) +**Impact**: Better navigability, smaller diff surface, easier code review + +### Data + +24 files exceed 300 lines. The top 5: + +| File | Lines | Suggested Split | +|------|-------|-----------------| +| `textfresser/domain/propagation/note-adapter.ts` | 993 | Extract parsing logic, warning sampling, and format helpers into separate modules | +| `main.ts` | 822 | Extract initialization phases (settings, managers, commanders) into `init-*.ts` modules | +| `types/literals/linguistic.ts` | 586 | Data file — may be fine as-is if it's just enum/const declarations | +| `librarian/healer/codex/codex-impact-to-actions.ts` | 572 | Split by action type (ensure, process, write-status, backlink) | +| `librarian/librarian.ts` | 571 | Extract codex handling and healing delegation into sub-modules | + +### Next Steps + +1. Start with `main.ts` — extract `initManagers()`, `initCommanders()`, `registerCommands()` into focused modules +2. Split `note-adapter.ts` — the warning-sampling logic (lines 95–140) and format parsing are independent concerns +3. Leave `linguistic.ts` alone if it's pure data declarations +4. Verify with `bun run typecheck:changed && bun test` after each split + +--- + +## 5. Audit Unsafe `as unknown` Type Casts + +**Severity**: Medium +**Effort**: Low-Medium +**Impact**: Type safety; fewer runtime surprises from cast-masked bugs + +### Data + +9 files contain `as unknown` casts (13 total occurrences): + +| File | Cast | Verdict | +|------|------|---------| +| `cd.ts:38,48` | Accessing non-public Obsidian API (`leftSplit.collapsed`, `commands`) | **Keep** — documented undocumented API access | +| `vault-reader.ts:107,113,118` | `tRef as unknown as TFolder/TFile` | **Fix** — add instanceof narrowing instead | +| `split-path-to-locator.ts:41` | Generic type correlation lost | **Fix** — use overloads per project convention | +| `canonicalize-to-destination.ts:119,226` | Generic result unwrapping | **Fix** — add overload signatures | +| `api-service.ts:274` | Zod v3/v4 boundary | **Keep** — documented version bridge | +| `facade.ts:358` | Accessing private `selfEventTracker` internals | **Fix** — expose via interface | +| `checkbox-behavior.ts:23` | Duck-typing librarian method | **Fix** — add proper interface | +| `idle-tracker.ts:20` | `window` extension for E2E | **Keep** — test-only bridge | + +### Next Steps + +1. Fix the 5 "Fix" items using proper narrowing, overloads, or interfaces +2. Keep the 3 "Keep" items but ensure they have explanatory comments (most already do) + +--- + +## 6. Reduce Deep Relative Imports + +**Severity**: Low +**Effort**: Medium (tsconfig paths + mass rewrite) +**Impact**: Readability; less fragile imports when files move + +### Data + +104 import statements have 4+ levels of `../`. The deepest reach 7–8 levels: + +``` +// 7 levels deep: +../../../../../../managers/obsidian/vault-action-manager/types/split-path +``` + +Hotspots: `librarian/healer/library-tree/tree-action/bulk-vault-action-adapter/` (deepest nesting in codebase). + +Most deep imports target a few shared modules: +- `utils/logger` (from everywhere) +- `types/split-path` (from healer tree modules) +- `stateless-helpers/*` (from managers and commanders) + +### Next Steps + +1. Add tsconfig `paths` aliases: `@/utils/*`, `@/types/*`, `@/helpers/*` +2. Update the esbuild config to resolve the aliases +3. Migrate the 104 deep imports in batches (healer tree modules first) +4. Set a lint rule to warn on 4+ `../` levels going forward + +--- + +## 7. Calibrate Tunable Parameters with TODO Markers + +**Severity**: Low +**Effort**: Low (data collection + adjustment) +**Impact**: Correctness of fuzzy matching and sampling thresholds + +### Data + +Three parameters have explicit TODO-calibration markers: + +| Parameter | File | Current Value | Purpose | +|-----------|------|---------------|---------| +| `SEARCH_RADIUS` | `stateless-helpers/multi-span.ts:75` | 50 chars | Anchor-calibrated span mapping for separable prefix verbs | +| `MORPHOLOGY_STEM_MATCH_MIN_LENGTH` | `generate/steps/propagate-morphemes.ts:65` | 4 chars | Minimum stem length for prefix-match equivalence | +| `WARN_SAMPLE_MAX_KEYS` | `propagation/note-adapter.ts:102` | 2000 keys | Warning dedup cache size before flush | + +### Next Steps + +1. Collect a sample of real separable-verb attestations to test `SEARCH_RADIUS` (50 may be too small for compound verbs) +2. Test `MORPHOLOGY_STEM_MATCH_MIN_LENGTH` against real morphology data — 4 chars prevents false positives like "ab" matching "abnehmen", but may miss valid 3-char stems +3. `WARN_SAMPLE_MAX_KEYS` at 2000 is reasonable for typical vaults — verify with large corpus stats + +--- + +## 8. Consolidate Utils into Helper Facades + +**Severity**: Low +**Effort**: Low +**Impact**: Consistency with project's `xxxHelper` facade philosophy + +### Data + +`src/stateless-helpers/` follows the facade pattern well (10+ `xxxHelper` modules). However, `src/utils/` has small standalone files that could be consolidated: + +| File | Functions | Candidate Facade | +|------|-----------|-----------------| +| `array-utils.ts` | `dedupeByKeyFirst`, `dedupeByKeyLast` | `arrayHelper` | +| `text-utils.ts` | `extractHashTags` | `textHelper` or merge into `markdownStripHelper` | +| `delimiter.ts` | `buildCanonicalDelimiter`, `buildFlexibleDelimiterPattern` | `delimiterHelper` | + +### Assessment + +This is low-priority. The current utils are small (1–3 functions each) and creating facades for 2-function modules adds ceremony without much benefit. Consider consolidating only when these modules grow, or when new related functions are added. + +### Next Steps + +1. Defer unless new functions are added to these modules +2. If `text-utils.ts` grows, merge into `markdownStripHelper` (related concern) +3. If `array-utils.ts` grows, create `arrayHelper` facade with proper barrel export diff --git a/documentation/book-of-work.md b/documentation/book-of-work.md deleted file mode 100644 index 54e35d82c..000000000 --- a/documentation/book-of-work.md +++ /dev/null @@ -1,15 +0,0 @@ -# Book Of Work - -## Deferred Follow-Ups (Morphological Relations V1) - -### 1) Prefix derivations: avoid redundant `` with equation -- Status: Deferred by request. -- Current behavior: prefix cases can render both: - - `` `[[base]]` - - `[[prefix|decorated]] + [[base]] = [[source]] *(gloss)*` -- Follow-up decision to implement later: for inferred prefix derivations, render only the equation and skip ``. - -### 2) Architecture doc table sync for Lexem POS section coverage -- Status: Deferred by request. -- Gap: `sectionsForLexemPos` in code includes `Morphology` for all Lexem POS, but the table in `/Users/annagorelova/work/Textfresser_vault/.obsidian/plugins/textfresser/src/documentaion/linguistics-and-prompt-smith-architecture.md` is only partially updated. -- Follow-up to implement later: update all POS rows in the table so docs exactly match `section-config.ts`. diff --git a/hands-on-runbook.md b/hands-on-runbook.md new file mode 100644 index 000000000..e8b8b4e7f --- /dev/null +++ b/hands-on-runbook.md @@ -0,0 +1,428 @@ +# Textfresser Hands-On Runbook (Obsidian CLI) + +Manual sanity checks for how Lemma/Generate commands interact with vault state. +Run against `cli-e2e-test-vault`. + +--- + +## Philosophy + +1. **Resettable runs** — recreate source notes before each scenario. No leftover state. +2. **Single eval for selection commands** — open file + select text + fire command in one `eval` to avoid editor focus races. +3. **Gate assertions with `whenIdle()`** — idle tracker covers healing cascades AND background Generate. +4. **Assert via vault files** — `read` + `files` + `search` are ground truth. Command return values are best-effort. +5. **Composable one-liners over long scripts** — the CLI is early-access. Short commands are easier to debug. + +--- + +## 0) Prerequisites + +### Obsidian Binary Version + +The CLI ships with Obsidian 1.12 (Early Access / Catalyst). The **Obsidian.app binary** must be **v1.11.7+**. + +Obsidian's auto-updater only patches the `.asar` runtime — the Electron shell binary stays at whatever version was installed. If the binary is too old, every CLI call hangs with `Loading updated app package` and never returns. + +```bash +defaults read /Applications/Obsidian.app/Contents/Info.plist CFBundleShortVersionString +# Must be >= 1.11.7 +# If 1.7.x → re-download from obsidian.md/download and replace /Applications/Obsidian.app +``` + +### Vault & Plugin + +- `cli-e2e-test-vault` exists and is open in Obsidian +- Plugin `cbcr-text-eater-de` is enabled +- CLI enabled: Obsidian Settings → General → Command line interface +- Vault path: `/Users/annagorelova/work/obsidian/cli-e2e-test-vault` + +--- + +## 1) Setup + +### Shell Variables + +```bash +OBS="/Applications/Obsidian.app/Contents/MacOS/Obsidian" +VAULT="cli-e2e-test-vault" +PLUGIN="cbcr-text-eater-de" +SRC="Outside/Textfresser-Lemma-Manual.md" +``` + +### Or: Source the Helpers + +```bash +source scripts/runbook/helpers.sh +# Provides: wait_idle, read_source, reset_source, list_entries, read_entry, +# nuke_entries, reload_plugin, check_no_nested_wikilinks, check_surface_linked +``` + +### Preflight (Must All Return Instantly) + +```bash +$OBS vault=$VAULT vault info=path +# → /Users/annagorelova/work/obsidian/cli-e2e-test-vault + +$OBS vault=$VAULT eval code="'cli works'" +# → => cli works + +$OBS vault=$VAULT eval code="(async()=>{const p=app.plugins.plugins['$PLUGIN'];return p?'plugin loaded':'MISSING'})()" +# → => plugin loaded +``` + +If any command hangs → see [Troubleshooting](#troubleshooting). + +--- + +## 2) Inline Helpers + +For copy-paste use without sourcing scripts: + +```bash +wait_idle() { + $OBS vault=$VAULT eval \ + code="(async()=>{await app.plugins.plugins['$PLUGIN'].whenIdle();return 'idle'})()" +} + +reset_source() { + $OBS vault=$VAULT eval \ + code="(async()=>{const f=app.vault.getAbstractFileByPath('$SRC');if(f)await app.vault.trash(f,true);return 'ok'})()" + $OBS vault=$VAULT create path="$SRC" \ + content="Der Mann liest heute ein Buch.\nDie Katze schlaeft dort.\nDer Plan wirkt klar.\nDer Ablauf bleibt deutlich." silent +} + +read_source() { + $OBS vault=$VAULT read path="$SRC" +} + +check_nested() { + read_source | grep -E '\[\[[^\]]*\[\[' && echo "FAIL: nested wikilinks" || echo "OK: no nesting" +} + +check_entries() { + $OBS vault=$VAULT files folder="Worter/de" ext=md +} + +lemma_state() { + $OBS vault=$VAULT eval \ + code="(async()=>{const p=app.plugins.plugins['$PLUGIN'];const lr=p.textfresser.getState().latestLemmaResult;return lr?JSON.stringify({lemma:lr.lemma,pos:lr.posLikeKind,unit:lr.linguisticUnit}):JSON.stringify(null)})()" +} +``` + +### `lemma_fire` — The Recommended Way + +Use the Bun script to avoid shell escaping issues (especially for umlauts, `!`, `$`): + +```bash +bun scripts/runbook/lemma-fire.ts Mann +bun scripts/runbook/lemma-fire.ts Katze +bun scripts/runbook/lemma-fire.ts klar +``` + +The script uses `Bun.spawn` (no shell) → no zsh mangling. It tries `textfresser.executeCommand()` first, falls back to `app.commands.executeCommandById()`, waits for idle, and polls for the wikilink. + +**Fallback** — inline eval for simple ASCII surfaces: + +```bash +lemma_fire() { + local surface="$1" + $OBS vault=$VAULT eval code="(async()=>{ + const file=app.vault.getAbstractFileByPath('$SRC'); + if(!file) throw new Error('not found: $SRC'); + const leaf=app.workspace.getMostRecentLeaf()??app.workspace.getLeaf(true); + await leaf.openFile(file,{active:true}); + const view=leaf.view; + if(view?.getMode?.()!=='source') await view.setMode('source'); + const editor=view?.editor??app.workspace.activeEditor?.editor; + if(!editor) throw new Error('no editor'); + const content=editor.getValue(); + const idx=content.indexOf('$surface'); + if(idx===-1) throw new Error('surface not found: $surface'); + const toPos=(o)=>{const ls=content.slice(0,o).split('\n');return{line:ls.length-1,ch:ls[ls.length-1].length}}; + editor.setSelection(toPos(idx),toPos(idx+'$surface'.length)); + editor.focus?.(); + app.commands.executeCommandById('$PLUGIN:lemma'); + return 'ok'; + })()" +} +``` + +--- + +## 3) Scenarios + +### A: Idempotence on Same Surface + +Two Lemma runs on the same token must not create `[[...[[...]]...]]`. + +```bash +reset_source + +bun scripts/runbook/lemma-fire.ts Mann +wait_idle +bun scripts/runbook/lemma-fire.ts Mann +wait_idle + +read_source +check_nested +lemma_state +check_entries +``` + +**Expected**: `Mann` linked once (as `[[Mann]]` or `[[something|Mann]]`). No nested wikilinks. A dictionary entry may appear in `Worter/de/` if Gemini API key is configured. + +### B: Back-to-Back on Different Tokens + +Tests pending-generate queue behavior. + +```bash +reset_source + +bun scripts/runbook/lemma-fire.ts klar +bun scripts/runbook/lemma-fire.ts deutlich +wait_idle + +read_source +check_nested +check_entries +``` + +**Expected**: both `klar` and `deutlich` are wikilinked. No nesting. + +### C: Burst on 4 Tokens + +```bash +reset_source + +for w in Mann Katze klar deutlich; do + bun scripts/runbook/lemma-fire.ts "$w" +done +wait_idle + +read_source +check_nested +check_entries +``` + +**Expected**: all 4 linked. Background Generate runs for the last pending Lemma (earlier ones get superseded). + +### D: Re-Encounter (Existing Entry) + +Lemma on an already-known word should append an attestation, not re-create the entry. + +```bash +# First, create an entry via scenario A +reset_source +bun scripts/runbook/lemma-fire.ts Mann +wait_idle + +# Now fire on "Mann" in a different source +SRC2="Outside/Textfresser-Re-Encounter.md" +$OBS vault=$VAULT create path="$SRC2" content="Der alte Mann ging spazieren." silent + +SRC="$SRC2" bun scripts/runbook/lemma-fire.ts Mann +wait_idle + +$OBS vault=$VAULT read path="$SRC2" +# → "Mann" should link to the SAME entry (not a new one) + +# Clean up +$OBS vault=$VAULT delete path="$SRC2" +SRC="Outside/Textfresser-Lemma-Manual.md" # restore +``` + +### E: Inspect Generated Entry + +After any scenario that creates dictionary entries: + +```bash +check_entries + +# Read a specific entry (adjust path) +$OBS vault=$VAULT read path="Worter/de/Mann.md" + +# Check frontmatter +$OBS vault=$VAULT property:read name=noteKind path="Worter/de/Mann.md" +# → DictEntry +``` + +**Expected**: entry has `noteKind: DictEntry`, contains sections (Header, Morphem, Inflection, Translation, Attestation). + +### F: Librarian Healing Sanity + +Not Textfresser-specific — validates the healing pipeline after vault mutations. + +```bash +# Create a file in Library → should get suffix-healed +$OBS vault=$VAULT eval code="(async()=>{await app.vault.createFolder('Library/TestSection');await app.vault.create('Library/TestSection/MyNote.md','# Test');return 'ok'})()" +wait_idle + +$OBS vault=$VAULT files folder="Library/TestSection" ext=md +# Expected: __-TestSection.md (codex) + MyNote-TestSection.md (suffix-healed) + +# Clean up +$OBS vault=$VAULT eval code="(async()=>{const f=app.vault.getAbstractFileByPath('Library/TestSection');if(f)await app.vault.trash(f,true);return 'ok'})()" +wait_idle +``` + +--- + +## CLI Quick Reference + +### File Operations + +| Command | Syntax | +|---------|--------| +| List files | `$OBS vault=$VAULT files [folder="X"] [ext=md]` | +| Read file | `$OBS vault=$VAULT read path="X.md"` | +| Create file | `$OBS vault=$VAULT create path="X.md" content="..." [overwrite] [silent]` | +| Append | `$OBS vault=$VAULT append path="X.md" content="..."` | +| Delete file | `$OBS vault=$VAULT delete path="X.md"` | +| Search | `$OBS vault=$VAULT search query="text"` | + +### Plugin & Vault + +| Command | Syntax | +|---------|--------| +| Vault path | `$OBS vault=$VAULT vault info=path` | +| Reload plugin | `$OBS vault=$VAULT plugin:reload id=$PLUGIN` | +| List commands | `$OBS vault=$VAULT commands` | +| Tags | `$OBS vault=$VAULT tags` | +| Read property | `$OBS vault=$VAULT property:read name=KEY path="file.md"` | +| Set property | `$OBS vault=$VAULT property:set name=KEY value=VAL path="file.md"` | + +### Eval + +```bash +$OBS vault=$VAULT eval code="" +``` + +- Output prefixed with `=> `. Errors prefixed with `Error:`. +- Access to `app.*` (Obsidian API) and `app.plugins.plugins['cbcr-text-eater-de']`. +- For async: wrap in `(async()=>{...})()`. + +### Eval Recipes + +```bash +# Create folder (CLI create doesn't handle folders) +$OBS vault=$VAULT eval code="(async()=>{await app.vault.createFolder('Library/New');return 'ok'})()" + +# Delete folder (CLI delete only handles files) +$OBS vault=$VAULT eval code="(async()=>{const f=app.vault.getAbstractFileByPath('Library/Old');if(f)await app.vault.trash(f,true);return 'ok'})()" + +# Rename/move (uses fileManager for proper event emission) +$OBS vault=$VAULT eval code="(async()=>{const f=app.vault.getAbstractFileByPath('A.md');await app.fileManager.renameFile(f,'B.md');return 'ok'})()" + +# Check file exists (CLI read returns exit 0 even for missing files) +$OBS vault=$VAULT eval code="app.vault.getAbstractFileByPath('path.md')?'yes':'no'" + +# Wait for plugin idle +$OBS vault=$VAULT eval code="(async()=>{await app.plugins.plugins['$PLUGIN'].whenIdle();return 'idle'})()" + +# Read Lemma state +$OBS vault=$VAULT eval code="(async()=>{const p=app.plugins.plugins['$PLUGIN'];const lr=p.textfresser.getState().latestLemmaResult;return JSON.stringify(lr)})()" + +# List loaded plugins +$OBS vault=$VAULT eval code="Object.keys(app.plugins.plugins).join(', ')" +``` + +--- + +## Gotchas + +### CLI Always Returns Exit 0 +Even on errors. Parse stdout for `Error:` prefix instead of checking `$?`. + +### `create` with Paths +Use `path=` for files in subdirectories. `name=` only works for root-level files (no `/` allowed). + +### Shell Escaping Eats Special Characters +zsh mangles `!`, `$`, and unicode inside double quotes. For eval code with umlauts or special chars, use `Bun.spawn` array args (no shell) — that's what `lemma-fire.ts` and the automated tests do. + +### Folder Operations Need `eval` +CLI `create`/`delete` only handle files. Folders require `app.vault.createFolder()` / `app.vault.trash()` via eval. + +### CLI Output Noise +Occasional lines like `Loaded updated app package ...` or `Checking for updates`. The test infra strips them; for manual use, just ignore. + +### `help` Opens TUI +`$OBS help` without a vault opens an interactive terminal UI. Not scriptable. + +### `version` Returns Nothing +`$OBS version` and `$OBS vault=$VAULT version` both return empty output as of 1.12.2. + +--- + +## Troubleshooting + +### CLI Hangs Forever + +**Most likely**: Obsidian.app binary is too old. Check: +```bash +defaults read /Applications/Obsidian.app/Contents/Info.plist CFBundleShortVersionString +``` +Must be `>= 1.11.7`. If `1.7.x`: +1. Quit Obsidian +2. Download latest from [obsidian.md/download](https://obsidian.md/download) (or `obsidianmd/obsidian-releases` on GitHub) +3. Replace `/Applications/Obsidian.app` +4. Relaunch + +**Second cause**: zombie CLI processes holding the IPC socket: +```bash +ps aux | grep -i "[O]bsidian" | grep -v Helper | grep -v Cursor +# Kill any stale CLI processes (NOT the main Obsidian GUI) +kill +``` + +### Vault Not Found + +```bash +cat ~/Library/Application\ Support/obsidian/obsidian.json +# Look for cli-e2e-test-vault with "open": true +``` + +If not open: +```bash +open "obsidian://open?vault=cli-e2e-test-vault" +sleep 3 +``` + +### Plugin Not Loaded + +```bash +$OBS vault=$VAULT eval code="Object.keys(app.plugins.plugins).join(', ')" +``` + +If missing: check `community-plugins.json` and that `main.js` + `manifest.json` exist in the plugin dir. + +### Deploy Fresh Build + +```bash +bun run build +cp main.js /Users/annagorelova/work/obsidian/cli-e2e-test-vault/.obsidian/plugins/cbcr-text-eater-de/main.js +cp manifest.json /Users/annagorelova/work/obsidian/cli-e2e-test-vault/.obsidian/plugins/cbcr-text-eater-de/manifest.json +$OBS vault=$VAULT plugin:reload id=$PLUGIN +``` + +--- + +## Full Reset + +```bash +# Trash source files +$OBS vault=$VAULT eval code="(async()=>{for(const p of['Outside/Textfresser-Lemma-Manual.md','Outside/Textfresser-Re-Encounter.md']){const f=app.vault.getAbstractFileByPath(p);if(f)await app.vault.trash(f,true)}return 'ok'})()" + +# Trash dictionary entries +$OBS vault=$VAULT eval code="(async()=>{const f=app.vault.getAbstractFileByPath('Worter');if(f)await app.vault.trash(f,true);return 'ok'})()" + +wait_idle +``` + +--- + +## Known Pain Points (2026-02-20) + +1. Zero-byte dictionary notes can be created and linked when background Generate fails silently. +2. `whenIdle()` resolves when async work finishes, but doesn't guarantee the entry is non-empty. +3. Shell eval with umlauts is fragile — always use `lemma-fire.ts` for non-ASCII surfaces. +4. `command id=...` returning `Executed` is not proof of success — check vault state. diff --git a/logbook/2026-02-20_05-17_cli-lemma-session.md b/logbook/2026-02-20_05-17_cli-lemma-session.md new file mode 100644 index 000000000..8b23a6b7f --- /dev/null +++ b/logbook/2026-02-20_05-17_cli-lemma-session.md @@ -0,0 +1,61 @@ +# CLI Lemma Session Log + +- Date: 2026-02-20 +- Time window: 05:15-05:18 (+0100) +- Vault: `cli-e2e-test-vault` +- Source: `textfresser/0_Synthetic_Test_To_Check_Morhp.md` +- Spacing policy: 4 seconds between lemma calls (within requested 3-5s) + +## 1) Pre-step: clean source note from wikilinks + +Action: +- Removed all `[[...]]` and `[[target|alias]]` wrappers from source note while preserving visible text. + +Result: +- Source had zero wikilinks after cleanup. + +## 2) Lemma poke sequence (4s spacing) + +Order: +1. `fährt` +2. `Fahrkarte` +3. `Abfahrt` +4. `arbeiten` +5. `Zusammenarbeit` +6. `Arbeit` +7. `Aufstehen` +8. `Aufräumen` +9. `Beitritt` + +Each call reported: +- `fired` +- `=> idle` + +## 3) Observed outcome + +Unexpected behavior: +- Source note remained plain text after all calls (no inserted wikilinks). +- `links-source` returned: `No links found.` +- Plugin state probe returned: + - `hasState: true` + - `hasLemma: true` + - `pending: false` +- `dev:errors` returned `No errors captured.` + +## 4) Worter/de state after this run + +Observed file tree: +- Only one markdown file present under `Worter/de`: + - `Worter/de/lexem/lemma/a/arb/arbei/Arbeit.md` + +File quality: +- `Arbeit.md` is zero-byte (`words: 0`, `characters: 0`). + +Most previously expected lemma targets were missing entirely in this run (`fahren`, `Fahrkarte`, `Abfahrt`, `Zusammenarbeit`, `aufstehen`, `aufräumen`, `Beitritt`, etc.). + +## 5) Session conclusion + +Something is off: +- Lemma execution path reports success-like signals (`fired` + idle), but source rewrite and entry generation are not happening consistently. +- Current run suggests a silent no-op/partial-write mode. + diff --git a/logbook/2026-02-20_05-20_command-route-probe.md b/logbook/2026-02-20_05-20_command-route-probe.md new file mode 100644 index 000000000..feca64b1f --- /dev/null +++ b/logbook/2026-02-20_05-20_command-route-probe.md @@ -0,0 +1,28 @@ +# Command Route Probe + +- Date: 2026-02-20 +- Time: ~05:20 (+0100) +- Vault: `cli-e2e-test-vault` +- Source: `textfresser/0_Synthetic_Test_To_Check_Morhp.md` + +## Test + +Used Obsidian CLI `eval` to: +1. Open source note in editor. +2. Select `fährt` in editor. +3. Execute `app.commands.executeCommandById('cbcr-text-eater-de:lemma')`. +4. Wait for `whenIdle()`. +5. Read source note back. + +## Result + +- Command route returned `command-fired`. +- Idle wait returned `idle`. +- Source remained unchanged (no wikilink insertion). + +## Conclusion + +No-marking issue reproduces through both paths: +- direct helper path (`textfresser-runbook lemma` -> `lemma-fire.ts`) +- direct Obsidian command route (`executeCommandById`) + diff --git a/logbook/2026-02-20_05-32_lemma-invocation-reliability.md b/logbook/2026-02-20_05-32_lemma-invocation-reliability.md new file mode 100644 index 000000000..87dc6b07e --- /dev/null +++ b/logbook/2026-02-20_05-32_lemma-invocation-reliability.md @@ -0,0 +1,38 @@ +# Lemma Invocation Reliability Investigation + +- Date: 2026-02-20 +- Vault: `cli-e2e-test-vault` +- Source: `textfresser/0_Synthetic_Test_To_Check_Morhp.md` + +## Problem + +Observed inconsistent behavior where lemma calls looked successful but source marking did not happen, or happened later than expected. + +## Root findings + +1. Previous helper (`scripts/runbook/lemma-fire.ts`) did not await `textfresser.executeCommand(...)` result. +2. Obsidian CLI `eval` intermittently returns empty stdout even when side effects happen. +3. Relying on eval response alone is unreliable in this vault session. + +## Reliable strategy + +Use post-condition verification as source of truth: +1. Trigger lemma via direct `textfresser` path. +2. Poll source file content via CLI `read` (not eval output) until the selected surface is linked. +3. If not linked within timeout, trigger fallback command-id route (`cbcr-text-eater-de:lemma`) and poll again. +4. Report success only when source content actually contains wikilink for the surface. + +## Implementation + +Updated `scripts/runbook/lemma-fire.ts` to: +- support two invocation strategies (textfresser + command-id fallback), +- tolerate empty eval output, +- verify success by reading source content and polling for link insertion. + +## Validation run + +After cleaning wikilinks and running 9 lemma calls with 4s spacing: +- source got marked for all tested surfaces, +- `links-source` returned expected targets, +- `scan-empty` reported all linked entries non-empty. + diff --git a/logbook/2026-02-20_10-26_synthetic-morph-lemma-pass.md b/logbook/2026-02-20_10-26_synthetic-morph-lemma-pass.md new file mode 100644 index 000000000..6be4da819 --- /dev/null +++ b/logbook/2026-02-20_10-26_synthetic-morph-lemma-pass.md @@ -0,0 +1,79 @@ +# Synthetic Morph Lemma Pass + +- Date: 2026-02-20 +- Time window: 10:23-10:26 (+0100) +- Vault: `cli-e2e-test-vault` +- Source: `textfresser/0_Synthetic_Test_To_Check_Morhp.md` +- Invocation: `scripts/runbook-cli/textfresser-runbook lemma ` +- Spacing policy: 4 seconds between calls + +## Scope + +Lemma called in sequence for: +1. `Fahrer` +2. `fährt` +3. `Fahrkarte` +4. `Abfahrt` +5. `unterschreibt` +6. `Unterschrift` +7. `Bauarbeiter` +8. `bauen` +9. `Neubau` +10. `stelle` +11. `Vorstellung` +12. `arbeiten` +13. `Zusammenarbeit` +14. `Arbeit` + +All calls returned `fired (textfresser)`. + +## Source outcome + +Resulting source note: + +```md +Der [[Fahrer]] [[Fahren|fährt]] mit der [[Fahrkarte]] zur [[Abfahrt]]. ^0 + +Sie [[unterschreiben|unterschreibt]] das Formular, und ihre [[Unterschrift]] steht schon unten. ^1 + +Die [[Bauarbeiter]] [[bauen]] heute einen [[Neubau]] am Stadtrand. ^2 + +Ich [[vorstellen|stelle]] mich kurz vor, und meine [[vorstellen|Vorstellung]] ist sehr knapp. ^3 + +Wir [[arbeiten]] im Team, und die [[Zusammenarbeit]] verbessert unsere [[Arbeit]]. ^4 +``` + +## Quirks + +1. Mixed target kinds for similar verbs: + - `fährt` linked as `[[Fahren|fährt]]` (lemma target). + - `bauen`/`arbeiten` linked as `[[bauen]]`/`[[arbeiten]]` (inflected bucket path). +2. `Vorstellung` was normalized to verb lemma target `[[vorstellen|Vorstellung]]` instead of a noun lemma target. +3. Case behavior varies by resolved target (`Fahren` capitalized vs `unterschreiben` lowercase). + +## Problems + +1. Potential POS drift on verb forms: + - `fährt` appears to resolve to `Fahren` (capitalized target), which can collide with noun interpretation. +2. Potential over-normalization on derivational nouns: + - `Vorstellung` mapped to `vorstellen`, reducing noun-level specificity in links. +3. Storage policy looks inconsistent to operators: + - some resolved links go to `Worter/de/lexem/lemma/...`, + - others go to `Worter/de/lexem/inflected/...`. + +## Proposals + +1. Add explicit POS guardrails in lemma linking: + - if model POS is verb, prefer lowercase infinitive target; + - avoid noun-case canonicalization unless POS is noun. +2. Add configurable noun-derivation policy: + - either keep noun lemma (`Vorstellung`) or map to base verb (`vorstellen`), but make it explicit and stable. +3. Add a post-link consistency check: + - flag when same POS family is split between `lemma` and `inflected` paths in one run. +4. Extend runbook output with per-surface trace: + - surface, resolved lemma, POS, chosen path kind (`lemma|inflected`) to make quirks visible immediately. + +## Integrity checks + +- `check-no-nested`: `OK: no nested wikilinks` +- `scan-empty`: 13 linked entries checked, all non-empty diff --git a/logbook/2026-02-20_10-35_synthetic-morph-lemma-pass-fresh-build.md b/logbook/2026-02-20_10-35_synthetic-morph-lemma-pass-fresh-build.md new file mode 100644 index 000000000..89a9ffe95 --- /dev/null +++ b/logbook/2026-02-20_10-35_synthetic-morph-lemma-pass-fresh-build.md @@ -0,0 +1,66 @@ +# Synthetic Morph Lemma Pass (Fresh Build) + +- Date: 2026-02-20 +- Time window: 10:31-10:35 (+0100) +- Vault: `cli-e2e-test-vault` +- Source: `textfresser/0_Synthetic_Test_To_Check_Morhp.md` +- Build status: `bun run build` completed before run (`main.js` rebuilt) +- Plugin reload: executed before lemma run +- Spacing policy: 4 seconds between calls + +## Execution sequence + +Lemma called in sequence for: +1. `Fahrer` +2. `fährt` +3. `Fahrkarte` +4. `Abfahrt` +5. `unterschreibt` +6. `Unterschrift` +7. `Bauarbeiter` +8. `bauen` +9. `Neubau` +10. `stelle` +11. `Vorstellung` +12. `arbeiten` +13. `Zusammenarbeit` +14. `Arbeit` + +All calls returned `fired (textfresser)`. + +## Source outcome + +```md +Der [[Fahrer]] [[Worter/de/lexem/lemma/f/fah/fahre/Fahren|fährt]] mit der [[Fahrkarte]] zur [[Abfahrt]]. ^0 + +Sie [[unterschreiben|unterschreibt]] das Formular, und ihre [[Unterschrift]] steht schon unten. ^1 + +Die [[Bauarbeiter]] [[bauen]] heute einen [[Neubau]] am Stadtrand. ^2 + +Ich [[vorstellen|stelle]] mich kurz vor, und meine [[vorstellen|Vorstellung]] ist sehr knapp. ^3 + +Wir [[arbeiten]] im Team, und die [[Zusammenarbeit]] verbessert unsere [[Arbeit]]. ^4 +``` + +## Quirks + +1. `reset-source` command currently recreates this path with a generic fixture text, not the synthetic morph fixture; source had to be restored manually before run. +2. `fährt` linked with explicit full path alias (`[[Worter/de/.../Fahren|fährt]]`) while most other links used short wiki targets. +3. Verb and derivational-noun normalization behavior remains mixed (`vorstellen|Vorstellung`, but noun lemmas retained for others). + +## Problems + +1. Fixture reset mismatch makes repeatable CLI e2e passes error-prone for this specific file. +2. Link target formatting is not canonicalized (mix of short targets and full-path target aliases). +3. POS/lemma policy is still ambiguous for cases like `fährt -> Fahren` and `Vorstellung -> vorstellen`. + +## Proposals + +1. Fix `reset-source` to support named fixtures and ensure `textfresser/0_Synthetic_Test_To_Check_Morhp.md` restores exact canonical synthetic content. +2. Add canonical wikilink output rule: either always short target or always full path, but not mixed in one note. +3. Add explicit POS policy checks in lemma pipeline and surface them in runbook output (`surface`, `lemma`, `pos`, `targetPath`, `targetKind`). + +## Integrity checks + +- `check-no-nested`: `OK: no nested wikilinks` +- `scan-empty`: 13 linked entries checked, all non-empty diff --git a/package.json b/package.json index 786b5ab12..58e7fafa6 100644 --- a/package.json +++ b/package.json @@ -47,7 +47,7 @@ "lint": "bun x biome check .", "prompt:stability": "bun tests/prompt-test/stability-runner.ts", "prompt:test": "bun tests/prompt-test/runner.ts", - "silo": "bun scripts/knowledge-silo.ts", + "runbook": "bash scripts/runbook-cli/textfresser-runbook", "splitter:run": "bun tests/splitter-runner.ts", "test": "bun run test:unit", "test:cli-e2e": "bun run build && bun test --env-file=.env.cli-e2e tests/cli-e2e/ --timeout 25000", diff --git a/scripts/knowledge-silo.ts b/scripts/knowledge-silo.ts deleted file mode 100644 index b8d5bdf59..000000000 --- a/scripts/knowledge-silo.ts +++ /dev/null @@ -1,435 +0,0 @@ -#!/usr/bin/env bun - -/** - * Knowledge Silo Detector - * - * Analyzes git history to detect modules where a single contributor dominates - * ownership, indicating a "knowledge silo" risk (low bus factor). - * - * Usage: bun scripts/knowledge-silo.ts [--days=N] [--threshold=N] - * --days Recency window for silo detection (default: 90) - * --threshold Ownership % above which a single author is flagged (default: 80) - */ - -import { existsSync } from "node:fs"; -import { $ } from "bun"; - -// ── Config ────────────────────────────────────────────────────────────────── - -interface CliArgs { - recencyDays: number; - siloThreshold: number; -} - -function parseArgs(): CliArgs { - const args = process.argv.slice(2); - let recencyDays = 90; - let siloThreshold = 80; - - for (const arg of args) { - const daysMatch = arg.match(/^--days=(\d+)$/); - if (daysMatch) { - recencyDays = Number(daysMatch[1]); - continue; - } - const thresholdMatch = arg.match(/^--threshold=(\d+)$/); - if (thresholdMatch) { - siloThreshold = Number(thresholdMatch[1]); - continue; - } - } - - return { recencyDays, siloThreshold }; -} - -// ── Types ─────────────────────────────────────────────────────────────────── - -interface AuthorStats { - linesAdded: number; - linesDeleted: number; - commits: number; - lastCommitDate: Date; -} - -interface FileStats { - authors: Map; -} - -interface ModuleStats { - files: number; - authors: Map; -} - -interface SiloReport { - module: string; - busFactor: number; - topAuthor: string; - topAuthorPct: number; - totalCommits: number; - totalLines: number; - lastOtherAuthorDate: Date | null; - daysSinceOtherAuthor: number | null; - riskLevel: "high" | "medium" | "low"; -} - -// ── Module Classification ─────────────────────────────────────────────────── - -/** Maps a file path to its logical module name (2-level deep for key dirs). */ -function classifyModule(filePath: string): string | null { - if (!filePath.startsWith("src/")) return null; - - const parts = filePath.replace("src/", "").split("/"); - if (parts.length < 2) return `src/${parts[0]}`; - - const topDir = parts[0]; - - // For commanders/ and managers/, use two levels (e.g. commanders/librarian) - if ( - (topDir === "commanders" || topDir === "managers") && - parts.length >= 2 - ) { - return `src/${topDir}/${parts[1]}`; - } - - // For prompt-smith/codegen vs prompt-smith/other - if (topDir === "prompt-smith" && parts.length >= 2) { - if (parts[1] === "codegen") return "src/prompt-smith/codegen"; - if (parts[1] === "schemas") return "src/prompt-smith/schemas"; - if (parts[1] === "prompt-parts") return "src/prompt-smith/prompt-parts"; - return `src/prompt-smith/${parts[1]}`; - } - - // Everything else: top-level module - return `src/${topDir}`; -} - -// ── Git Log Parsing ───────────────────────────────────────────────────────── - -async function parseGitLog(): Promise> { - const result = - await $`git log --numstat --format="COMMIT:%H|%an|%aI" --no-merges -- "src/**"`.text(); - - const fileStats = new Map(); - - let currentAuthor = ""; - let currentDate = new Date(); - - for (const line of result.split("\n")) { - if (line.startsWith("COMMIT:")) { - const parts = line.replace("COMMIT:", "").split("|"); - currentAuthor = parts[1] ?? "unknown"; - currentDate = new Date(parts[2] ?? ""); - continue; - } - - // numstat lines: \t\t - const numstatMatch = line.match(/^(\d+|-)\t(\d+|-)\t(.+)$/); - if (!numstatMatch) continue; - - const added = numstatMatch[1] === "-" ? 0 : Number(numstatMatch[1]); - const deleted = numstatMatch[2] === "-" ? 0 : Number(numstatMatch[2]); - const filePath = numstatMatch[3]!; - - if (!filePath.startsWith("src/")) continue; - - // Skip git rename paths (e.g. "src/{foo => bar}/file.ts") - if (filePath.includes("=>") || filePath.includes("{")) continue; - - let stats = fileStats.get(filePath); - if (!stats) { - stats = { authors: new Map() }; - fileStats.set(filePath, stats); - } - - let authorStats = stats.authors.get(currentAuthor); - if (!authorStats) { - authorStats = { - commits: 0, - lastCommitDate: currentDate, - linesAdded: 0, - linesDeleted: 0, - }; - stats.authors.set(currentAuthor, authorStats); - } - - authorStats.linesAdded += added; - authorStats.linesDeleted += deleted; - authorStats.commits += 1; - if (currentDate > authorStats.lastCommitDate) { - authorStats.lastCommitDate = currentDate; - } - } - - return fileStats; -} - -// ── Aggregation ───────────────────────────────────────────────────────────── - -function aggregateByModule( - fileStats: Map, -): Map { - const modules = new Map(); - - for (const [filePath, fStats] of fileStats) { - const moduleName = classifyModule(filePath); - if (!moduleName) continue; - - let mod = modules.get(moduleName); - if (!mod) { - mod = { authors: new Map(), files: 0 }; - modules.set(moduleName, mod); - } - mod.files++; - - for (const [author, aStats] of fStats.authors) { - let modAuthor = mod.authors.get(author); - if (!modAuthor) { - modAuthor = { - commits: 0, - lastCommitDate: aStats.lastCommitDate, - linesAdded: 0, - linesDeleted: 0, - }; - mod.authors.set(author, modAuthor); - } - modAuthor.linesAdded += aStats.linesAdded; - modAuthor.linesDeleted += aStats.linesDeleted; - modAuthor.commits += aStats.commits; - if (aStats.lastCommitDate > modAuthor.lastCommitDate) { - modAuthor.lastCommitDate = aStats.lastCommitDate; - } - } - } - - return modules; -} - -// ── Bus Factor Calculation ────────────────────────────────────────────────── - -/** - * Bus factor = minimum number of authors whose combined commit share - * exceeds 50% of the module's total commits. - */ -function computeBusFactor(authors: Map): number { - const totalCommits = [...authors.values()].reduce( - (sum, a) => sum + a.commits, - 0, - ); - if (totalCommits === 0) return 0; - - const sorted = [...authors.entries()].sort( - (a, b) => b[1].commits - a[1].commits, - ); - let accumulated = 0; - let count = 0; - for (const [, stats] of sorted) { - accumulated += stats.commits; - count++; - if (accumulated > totalCommits * 0.5) break; - } - return count; -} - -// ── Silo Detection ───────────────────────────────────────────────────────── - -function detectSilos( - modules: Map, - config: CliArgs, -): SiloReport[] { - const now = new Date(); - const reports: SiloReport[] = []; - - for (const [moduleName, mod] of modules) { - const totalCommits = [...mod.authors.values()].reduce( - (sum, a) => sum + a.commits, - 0, - ); - const totalLines = [...mod.authors.values()].reduce( - (sum, a) => sum + a.linesAdded + a.linesDeleted, - 0, - ); - - if (totalCommits < 5) continue; // skip trivial modules - - const busFactor = computeBusFactor(mod.authors); - - // Find top author by commits - const sorted = [...mod.authors.entries()].sort( - (a, b) => b[1].commits - a[1].commits, - ); - const topEntry = sorted[0]; - if (!topEntry) continue; - const [topAuthor, topStats] = topEntry; - const topAuthorPct = (topStats.commits / totalCommits) * 100; - - // Find last commit date by any non-top author - let lastOtherAuthorDate: Date | null = null; - for (const [author, stats] of mod.authors) { - if (author === topAuthor) continue; - if (!lastOtherAuthorDate || stats.lastCommitDate > lastOtherAuthorDate) { - lastOtherAuthorDate = stats.lastCommitDate; - } - } - - const daysSinceOtherAuthor = lastOtherAuthorDate - ? Math.floor( - (now.getTime() - lastOtherAuthorDate.getTime()) / (1000 * 3600 * 24), - ) - : null; - - // Risk assessment - let riskLevel: SiloReport["riskLevel"] = "low"; - if (topAuthorPct >= config.siloThreshold) { - if ( - daysSinceOtherAuthor === null || - daysSinceOtherAuthor > config.recencyDays - ) { - riskLevel = "high"; - } else { - riskLevel = "medium"; - } - } else if (busFactor <= 1) { - riskLevel = "medium"; - } - - reports.push({ - busFactor, - daysSinceOtherAuthor, - lastOtherAuthorDate, - module: moduleName, - riskLevel, - topAuthor, - topAuthorPct, - totalCommits, - totalLines, - }); - } - - return reports.sort((a, b) => { - const riskOrder = { high: 0, low: 2, medium: 1 }; - if (riskOrder[a.riskLevel] !== riskOrder[b.riskLevel]) { - return riskOrder[a.riskLevel] - riskOrder[b.riskLevel]; - } - return b.topAuthorPct - a.topAuthorPct; - }); -} - -// ── Markdown Report ───────────────────────────────────────────────────────── - -function formatReport(reports: SiloReport[], config: CliArgs): string { - const lines: string[] = []; - - lines.push("# Knowledge Silo Analysis"); - lines.push(""); - lines.push( - `> Generated: ${new Date().toISOString().split("T")[0]} | Silo threshold: ${config.siloThreshold}% | Recency window: ${config.recencyDays} days`, - ); - lines.push(""); - - // ── Bus Factor Table ── - lines.push("## Per-Module Bus Factor"); - lines.push(""); - lines.push( - "| Module | Bus Factor | Top Author | Top % | Commits | Lines |", - ); - lines.push( - "|--------|----------:|-----------:|------:|--------:|------:|", - ); - - for (const r of reports) { - lines.push( - `| ${r.module} | ${r.busFactor} | ${r.topAuthor} | ${r.topAuthorPct.toFixed(1)}% | ${r.totalCommits} | ${r.totalLines} |`, - ); - } - lines.push(""); - - // ── Knowledge Silos ── - const silos = reports.filter((r) => r.riskLevel !== "low"); - - lines.push("## Identified Knowledge Silos"); - lines.push(""); - - if (silos.length === 0) { - lines.push("No knowledge silos detected with current thresholds."); - } else { - for (const s of silos) { - const riskEmoji = - s.riskLevel === "high" ? "🔴" : s.riskLevel === "medium" ? "🟡" : "🟢"; - lines.push(`### ${riskEmoji} ${s.module} — ${s.riskLevel.toUpperCase()}`); - lines.push(""); - lines.push(`- **Top author**: ${s.topAuthor} (${s.topAuthorPct.toFixed(1)}% of commits)`); - lines.push(`- **Bus factor**: ${s.busFactor}`); - lines.push(`- **Total commits**: ${s.totalCommits}`); - if (s.daysSinceOtherAuthor !== null) { - lines.push( - `- **Last other-author commit**: ${s.daysSinceOtherAuthor} days ago`, - ); - } else { - lines.push("- **Last other-author commit**: never"); - } - lines.push(""); - } - } - - // ── Recommendations ── - lines.push("## Recommended Cross-Training Areas"); - lines.push(""); - - const highRisk = reports.filter((r) => r.riskLevel === "high"); - const mediumRisk = reports.filter((r) => r.riskLevel === "medium"); - - if (highRisk.length > 0) { - lines.push("**Priority 1 — Immediate attention:**"); - for (const r of highRisk) { - lines.push( - `- \`${r.module}\`: Only ${r.topAuthor} has meaningful ownership. Pair-program or do code reviews with a second contributor.`, - ); - } - lines.push(""); - } - - if (mediumRisk.length > 0) { - lines.push("**Priority 2 — Monitor:**"); - for (const r of mediumRisk) { - lines.push( - `- \`${r.module}\`: Bus factor is ${r.busFactor}. Encourage contributions from additional team members.`, - ); - } - lines.push(""); - } - - if (highRisk.length === 0 && mediumRisk.length === 0) { - lines.push( - "All modules have adequate contributor diversity. No immediate action needed.", - ); - lines.push(""); - } - - return lines.join("\n"); -} - -// ── Main ──────────────────────────────────────────────────────────────────── - -async function main() { - const config = parseArgs(); - - const fileStats = await parseGitLog(); - const modules = aggregateByModule(fileStats); - - // Filter to modules that still exist on disk - for (const moduleName of [...modules.keys()]) { - if (!existsSync(moduleName)) { - modules.delete(moduleName); - } - } - - const reports = detectSilos(modules, config); - const markdown = formatReport(reports, config); - - console.log(markdown); -} - -main().catch((err) => { - console.error("Failed to run knowledge silo analysis:", err); - process.exit(1); -}); diff --git a/scripts/runbook-cli/commands/check-entry.sh b/scripts/runbook-cli/commands/check-entry.sh new file mode 100755 index 000000000..e59c094c9 --- /dev/null +++ b/scripts/runbook-cli/commands/check-entry.sh @@ -0,0 +1,18 @@ +#!/usr/bin/env bash +set -euo pipefail + +if [[ -n "${BASH_SOURCE[0]:-}" ]]; then + _CMD_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +else + _CMD_DIR="$(cd "$(dirname "$0")" && pwd)" +fi + +source "$_CMD_DIR/../lib.sh" + +path="${1:-}" +if [[ -z "$path" ]]; then + echo "Usage: textfresser-runbook check-entry " + exit 1 +fi + +check_entry_nonempty "$path" diff --git a/scripts/runbook-cli/commands/check-no-nested.sh b/scripts/runbook-cli/commands/check-no-nested.sh new file mode 100755 index 000000000..59397a5ce --- /dev/null +++ b/scripts/runbook-cli/commands/check-no-nested.sh @@ -0,0 +1,12 @@ +#!/usr/bin/env bash +set -euo pipefail + +if [[ -n "${BASH_SOURCE[0]:-}" ]]; then + _CMD_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +else + _CMD_DIR="$(cd "$(dirname "$0")" && pwd)" +fi + +source "$_CMD_DIR/../lib.sh" + +check_no_nested_wikilinks diff --git a/scripts/runbook-cli/commands/doctor.sh b/scripts/runbook-cli/commands/doctor.sh new file mode 100755 index 000000000..d5d6be633 --- /dev/null +++ b/scripts/runbook-cli/commands/doctor.sh @@ -0,0 +1,38 @@ +#!/usr/bin/env bash +set -euo pipefail + +if [[ -n "${BASH_SOURCE[0]:-}" ]]; then + _CMD_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +else + _CMD_DIR="$(cd "$(dirname "$0")" && pwd)" +fi + +source "$_CMD_DIR/../lib.sh" + +print_section "Obsidian process" +if pgrep -l -f "Obsidian.app/Contents/MacOS/Obsidian$" >/dev/null 2>&1; then + pgrep -l -f "Obsidian.app/Contents/MacOS/Obsidian$" +else + echo "Obsidian process not found." +fi + +print_section "CLI and vault" +obsidian version +if ! obsidian vault="$VAULT" vault info=path; then + echo "Vault is not reachable through CLI. Open vault '$VAULT' in Obsidian." + exit 1 +fi + +print_section "Plugin status" +obsidian vault="$VAULT" plugin id="$PLUGIN_ID" +obsidian vault="$VAULT" commands | rg "$PLUGIN_ID" || { + echo "Plugin commands are not visible to CLI." + exit 1 +} + +print_section "IPC smoke test" +obsidian vault="$VAULT" files ext=md | sed -n "1,10p" + +print_section "Troubleshooting hints" +echo "If CLI hangs: reopen Obsidian, ensure vault is open, then rerun preflight." +echo "If command output says Executed but files did not change: run scan-empty and inspect source with read-source." diff --git a/scripts/runbook-cli/commands/lemma.sh b/scripts/runbook-cli/commands/lemma.sh new file mode 100755 index 000000000..15702fd58 --- /dev/null +++ b/scripts/runbook-cli/commands/lemma.sh @@ -0,0 +1,18 @@ +#!/usr/bin/env bash +set -euo pipefail + +if [[ -n "${BASH_SOURCE[0]:-}" ]]; then + _CMD_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +else + _CMD_DIR="$(cd "$(dirname "$0")" && pwd)" +fi + +source "$_CMD_DIR/../lib.sh" + +surface="${1:-}" +if [[ -z "$surface" ]]; then + echo "Usage: textfresser-runbook lemma " + exit 1 +fi + +run_lemma "$surface" diff --git a/scripts/runbook-cli/commands/links-source.sh b/scripts/runbook-cli/commands/links-source.sh new file mode 100755 index 000000000..a4d0505ee --- /dev/null +++ b/scripts/runbook-cli/commands/links-source.sh @@ -0,0 +1,12 @@ +#!/usr/bin/env bash +set -euo pipefail + +if [[ -n "${BASH_SOURCE[0]:-}" ]]; then + _CMD_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +else + _CMD_DIR="$(cd "$(dirname "$0")" && pwd)" +fi + +source "$_CMD_DIR/../lib.sh" + +obsidian vault="$VAULT" links path="$SRC" diff --git a/scripts/runbook-cli/commands/list-entries.sh b/scripts/runbook-cli/commands/list-entries.sh new file mode 100755 index 000000000..798e19f4e --- /dev/null +++ b/scripts/runbook-cli/commands/list-entries.sh @@ -0,0 +1,12 @@ +#!/usr/bin/env bash +set -euo pipefail + +if [[ -n "${BASH_SOURCE[0]:-}" ]]; then + _CMD_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +else + _CMD_DIR="$(cd "$(dirname "$0")" && pwd)" +fi + +source "$_CMD_DIR/../lib.sh" + +list_entries diff --git a/scripts/runbook-cli/commands/preflight.sh b/scripts/runbook-cli/commands/preflight.sh new file mode 100755 index 000000000..96006f1fc --- /dev/null +++ b/scripts/runbook-cli/commands/preflight.sh @@ -0,0 +1,12 @@ +#!/usr/bin/env bash +set -euo pipefail + +if [[ -n "${BASH_SOURCE[0]:-}" ]]; then + _CMD_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +else + _CMD_DIR="$(cd "$(dirname "$0")" && pwd)" +fi + +source "$_CMD_DIR/../lib.sh" + +run_preflight_checks diff --git a/scripts/runbook-cli/commands/read-source.sh b/scripts/runbook-cli/commands/read-source.sh new file mode 100755 index 000000000..a539c3ed3 --- /dev/null +++ b/scripts/runbook-cli/commands/read-source.sh @@ -0,0 +1,12 @@ +#!/usr/bin/env bash +set -euo pipefail + +if [[ -n "${BASH_SOURCE[0]:-}" ]]; then + _CMD_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +else + _CMD_DIR="$(cd "$(dirname "$0")" && pwd)" +fi + +source "$_CMD_DIR/../lib.sh" + +read_source diff --git a/scripts/runbook-cli/commands/reload-plugin.sh b/scripts/runbook-cli/commands/reload-plugin.sh new file mode 100755 index 000000000..b67f70819 --- /dev/null +++ b/scripts/runbook-cli/commands/reload-plugin.sh @@ -0,0 +1,12 @@ +#!/usr/bin/env bash +set -euo pipefail + +if [[ -n "${BASH_SOURCE[0]:-}" ]]; then + _CMD_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +else + _CMD_DIR="$(cd "$(dirname "$0")" && pwd)" +fi + +source "$_CMD_DIR/../lib.sh" + +reload_plugin diff --git a/scripts/runbook-cli/commands/reset-source.sh b/scripts/runbook-cli/commands/reset-source.sh new file mode 100755 index 000000000..5abe759ec --- /dev/null +++ b/scripts/runbook-cli/commands/reset-source.sh @@ -0,0 +1,13 @@ +#!/usr/bin/env bash +set -euo pipefail + +if [[ -n "${BASH_SOURCE[0]:-}" ]]; then + _CMD_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +else + _CMD_DIR="$(cd "$(dirname "$0")" && pwd)" +fi + +source "$_CMD_DIR/../lib.sh" + +reset_source +wait_idle diff --git a/scripts/runbook-cli/commands/reset.sh b/scripts/runbook-cli/commands/reset.sh new file mode 100755 index 000000000..911e4695c --- /dev/null +++ b/scripts/runbook-cli/commands/reset.sh @@ -0,0 +1,17 @@ +#!/usr/bin/env bash +set -euo pipefail + +if [[ -n "${BASH_SOURCE[0]:-}" ]]; then + _CMD_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +else + _CMD_DIR="$(cd "$(dirname "$0")" && pwd)" +fi + +source "$_CMD_DIR/../lib.sh" + +nuke_entries +reset_source +reload_plugin +wait_idle + +echo "Clean slate ready." diff --git a/scripts/runbook-cli/commands/scan-empty.sh b/scripts/runbook-cli/commands/scan-empty.sh new file mode 100755 index 000000000..423aa04af --- /dev/null +++ b/scripts/runbook-cli/commands/scan-empty.sh @@ -0,0 +1,29 @@ +#!/usr/bin/env bash +set -euo pipefail + +if [[ -n "${BASH_SOURCE[0]:-}" ]]; then + _CMD_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +else + _CMD_DIR="$(cd "$(dirname "$0")" && pwd)" +fi + +source "$_CMD_DIR/../lib.sh" + +print_section "Scanning linked dictionary entries" +fail=0 +count=0 +while IFS= read -r path; do + [[ -z "$path" ]] && continue + count=$((count + 1)) + if ! check_entry_nonempty "$path"; then + fail=1 + fi +done < <(collect_linked_entries) + +echo "Checked $count linked entries from $SRC" +if (( fail > 0 )); then + echo "Found empty or missing entries." + exit 1 +fi + +echo "All linked entries are non-empty." diff --git a/scripts/runbook-cli/commands/show-config.sh b/scripts/runbook-cli/commands/show-config.sh new file mode 100755 index 000000000..0cac37d22 --- /dev/null +++ b/scripts/runbook-cli/commands/show-config.sh @@ -0,0 +1,15 @@ +#!/usr/bin/env bash +set -euo pipefail + +if [[ -n "${BASH_SOURCE[0]:-}" ]]; then + _CMD_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +else + _CMD_DIR="$(cd "$(dirname "$0")" && pwd)" +fi + +source "$_CMD_DIR/../lib.sh" + +echo "VAULT=$VAULT" +echo "PLUGIN_ID=$PLUGIN_ID" +echo "SRC=$SRC" +echo "OBSIDIAN_BIN=$OBSIDIAN_BIN" diff --git a/scripts/runbook-cli/commands/wait-idle.sh b/scripts/runbook-cli/commands/wait-idle.sh new file mode 100755 index 000000000..49d4fab65 --- /dev/null +++ b/scripts/runbook-cli/commands/wait-idle.sh @@ -0,0 +1,12 @@ +#!/usr/bin/env bash +set -euo pipefail + +if [[ -n "${BASH_SOURCE[0]:-}" ]]; then + _CMD_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +else + _CMD_DIR="$(cd "$(dirname "$0")" && pwd)" +fi + +source "$_CMD_DIR/../lib.sh" + +wait_idle diff --git a/scripts/runbook-cli/lib.sh b/scripts/runbook-cli/lib.sh new file mode 100644 index 000000000..88ac973ea --- /dev/null +++ b/scripts/runbook-cli/lib.sh @@ -0,0 +1,94 @@ +#!/usr/bin/env bash +set -euo pipefail + +if [[ -n "${BASH_SOURCE[0]:-}" ]]; then + _RUNBOOK_CLI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +else + _RUNBOOK_CLI_DIR="$(cd "$(dirname "$0")" && pwd)" +fi + +export VAULT="${VAULT:-cli-e2e-test-vault}" +export PLUGIN_ID="${PLUGIN_ID:-cbcr-text-eater-de}" +export SRC="${SRC:-textfresser/0_Synthetic_Test_To_Check_Morhp.md}" + +source "$_RUNBOOK_CLI_DIR/../runbook/helpers.sh" + +LEMMA_FIRE_SCRIPT="$_RUNBOOK_CLI_DIR/../runbook/lemma-fire.ts" + +print_section() { + printf "\n== %s ==\n" "$1" +} + +require_command() { + local cmd="$1" + if ! command -v "$cmd" >/dev/null 2>&1; then + echo "Missing required command: $cmd" + return 1 + fi +} + +run_lemma() { + local surface="$1" + bun "$LEMMA_FIRE_SCRIPT" "$surface" +} + +run_lemma_on_src() { + local src_path="$1" + local surface="$2" + SRC="$src_path" bun "$LEMMA_FIRE_SCRIPT" "$surface" +} + +run_preflight_checks() { + local version_output + + require_command obsidian + require_command bun + require_command rg + + print_section "Version" + version_output="$(obsidian version 2>/dev/null || true)" + if [[ -z "$version_output" ]]; then + version_output="$(obsidian vault="$VAULT" version 2>/dev/null || true)" + fi + if [[ -n "$version_output" ]]; then + echo "$version_output" + else + echo "(version output empty)" + fi + + print_section "Vault" + obsidian vault="$VAULT" vault info=path + + print_section "Plugin" + obsidian vault="$VAULT" plugin id="$PLUGIN_ID" + obsidian vault="$VAULT" commands | rg "$PLUGIN_ID" +} + +check_entry_nonempty() { + local path="$1" + local out words + out="$(obsidian vault="$VAULT" wordcount path="$path" &1 || true)" + + if [[ "$out" == Error:* ]]; then + echo "MISSING $path" + return 2 + fi + + words="$(printf "%s\n" "$out" | rg "^words:" | tr -cd '0-9')" + if [[ -z "$words" ]]; then + echo "UNKNOWN $path ($out)" + return 3 + fi + + if (( words > 0 )); then + echo "OK $path ($words words)" + return 0 + fi + + echo "EMPTY $path" + return 1 +} + +collect_linked_entries() { + obsidian vault="$VAULT" links path="$SRC" | rg "^Worter/de/" | sort -u || true +} diff --git a/scripts/runbook-cli/textfresser-runbook b/scripts/runbook-cli/textfresser-runbook new file mode 100755 index 000000000..812a93461 --- /dev/null +++ b/scripts/runbook-cli/textfresser-runbook @@ -0,0 +1,55 @@ +#!/usr/bin/env bash +set -euo pipefail + +if [[ -n "${BASH_SOURCE[0]:-}" ]]; then + _RUNBOOK_CLI_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +else + _RUNBOOK_CLI_DIR="$(cd "$(dirname "$0")" && pwd)" +fi + +usage() { + echo "Textfresser runbook CLI" + echo "" + echo "Usage:" + echo " scripts/runbook-cli/textfresser-runbook [args]" + echo "" + echo "General commands:" + echo " preflight Check CLI/vault/plugin reachability" + echo " doctor Run deeper diagnostics and IPC smoke checks" + echo " show-config Print VAULT/PLUGIN_ID/SRC/OBSIDIAN_BIN" + echo " wait-idle Wait for Textfresser pending work to settle" + echo " lemma Fire Lemma for a surface in SRC" + echo " read-source Read SRC content" + echo " links-source List outgoing links from SRC" + echo " list-entries List Worter/de markdown entries" + echo " check-no-nested Detect nested wikilinks in SRC" + echo " check-entry

Validate that a dictionary entry is non-empty" + echo " scan-empty Check all linked Worter entries for emptiness" + echo " reload-plugin Reload plugin in target vault" + echo " reset-source Recreate SRC fixture and wait for idle" + echo " reset Full clean slate (trash Worter + reset SRC + reload plugin)" + echo "" + echo "Environment overrides:" + echo " VAULT, PLUGIN_ID, SRC, OBSIDIAN_BIN" +} + +cmd="${1:-help}" +shift || true + +case "$cmd" in +help|-h|--help) + usage + ;; +preflight|doctor|show-config|wait-idle|read-source|links-source|list-entries|check-no-nested|scan-empty|reload-plugin|reset-source|reset) + exec "$_RUNBOOK_CLI_DIR/commands/$cmd.sh" "$@" + ;; +lemma|check-entry) + exec "$_RUNBOOK_CLI_DIR/commands/$cmd.sh" "$@" + ;; +*) + echo "Unknown command: $cmd" + echo "" + usage + exit 1 + ;; +esac diff --git a/scripts/runbook/env.sh b/scripts/runbook/env.sh new file mode 100755 index 000000000..399fd3318 --- /dev/null +++ b/scripts/runbook/env.sh @@ -0,0 +1,8 @@ +#!/usr/bin/env bash +# Shared variables for runbook scripts. +# Source this file: source scripts/runbook/env.sh + +export VAULT="${VAULT:-cli-e2e-test-vault}" +export PLUGIN_ID="${PLUGIN_ID:-cbcr-text-eater-de}" +export SRC="${SRC:-Outside/Textfresser-Lemma-Manual.md}" +export OBSIDIAN_BIN="${OBSIDIAN_BIN:-/Applications/Obsidian.app/Contents/MacOS/Obsidian}" diff --git a/scripts/runbook/helpers.sh b/scripts/runbook/helpers.sh new file mode 100755 index 000000000..b24c76f3d --- /dev/null +++ b/scripts/runbook/helpers.sh @@ -0,0 +1,75 @@ +#!/usr/bin/env bash +# Shell helpers for the hands-on runbook. +# Source this file: source scripts/runbook/helpers.sh +# +# Provides: wait_idle, read_source, reset_source, list_entries, read_entry, nuke_entries +# For lemma_fire see lemma-fire.ts (requires bun). + +# Resolve script directory (works in both bash and zsh) +if [[ -n "${BASH_SOURCE[0]:-}" ]]; then + _RUNBOOK_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +elif [[ -n "${(%):-%x}" ]]; then + _RUNBOOK_DIR="$(cd "$(dirname "${(%):-%x}")" && pwd)" +else + _RUNBOOK_DIR="$(cd "$(dirname "$0")" && pwd)" +fi +source "$_RUNBOOK_DIR/env.sh" + +wait_idle() { + "$OBSIDIAN_BIN" vault="$VAULT" eval \ + code="(async()=>{await app.plugins.plugins['$PLUGIN_ID'].whenIdle();return 'idle'})()" +} + +read_source() { + "$OBSIDIAN_BIN" vault="$VAULT" read path="$SRC" +} + +reset_source() { + "$OBSIDIAN_BIN" vault="$VAULT" eval \ + code="(async()=>{const f=app.vault.getAbstractFileByPath('$SRC');if(f)await app.vault.trash(f,true);return 'ok'})()" + "$OBSIDIAN_BIN" vault="$VAULT" create path="$SRC" \ + content="Der Mann liest heute ein Buch.\nDie Katze schlaeft dort.\nEr faengt morgen frueh an.\nDas machen wir auf jeden Fall zusammen.\nDer Plan wirkt klar.\nDer Ablauf bleibt deutlich." silent +} + +list_entries() { + "$OBSIDIAN_BIN" vault="$VAULT" files folder="Worter/de" ext=md +} + +read_entry() { + "$OBSIDIAN_BIN" vault="$VAULT" read path="$1" +} + +nuke_entries() { + "$OBSIDIAN_BIN" vault="$VAULT" eval \ + code="(async()=>{const f=app.vault.getAbstractFileByPath('Worter');if(f)await app.vault.trash(f,true);return 'ok'})()" +} + +reload_plugin() { + "$OBSIDIAN_BIN" vault="$VAULT" plugin:reload id="$PLUGIN_ID" +} + +check_no_nested_wikilinks() { + local content + content=$("$OBSIDIAN_BIN" vault="$VAULT" read path="$SRC" 2>&1) + if echo "$content" | grep -qE '\[\[[^\]]*\[\['; then + echo "FAIL: nested wikilinks detected" + echo "$content" | grep -E '\[\[[^\]]*\[\[' + return 1 + else + echo "OK: no nested wikilinks" + return 0 + fi +} + +check_surface_linked() { + local surface="$1" + local content + content=$("$OBSIDIAN_BIN" vault="$VAULT" read path="$SRC" 2>&1) + if echo "$content" | grep -qE "\[\[.*${surface}.*\]\]"; then + echo "OK: $surface is linked" + return 0 + else + echo "FAIL: $surface is NOT linked" + return 1 + fi +} diff --git a/scripts/runbook/lemma-fire.ts b/scripts/runbook/lemma-fire.ts new file mode 100644 index 000000000..0446a886d --- /dev/null +++ b/scripts/runbook/lemma-fire.ts @@ -0,0 +1,245 @@ +#!/usr/bin/env bun +/** + * Reliable Lemma trigger for manual runbook usage. + * + * Steps: + * 1) Invoke textfresser.executeCommand("Lemma", context) and await result + * 2) wait plugin.whenIdle() + * 3) verify source contains wikilink for surface + * 4) fallback: editor selection + command-id execution + */ + +type LemmaExecResult = { + ok: boolean; + reason: string | null; + strategy: "command-id" | "textfresser"; +}; + +const VAULT = process.env.VAULT ?? "cli-e2e-test-vault"; +const SRC = process.env.SRC ?? "Outside/Textfresser-Lemma-Manual.md"; +const PLUGIN_ID = process.env.PLUGIN_ID ?? "cbcr-text-eater-de"; +const BIN = + process.env.OBSIDIAN_BIN ?? + "/Applications/Obsidian.app/Contents/MacOS/Obsidian"; +const TIMEOUT_MS = Number(process.env.TIMEOUT_MS ?? "20000"); +const LINK_TIMEOUT_MS = Number(process.env.LINK_TIMEOUT_MS ?? "12000"); + +const surface = process.argv[2]; +if (!surface) { + console.error("Usage: bun scripts/runbook/lemma-fire.ts "); + process.exit(1); +} + +function stripCliNoise(text: string): string { + return text + .replace(/.*Loading updated app package.*\n?/g, "") + .replace(/.*Checking for updates.*\n?/g, "") + .trim(); +} + +async function runEval(code: string): Promise { + const proc = Bun.spawn([BIN, `vault=${VAULT}`, "eval", `code=${code}`], { + stderr: "pipe", + stdout: "pipe", + }); + + const timer = setTimeout(() => { + proc.kill(); + }, TIMEOUT_MS); + + const stdout = await new Response(proc.stdout).text(); + const stderr = await new Response(proc.stderr).text(); + const exitCode = await proc.exited; + clearTimeout(timer); + + const out = stripCliNoise(stdout); + const err = stripCliNoise(stderr); + + if (exitCode !== 0) { + throw new Error( + `CLI exited with code ${exitCode}: ${out || ""} ${err || ""}`, + ); + } + if (out.startsWith("Error:")) { + throw new Error(`${out}${err ? `\n${err}` : ""}`); + } + return out.replace(/^=> /, ""); +} + +async function runCli(args: string[]): Promise { + const proc = Bun.spawn([BIN, `vault=${VAULT}`, ...args], { + stderr: "pipe", + stdout: "pipe", + }); + + const timer = setTimeout(() => { + proc.kill(); + }, TIMEOUT_MS); + + const stdout = await new Response(proc.stdout).text(); + const stderr = await new Response(proc.stderr).text(); + const exitCode = await proc.exited; + clearTimeout(timer); + + const out = stripCliNoise(stdout); + const err = stripCliNoise(stderr); + + if (exitCode !== 0) { + throw new Error( + `CLI exited with code ${exitCode}: ${out || ""} ${err || ""}`, + ); + } + if (out.startsWith("Error:")) { + throw new Error(`${out}${err ? `\n${err}` : ""}`); + } + return out; +} + +function isLinked(content: string, s: string): boolean { + return content.includes(`[[${s}]]`) || content.includes(`|${s}]]`); +} + +async function sleep(ms: number): Promise { + await new Promise((resolve) => setTimeout(resolve, ms)); +} + +async function waitForLink(surfaceText: string): Promise { + const start = Date.now(); + while (Date.now() - start < LINK_TIMEOUT_MS) { + const content = await runCli(["read", `path=${SRC}`]); + if (isLinked(content, surfaceText)) { + return true; + } + await sleep(400); + } + return false; +} + +const srcLiteral = JSON.stringify(SRC); +const surfaceLiteral = JSON.stringify(surface); +const pluginLiteral = JSON.stringify(PLUGIN_ID); +const commandIdLiteral = JSON.stringify(`${PLUGIN_ID}:lemma`); + +const waitIdleCode = + `(async()=>{` + + `const plugin=app.plugins.plugins[${pluginLiteral}];` + + `if(!plugin)throw new Error('plugin missing');` + + `await plugin.whenIdle();` + + `return 'idle';` + + `})()`; + +const textfresserExecCode = + `(async()=>{` + + `const plugin=app.plugins.plugins[${pluginLiteral}];` + + `if(!plugin||!plugin.textfresser)return JSON.stringify({ok:false,strategy:'textfresser',reason:'plugin-missing'});` + + `const tf=plugin.textfresser;` + + `const path=${srcLiteral};` + + `const file=app.vault.getAbstractFileByPath(path);` + + `if(!file)return JSON.stringify({ok:false,strategy:'textfresser',reason:'file-missing'});` + + `const content=await app.vault.read(file);` + + `const surface=${surfaceLiteral};` + + `const start=content.indexOf(surface);` + + `if(start===-1)return JSON.stringify({ok:false,strategy:'textfresser',reason:'surface-not-found'});` + + `const lines=content.split('\\\\n');` + + `let lineStart=0;let surroundingRawBlock=null;` + + `for(const line of lines){const lineEnd=lineStart+line.length;if(start>=lineStart&&start<=lineEnd){surroundingRawBlock=line;break}lineStart=lineEnd+1}` + + `if(surroundingRawBlock===null)return JSON.stringify({ok:false,strategy:'textfresser',reason:'block-not-found'});` + + `const parts=path.split('/');const fileName=parts.pop();if(!fileName)return JSON.stringify({ok:false,strategy:'textfresser',reason:'splitpath-file'});` + + `const dot=fileName.lastIndexOf('.');if(dot===-1)return JSON.stringify({ok:false,strategy:'textfresser',reason:'splitpath-ext'});` + + `const splitPath={basename:fileName.slice(0,dot),extension:fileName.slice(dot+1),kind:'MdFile',pathParts:parts};` + + `const result=await tf.executeCommand('Lemma',{activeFile:{content,splitPath},selection:{selectionStartInBlock:start-lineStart,splitPathToFileWithSelection:splitPath,surroundingRawBlock,text:surface}},()=>{});` + + `if(result.isErr&&result.isErr()){const e=result.error||{};return JSON.stringify({ok:false,strategy:'textfresser',reason:String(e.kind||'unknown')+':'+String(e.reason||'')});}` + + `return JSON.stringify({ok:true,strategy:'textfresser',reason:null});` + + `})()`; + +const commandRouteExecCode = + `(async()=>{` + + `const path=${srcLiteral};` + + `const selectedText=${surfaceLiteral};` + + `const file=app.vault.getAbstractFileByPath(path);` + + `if(!file)return JSON.stringify({ok:false,strategy:'command-id',reason:'file-missing'});` + + `const leaf=app.workspace.getMostRecentLeaf()??app.workspace.getLeaf(true);` + + `await leaf.openFile(file,{active:true});` + + `const view=leaf.view;` + + `if(view&&typeof view.getMode==='function'&&typeof view.setMode==='function'&&view.getMode()!=='source'){await view.setMode('source')}` + + `const editor=(view&&'editor' in view&&view.editor)?view.editor:app.workspace.activeEditor?.editor;` + + `if(!editor)return JSON.stringify({ok:false,strategy:'command-id',reason:'no-editor'});` + + `const content=editor.getValue();` + + `const start=content.indexOf(selectedText);` + + `if(start===-1)return JSON.stringify({ok:false,strategy:'command-id',reason:'surface-not-found'});` + + `const offsetToPos=(offset)=>{const prefix=content.slice(0,offset);const lines=prefix.split('\\\\n');const line=lines.length-1;const ch=lines[line]?.length??0;return {line,ch}};` + + `const from=offsetToPos(start);const to=offsetToPos(start+selectedText.length);` + + `editor.setSelection(from,to);` + + `if(typeof editor.focus==='function'){editor.focus();}` + + `await new Promise((resolve)=>setTimeout(resolve,50));` + + `const ok=app.commands.executeCommandById(${commandIdLiteral});` + + `if(!ok)return JSON.stringify({ok:false,strategy:'command-id',reason:'command-failed'});` + + `return JSON.stringify({ok:true,strategy:'command-id',reason:null});` + + `})()`; + +async function executeByStrategy( + strategy: "command-id" | "textfresser", +): Promise { + const raw = + strategy === "textfresser" + ? await runEval(textfresserExecCode) + : await runEval(commandRouteExecCode); + + if (!raw) { + return { + ok: false, + reason: "empty-cli-response", + strategy, + }; + } + + try { + return JSON.parse(raw) as LemmaExecResult; + } catch { + return { + ok: false, + reason: `unexpected-response:${raw}`, + strategy, + }; + } +} + +const beforeContent = await runCli(["read", `path=${SRC}`]); +const beforeLinked = isLinked(beforeContent, surface); +if (beforeLinked) { + console.log("fired (already-linked)"); + process.exit(0); +} + +const firstTry = await executeByStrategy("textfresser"); +if (!firstTry.ok && firstTry.reason !== "empty-cli-response") { + console.error( + `lemma-fire first strategy failed [${firstTry.strategy}]: ${firstTry.reason ?? "unknown"}`, + ); +} + +try { + await runEval(waitIdleCode); +} catch {} +if (await waitForLink(surface)) { + console.log("fired (textfresser)"); + process.exit(0); +} + +const fallbackTry = await executeByStrategy("command-id"); +if (!fallbackTry.ok && fallbackTry.reason !== "empty-cli-response") { + console.error( + `lemma-fire fallback failed [${fallbackTry.strategy}]: ${fallbackTry.reason ?? "unknown"}`, + ); +} + +try { + await runEval(waitIdleCode); +} catch {} +if (await waitForLink(surface)) { + console.log("fired (command-id)"); + process.exit(0); +} + +console.error("lemma-fire failed: command(s) executed but source remains unlinked"); +process.exit(1); diff --git a/scripts/typecheck-changed.sh b/scripts/typecheck-changed.sh index 7ad59c346..119edf7da 100755 --- a/scripts/typecheck-changed.sh +++ b/scripts/typecheck-changed.sh @@ -1,11 +1,55 @@ #!/bin/bash # Typecheck only files changed vs master -CHANGED=$(git diff --name-only master...HEAD -- '*.ts' '*.tsx' | tr '\n' '|' | sed 's/|$//') +set -u -if [ -z "$CHANGED" ]; then +CHANGED_FILE_LIST=$(git diff --name-only master...HEAD -- '*.ts' '*.tsx') + +if [ -z "$CHANGED_FILE_LIST" ]; then echo "No TypeScript files changed vs master" exit 0 fi -bun x tsc --noEmit 2>&1 | grep -E "^($CHANGED)" +CHANGED_FILES_TMP=$(mktemp) +trap 'rm -f "$CHANGED_FILES_TMP"' EXIT +printf '%s\n' "$CHANGED_FILE_LIST" > "$CHANGED_FILES_TMP" + +TSC_OUTPUT=$(bun x tsc --noEmit --pretty false 2>&1) +TSC_EXIT=$? + +if [ $TSC_EXIT -eq 0 ]; then + echo "TypeScript check passed" + exit 0 +fi + +FILTERED_OUTPUT=$( + printf '%s\n' "$TSC_OUTPUT" | awk ' + NR == FNR { + files[$0] = 1 + next + } + { + for (file in files) { + if (index($0, file "(") == 1 || index($0, file ":") == 1) { + print + found = 1 + break + } + } + } + END { + if (!found) { + exit 1 + } + } + ' "$CHANGED_FILES_TMP" - +) +FILTER_EXIT=$? + +if [ $FILTER_EXIT -eq 0 ]; then + printf '%s\n' "$FILTERED_OUTPUT" + exit 1 +fi + +echo "TypeScript has errors, but none are in files changed vs master." +exit 0 diff --git a/src/commanders/librarian/bookkeeper/segmenter/stream/sentence-segmenter.ts b/src/commanders/librarian/bookkeeper/segmenter/stream/sentence-segmenter.ts index dfb47b033..1a1535520 100644 --- a/src/commanders/librarian/bookkeeper/segmenter/stream/sentence-segmenter.ts +++ b/src/commanders/librarian/bookkeeper/segmenter/stream/sentence-segmenter.ts @@ -105,7 +105,7 @@ function countZwsBefore(positions: number[], index: number): number { /** * Segments text preserving paragraph structure. - * Returns sentences with paragraph boundary markers (legacy version). + * Returns sentences with paragraph boundary markers. */ export function segmentWithParagraphs( content: string, diff --git a/src/commanders/librarian/codecs/index.ts b/src/commanders/librarian/codecs/index.ts index 4cb46bfef..865ffd5ac 100644 --- a/src/commanders/librarian/codecs/index.ts +++ b/src/commanders/librarian/codecs/index.ts @@ -80,7 +80,6 @@ export type { } from "./split-path-inside-library"; export type { AnyCanonicalSplitPathInsideLibrary, - CanonicalSplitPathInsideLibrary, CanonicalSplitPathInsideLibraryOf, CanonicalSplitPathToFileInsideLibrary, CanonicalSplitPathToFolderInsideLibrary, diff --git a/src/commanders/librarian/codecs/library-path/index.ts b/src/commanders/librarian/codecs/library-path/index.ts index 5db612d48..f534e8a33 100644 --- a/src/commanders/librarian/codecs/library-path/index.ts +++ b/src/commanders/librarian/codecs/library-path/index.ts @@ -26,7 +26,7 @@ export type LibraryPath = { readonly segments: readonly string[]; /** File extension if this is a file, undefined for folders */ readonly extension?: string; - /** SplitPath kind for compatibility */ + /** SplitPath kind */ readonly kind: SplitPathKind; }; diff --git a/src/commanders/librarian/codecs/locator/index.ts b/src/commanders/librarian/codecs/locator/index.ts index 2d3271690..488f4b6ad 100644 --- a/src/commanders/librarian/codecs/locator/index.ts +++ b/src/commanders/librarian/codecs/locator/index.ts @@ -1,8 +1,8 @@ export type { LocatorCodecs } from "./make"; export { makeLocatorCodecs } from "./make"; export type { + AnyCanonicalSplitPathInsideLibrary, AnyNodeLocator, - CanonicalSplitPathInsideLibrary, CanonicalSplitPathToFileInsideLibrary, CanonicalSplitPathToFolderInsideLibrary, CanonicalSplitPathToMdFileInsideLibrary, diff --git a/src/commanders/librarian/codecs/locator/internal/from.ts b/src/commanders/librarian/codecs/locator/internal/from.ts index 9f85292de..4052f2d54 100644 --- a/src/commanders/librarian/codecs/locator/internal/from.ts +++ b/src/commanders/librarian/codecs/locator/internal/from.ts @@ -5,7 +5,13 @@ import { makeNodeSegmentId } from "../../../healer/library-tree/tree-node/codecs import { TreeNodeKind } from "../../../healer/library-tree/tree-node/types/atoms"; import type { CodecError } from "../../errors"; import { makeLocatorError } from "../../errors"; -import type { SegmentIdCodecs } from "../../segment-id"; +import type { + FileNodeSegmentId, + ScrollNodeSegmentId, + SectionNodeSegmentId, + SectionNodeSegmentIdChain, + SegmentIdCodecs, +} from "../../segment-id"; import type { AnyCanonicalSplitPathInsideLibrary, CanonicalSplitPathInsideLibraryOf, @@ -26,7 +32,7 @@ type SegmentIdForKind = { function serializeAndBuildLocator( segmentId: SegmentIdCodecs, components: { coreName: string; targetKind: NK; extension?: string }, - segmentIdChainToParent: ReturnType[], + segmentIdChainToParent: SectionNodeSegmentIdChain, errorContext: Record, ): Result< { @@ -84,12 +90,14 @@ export function canonicalSplitPathInsideLibraryToLocator( sp: AnyCanonicalSplitPathInsideLibrary, ): Result { // Both pathParts and segmentIdChainToParent INCLUDE Library root - const segmentIdChainToParent = sp.pathParts.map((nodeName) => - makeNodeSegmentId({ - children: {}, - kind: TreeNodeKind.Section, - nodeName, - }), + const segmentIdChainToParent: SectionNodeSegmentIdChain = sp.pathParts.map( + (nodeName) => + makeNodeSegmentId({ + children: {}, + kind: TreeNodeKind.Section, + // pathParts are section names by codec invariant + nodeName: nodeName as (typeof sp.pathParts)[number], + }) as SectionNodeSegmentId, ); switch (sp.kind) { diff --git a/src/commanders/librarian/codecs/locator/types.ts b/src/commanders/librarian/codecs/locator/types.ts index 264b44b68..e2622cea1 100644 --- a/src/commanders/librarian/codecs/locator/types.ts +++ b/src/commanders/librarian/codecs/locator/types.ts @@ -1,5 +1,5 @@ import type { - CanonicalSplitPathInsideLibrary, + AnyCanonicalSplitPathInsideLibrary, CanonicalSplitPathToFileInsideLibrary, CanonicalSplitPathToFolderInsideLibrary, CanonicalSplitPathToMdFileInsideLibrary, @@ -23,7 +23,7 @@ export type { SectionNodeLocator, ScrollNodeLocator, FileNodeLocator, - CanonicalSplitPathInsideLibrary, + AnyCanonicalSplitPathInsideLibrary, CanonicalSplitPathToFileInsideLibrary, CanonicalSplitPathToFolderInsideLibrary, CanonicalSplitPathToMdFileInsideLibrary, diff --git a/src/commanders/librarian/codecs/split-path-with-separated-suffix/index.ts b/src/commanders/librarian/codecs/split-path-with-separated-suffix/index.ts index 2fabdbe08..2872e7cf6 100644 --- a/src/commanders/librarian/codecs/split-path-with-separated-suffix/index.ts +++ b/src/commanders/librarian/codecs/split-path-with-separated-suffix/index.ts @@ -3,8 +3,6 @@ export { makeSplitPathWithSeparatedSuffixCodecs } from "./make"; export type { AnyCanonicalSplitPathInsideLibrary, CanonicalSeparatedSuffixedBasename, - // Legacy alias - CanonicalSplitPathInsideLibrary, CanonicalSplitPathInsideLibraryOf, CanonicalSplitPathToFileInsideLibrary, CanonicalSplitPathToFolderInsideLibrary, diff --git a/src/commanders/librarian/codecs/split-path-with-separated-suffix/types.ts b/src/commanders/librarian/codecs/split-path-with-separated-suffix/types.ts index 0abbf28bf..0b5361816 100644 --- a/src/commanders/librarian/codecs/split-path-with-separated-suffix/types.ts +++ b/src/commanders/librarian/codecs/split-path-with-separated-suffix/types.ts @@ -75,7 +75,3 @@ export type AnyCanonicalSplitPathInsideLibrary = | CanonicalSplitPathToFolderInsideLibrary | CanonicalSplitPathToFileInsideLibrary | CanonicalSplitPathToMdFileInsideLibrary; - -// Legacy alias for backward compatibility -export type CanonicalSplitPathInsideLibrary = - AnyCanonicalSplitPathInsideLibrary; diff --git a/src/commanders/librarian/commands/index.ts b/src/commanders/librarian/commands/index.ts index 67289f697..0c52e9c3d 100644 --- a/src/commanders/librarian/commands/index.ts +++ b/src/commanders/librarian/commands/index.ts @@ -8,5 +8,4 @@ export const commandFnForCommandKind = { [LibrarianCommandKind.GoToPrevPage]: goToPrevPageCommand, [LibrarianCommandKind.SplitInBlocks]: splitInBlocksCommand, [LibrarianCommandKind.SplitToPages]: splitToPagesCommand, - [LibrarianCommandKind.MakeText]: splitToPagesCommand, // legacy alias } satisfies Record; diff --git a/src/commanders/librarian/commands/split-in-blocks.ts b/src/commanders/librarian/commands/split-in-blocks.ts index 1f8631683..9f67de88a 100644 --- a/src/commanders/librarian/commands/split-in-blocks.ts +++ b/src/commanders/librarian/commands/split-in-blocks.ts @@ -1,4 +1,4 @@ -import { err, ok, ResultAsync } from "neverthrow"; +import { errAsync, okAsync, ResultAsync } from "neverthrow"; import { type VaultAction, VaultActionKind, @@ -20,9 +20,7 @@ export const splitInBlocksCommand: LibrarianCommandFn = (input) => { if (!selection?.text?.trim()) { notify("No text selected"); - return ResultAsync.fromResult( - err({ kind: CommandErrorKind.NoSelection }), - ); + return errAsync({ kind: CommandErrorKind.NoSelection }); } const highestBlockNumber = blockIdHelper.findHighestNumber( @@ -51,14 +49,12 @@ export const splitInBlocksCommand: LibrarianCommandFn = (input) => { if (result.isErr()) { const reason = result.error.map((e) => e.error).join(", "); notify(`Error: ${reason}`); - return ResultAsync.fromResult( - err({ - kind: CommandErrorKind.DispatchFailed, - reason, - }), - ); + return errAsync({ + kind: CommandErrorKind.DispatchFailed, + reason, + }); } notify(`Split into ${blockCount} blocks`); - return ResultAsync.fromResult(ok(undefined)); + return okAsync(undefined); }); }; diff --git a/src/commanders/librarian/commands/types.ts b/src/commanders/librarian/commands/types.ts index 7f12aaa7f..2e696ded5 100644 --- a/src/commanders/librarian/commands/types.ts +++ b/src/commanders/librarian/commands/types.ts @@ -20,7 +20,6 @@ const LIBRARIAN_COMMAND_KIND_STR = [ "GoToPrevPage", "SplitInBlocks", "SplitToPages", - "MakeText", ] as const; export const LibrarianCommandKindSchema = z.enum(LIBRARIAN_COMMAND_KIND_STR); diff --git a/src/commanders/librarian/healer/library-tree/codex/backlink-transforms.ts b/src/commanders/librarian/healer/library-tree/codex/backlink-transforms.ts index c0dec6edf..97c78fa33 100644 --- a/src/commanders/librarian/healer/library-tree/codex/backlink-transforms.ts +++ b/src/commanders/librarian/healer/library-tree/codex/backlink-transforms.ts @@ -1,9 +1,6 @@ /** * Transforms for processing backlinks and content. * Used by ProcessMdFile actions to update codexes and scrolls. - * - * NOTE: This file re-exports from transforms/ subdirectory for backward compatibility. - * New code should import directly from transforms/codex-transforms or transforms/scroll-transforms. */ export { diff --git a/src/commanders/librarian/healer/library-tree/codex/codex-split-path.ts b/src/commanders/librarian/healer/library-tree/codex/codex-split-path.ts index d8ac2c0dd..ed026847e 100644 --- a/src/commanders/librarian/healer/library-tree/codex/codex-split-path.ts +++ b/src/commanders/librarian/healer/library-tree/codex/codex-split-path.ts @@ -1,8 +1,5 @@ /** * Compute the canonical split path for a section's codex file. - * - * NOTE: This file is a thin wrapper around PathFinder.buildCodexSplitPath - * for backward compatibility. New code should import from paths/path-finder directly. */ import type { Codecs, SplitPathToMdFileInsideLibrary } from "../../../codecs"; @@ -12,7 +9,7 @@ import { PREFIX_OF_CODEX } from "./literals"; /** * Compute codex split path from section chain. - * Throws on error for backward compatibility. + * Throws on error. * * @param sectionChain - Full chain including Library root, e.g. ["Library﹘Section﹘", "A﹘Section﹘"] * @param codecs - Codec API for parsing segment IDs diff --git a/src/commanders/librarian/healer/library-tree/codex/parse-codex-click.ts b/src/commanders/librarian/healer/library-tree/codex/parse-codex-click.ts index 17bc51741..cedb8bc1a 100644 --- a/src/commanders/librarian/healer/library-tree/codex/parse-codex-click.ts +++ b/src/commanders/librarian/healer/library-tree/codex/parse-codex-click.ts @@ -8,7 +8,6 @@ import { getParsedUserSettings } from "../../../../../global-state/global-state" import { makeCodecRulesFromSettings, makeCodecs } from "../../../codecs"; import type { SectionNodeSegmentId } from "../../../codecs/segment-id"; import { NodeSegmentIdSeparator } from "../../../codecs/segment-id/types/segment-id"; -import { adaptCodecResult } from "../tree-action/bulk-vault-action-adapter/layers/translate-material-event/error-adapters"; import { TreeNodeKind } from "../tree-node/types/atoms"; import { PREFIX_OF_CODEX } from "./literals"; @@ -70,9 +69,9 @@ export function parseCodexLinkTarget( const codecs = makeCodecs(rules); // Parse as suffixed basename - const parseResult = adaptCodecResult( - codecs.suffix.parseSeparatedSuffix(linkTarget), - ); + const parseResult = codecs.suffix + .parseSeparatedSuffix(linkTarget) + .mapErr((error) => error.message); if (parseResult.isErr()) { return err(`Failed to parse link target: ${parseResult.error}`); diff --git a/src/commanders/librarian/healer/library-tree/tree-action/bulk-vault-action-adapter/layers/library-scope/codecs/events/make-event-vault-scoped.ts b/src/commanders/librarian/healer/library-tree/tree-action/bulk-vault-action-adapter/layers/library-scope/codecs/events/make-event-vault-scoped.ts index 91e3d18ea..d5b4f7e93 100644 --- a/src/commanders/librarian/healer/library-tree/tree-action/bulk-vault-action-adapter/layers/library-scope/codecs/events/make-event-vault-scoped.ts +++ b/src/commanders/librarian/healer/library-tree/tree-action/bulk-vault-action-adapter/layers/library-scope/codecs/events/make-event-vault-scoped.ts @@ -1,7 +1,7 @@ import type { CodecRules } from "../../../../../../../../codecs/rules"; import type { AnySplitPathInsideLibrary } from "../../../../../../../../codecs/split-path-inside-library/types/split-path-inside-library"; import { visitInsideEvent } from "../../helpers/scoped-event-helpers"; -import type { DescopedEvent } from "../../types/generics"; +import type { DescopedEvent } from "../../types/generics/scoped-event"; import type { LibraryScopedVaultEvent } from "../../types/scoped-event"; import { makeVaultScopedSplitPath } from "../split-path-inside-the-library"; diff --git a/src/commanders/librarian/healer/library-tree/tree-action/bulk-vault-action-adapter/layers/translate-material-event/error-adapters.ts b/src/commanders/librarian/healer/library-tree/tree-action/bulk-vault-action-adapter/layers/translate-material-event/error-adapters.ts deleted file mode 100644 index 341acc1d3..000000000 --- a/src/commanders/librarian/healer/library-tree/tree-action/bulk-vault-action-adapter/layers/translate-material-event/error-adapters.ts +++ /dev/null @@ -1,27 +0,0 @@ -import type { Result } from "neverthrow"; -import type { CodecError } from "../../../../../../codecs/errors"; - -/** - * Temporary adapter to convert CodecError to string for migration. - * TODO: Remove after full migration to CodecError. - */ -export function codecErrorToString(error: CodecError): string { - switch (error.kind) { - case "SegmentIdError": - case "SuffixError": - case "SplitPathError": - case "LocatorError": - case "ZodError": - return error.message; - } -} - -/** - * Temporary adapter to convert Result to Result. - * TODO: Remove after full migration to CodecError. - */ -export function adaptCodecResult( - result: Result, -): Result { - return result.mapErr(codecErrorToString); -} diff --git a/src/commanders/librarian/healer/library-tree/tree-action/bulk-vault-action-adapter/layers/translate-material-event/policy-and-intent/intent/infer-intent.ts b/src/commanders/librarian/healer/library-tree/tree-action/bulk-vault-action-adapter/layers/translate-material-event/policy-and-intent/intent/infer-intent.ts index 126d80e9d..cb0d0b7bd 100644 --- a/src/commanders/librarian/healer/library-tree/tree-action/bulk-vault-action-adapter/layers/translate-material-event/policy-and-intent/intent/infer-intent.ts +++ b/src/commanders/librarian/healer/library-tree/tree-action/bulk-vault-action-adapter/layers/translate-material-event/policy-and-intent/intent/infer-intent.ts @@ -1,7 +1,6 @@ import type { Codecs } from "../../../../../../../../codecs"; import { TreeNodeKind } from "../../../../../../tree-node/types/atoms"; import type { RenameTreeNodeNodeMaterializedEvent } from "../../../materialized-node-events/types"; -import { adaptCodecResult } from "../../error-adapters"; import { RenameIntent } from "./types"; /** @@ -71,9 +70,9 @@ export function inferRenameIntent( if (!basenameChanged) return RenameIntent.Move; // basename changed: check if it's a "move-by-name" or just a rename - const sepRes = adaptCodecResult( - codecs.suffix.parseSeparatedSuffix(to.basename), - ); + const sepRes = codecs.suffix + .parseSeparatedSuffix(to.basename) + .mapErr((error) => error.message); // if invalid basename, treat as plain rename (will be rejected/healed later anyway) if (sepRes.isErr()) return RenameIntent.Rename; diff --git a/src/commanders/librarian/healer/library-tree/tree-action/bulk-vault-action-adapter/layers/translate-material-event/translate-material-events.ts b/src/commanders/librarian/healer/library-tree/tree-action/bulk-vault-action-adapter/layers/translate-material-event/translate-material-events.ts index 7f24d4a08..48908efb1 100644 --- a/src/commanders/librarian/healer/library-tree/tree-action/bulk-vault-action-adapter/layers/translate-material-event/translate-material-events.ts +++ b/src/commanders/librarian/healer/library-tree/tree-action/bulk-vault-action-adapter/layers/translate-material-event/translate-material-events.ts @@ -5,7 +5,6 @@ import { MaterializedEventKind as MaterializedEventType, type MaterializedNodeEvent, } from "../materialized-node-events/types"; -import { adaptCodecResult } from "./error-adapters"; import { traslateCreateMaterializedEvent } from "./translators/translate-create-material-event"; import { traslateDeleteMaterializedEvent } from "./translators/translate-delete-material-event"; import { traslateRenameMaterializedEvent } from "./translators/traslate-rename-materila-event"; @@ -72,8 +71,8 @@ export const translateMaterializedEvents = ( function isCodexEvent(ev: MaterializedNodeEvent, codecs: Codecs): boolean { const splitPath = ev.kind === MaterializedEventType.Rename ? ev.to : ev.splitPath; - const result = adaptCodecResult( - codecs.suffix.parseSeparatedSuffix(splitPath.basename), - ); + const result = codecs.suffix + .parseSeparatedSuffix(splitPath.basename) + .mapErr((error) => error.message); return result.isOk() && result.value.coreName === PREFIX_OF_CODEX; } diff --git a/src/commanders/librarian/healer/library-tree/tree-action/bulk-vault-action-adapter/layers/translate-material-event/translators/helpers/event-to-locator.ts b/src/commanders/librarian/healer/library-tree/tree-action/bulk-vault-action-adapter/layers/translate-material-event/translators/helpers/event-to-locator.ts index c92feaa4c..5acc86f27 100644 --- a/src/commanders/librarian/healer/library-tree/tree-action/bulk-vault-action-adapter/layers/translate-material-event/translators/helpers/event-to-locator.ts +++ b/src/commanders/librarian/healer/library-tree/tree-action/bulk-vault-action-adapter/layers/translate-material-event/translators/helpers/event-to-locator.ts @@ -12,7 +12,6 @@ import { type MaterializedNodeEvent, type TreeNodeLocatorForEvent, } from "../../../materialized-node-events/types"; -import { adaptCodecResult } from "../../error-adapters"; import { inferPolicyAndIntent } from "../../policy-and-intent/infer-policy-and-intent"; export function tryMakeDestinationLocatorFromEvent< @@ -21,9 +20,9 @@ export function tryMakeDestinationLocatorFromEvent< const cspRes = tryMakeCanonicalSplitPathToDestination(ev, codecs); if (cspRes.isErr()) return err(cspRes.error); - const locatorRes = adaptCodecResult( - codecs.locator.canonicalSplitPathInsideLibraryToLocator(cspRes.value), - ); + const locatorRes = codecs.locator + .canonicalSplitPathInsideLibraryToLocator(cspRes.value) + .mapErr((error) => error.message); if (locatorRes.isErr()) return err(locatorRes.error); // Cast is safe: locator type corresponds to event's split path kind @@ -37,11 +36,9 @@ const tryMakeCanonicalSplitPathToDestination = < codecs: Codecs, ): Result, string> => { if (ev.kind === MaterializedEventKind.Delete) { - const withSeparatedSuffixResult = adaptCodecResult( - codecs.splitPathWithSeparatedSuffix.splitPathInsideLibraryToWithSeparatedSuffix( - ev.splitPath, - ), - ); + const withSeparatedSuffixResult = codecs.splitPathWithSeparatedSuffix + .splitPathInsideLibraryToWithSeparatedSuffix(ev.splitPath) + .mapErr((error) => error.message); if (withSeparatedSuffixResult.isErr()) { return err(withSeparatedSuffixResult.error) as Result< CanonicalSplitPathToDestination, @@ -50,13 +47,11 @@ const tryMakeCanonicalSplitPathToDestination = < } const { splitPathToLibraryRoot } = getParsedUserSettings(); const libraryRootName = splitPathToLibraryRoot.basename; - const r = adaptCodecResult( - canonizeSplitPathWithSeparatedSuffix( - codecs.suffix, - libraryRootName, - withSeparatedSuffixResult.value, - ), - ); + const r = canonizeSplitPathWithSeparatedSuffix( + codecs.suffix, + libraryRootName, + withSeparatedSuffixResult.value, + ).mapErr((error) => error.message); return r as Result, string>; } diff --git a/src/commanders/librarian/healer/library-tree/tree-action/bulk-vault-action-adapter/layers/translate-material-event/translators/helpers/split-path-to-locator.ts b/src/commanders/librarian/healer/library-tree/tree-action/bulk-vault-action-adapter/layers/translate-material-event/translators/helpers/split-path-to-locator.ts index 12b258dc5..e0b7b815d 100644 --- a/src/commanders/librarian/healer/library-tree/tree-action/bulk-vault-action-adapter/layers/translate-material-event/translators/helpers/split-path-to-locator.ts +++ b/src/commanders/librarian/healer/library-tree/tree-action/bulk-vault-action-adapter/layers/translate-material-event/translators/helpers/split-path-to-locator.ts @@ -7,7 +7,6 @@ import type { } from "../../../../../../../../codecs"; import { canonizeSplitPathWithSeparatedSuffix } from "../../../../../utils/canonical-naming/canonicalization-policy"; import type { TreeNodeLocatorForLibraryScopedSplitPath } from "../../../materialized-node-events/types"; -import { adaptCodecResult } from "../../error-adapters"; export function tryMakeTargetLocatorFromLibraryScopedSplitPath< SK extends SplitPathKind, @@ -15,11 +14,9 @@ export function tryMakeTargetLocatorFromLibraryScopedSplitPath< sp: SplitPathInsideLibraryOf, codecs: Codecs, ): Result, string> { - const withSeparatedSuffixResult = adaptCodecResult( - codecs.splitPathWithSeparatedSuffix.splitPathInsideLibraryToWithSeparatedSuffix( - sp, - ), - ); + const withSeparatedSuffixResult = codecs.splitPathWithSeparatedSuffix + .splitPathInsideLibraryToWithSeparatedSuffix(sp) + .mapErr((error) => error.message); if (withSeparatedSuffixResult.isErr()) { return err(withSeparatedSuffixResult.error); @@ -27,18 +24,16 @@ export function tryMakeTargetLocatorFromLibraryScopedSplitPath< const { splitPathToLibraryRoot } = getParsedUserSettings(); const libraryRootName = splitPathToLibraryRoot.basename; - const cspRes = adaptCodecResult( - canonizeSplitPathWithSeparatedSuffix( - codecs.suffix, - libraryRootName, - withSeparatedSuffixResult.value, - ), - ); + const cspRes = canonizeSplitPathWithSeparatedSuffix( + codecs.suffix, + libraryRootName, + withSeparatedSuffixResult.value, + ).mapErr((error) => error.message); if (cspRes.isErr()) return err(cspRes.error); - const locatorRes = adaptCodecResult( - codecs.locator.canonicalSplitPathInsideLibraryToLocator(cspRes.value), - ); + const locatorRes = codecs.locator + .canonicalSplitPathInsideLibraryToLocator(cspRes.value) + .mapErr((error) => error.message); if (locatorRes.isErr()) return err(locatorRes.error); // Cast is safe: locator type corresponds to split path kind diff --git a/src/commanders/librarian/healer/library-tree/tree-action/utils/canonical-naming/canonical-split-path-codec.ts b/src/commanders/librarian/healer/library-tree/tree-action/utils/canonical-naming/canonical-split-path-codec.ts index 3f30456bc..d01579339 100644 --- a/src/commanders/librarian/healer/library-tree/tree-action/utils/canonical-naming/canonical-split-path-codec.ts +++ b/src/commanders/librarian/healer/library-tree/tree-action/utils/canonical-naming/canonical-split-path-codec.ts @@ -2,8 +2,8 @@ import { err, ok, type Result } from "neverthrow"; import { getParsedUserSettings } from "../../../../../../../global-state/global-state"; import type { AnySplitPath } from "../../../../../../../managers/obsidian/vault-action-manager/types/split-path"; import type { + AnyCanonicalSplitPathInsideLibrary, AnySplitPathInsideLibrary, - CanonicalSplitPathInsideLibrary, CanonicalSplitPathToFileInsideLibrary, CanonicalSplitPathToFolderInsideLibrary, CanonicalSplitPathToMdFileInsideLibrary, @@ -26,10 +26,10 @@ export function tryParseCanonicalSplitPathInsideLibrary( ): Result; export function tryParseCanonicalSplitPathInsideLibrary( sp: AnySplitPathInsideLibrary, -): Result; +): Result; export function tryParseCanonicalSplitPathInsideLibrary( sp: AnySplitPathInsideLibrary, -): Result { +): Result { const pathPartsRes = tryParsePathParts(sp.pathParts); if (pathPartsRes.isErr()) return err(pathPartsRes.error); @@ -61,7 +61,7 @@ export function tryParseCanonicalSplitPathInsideLibrary( return err(canonizedResult.error.message); } - return ok(canonizedResult.value as CanonicalSplitPathInsideLibrary); + return ok(canonizedResult.value as AnyCanonicalSplitPathInsideLibrary); } function tryParsePathParts( @@ -97,10 +97,10 @@ export function makeRegularSplitPathInsideLibrary( sp: CanonicalSplitPathToMdFileInsideLibrary, ): SplitPathToMdFileInsideLibrary; export function makeRegularSplitPathInsideLibrary( - sp: CanonicalSplitPathInsideLibrary, + sp: AnyCanonicalSplitPathInsideLibrary, ): AnySplitPathInsideLibrary; export function makeRegularSplitPathInsideLibrary( - sp: CanonicalSplitPathInsideLibrary, + sp: AnyCanonicalSplitPathInsideLibrary, ): AnySplitPathInsideLibrary { const settings = getParsedUserSettings(); const rules = makeCodecRulesFromSettings(settings); diff --git a/src/commanders/librarian/healer/library-tree/tree-action/utils/canonical-naming/canonicalize-to-destination.ts b/src/commanders/librarian/healer/library-tree/tree-action/utils/canonical-naming/canonicalize-to-destination.ts index 1116deebf..c4fb1ac9e 100644 --- a/src/commanders/librarian/healer/library-tree/tree-action/utils/canonical-naming/canonicalize-to-destination.ts +++ b/src/commanders/librarian/healer/library-tree/tree-action/utils/canonical-naming/canonicalize-to-destination.ts @@ -2,15 +2,14 @@ import { err, ok, type Result } from "neverthrow"; import { getParsedUserSettings } from "../../../../../../../global-state/global-state"; import type { SplitPathKind } from "../../../../../../../managers/obsidian/vault-action-manager/types/split-path"; import type { + AnyCanonicalSplitPathInsideLibrary, AnySplitPathInsideLibrary, - CanonicalSplitPathInsideLibrary, CanonicalSplitPathInsideLibraryOf, Codecs, SplitPathInsideLibraryOf, SplitPathInsideLibraryWithSeparatedSuffixOf, } from "../../../../../codecs"; import type { NodeName } from "../../../../../types/schemas/node-name"; -import { adaptCodecResult } from "../../bulk-vault-action-adapter/layers/translate-material-event/error-adapters"; import { RenameIntent } from "../../bulk-vault-action-adapter/layers/translate-material-event/policy-and-intent/intent/types"; import { ChangePolicy } from "../../bulk-vault-action-adapter/layers/translate-material-event/policy-and-intent/policy/types"; import { @@ -44,14 +43,14 @@ export function tryCanonicalizeSplitPathToDestination( policy: ChangePolicy, intent: RenameIntent | undefined, // undefined = not rename codecs: Codecs, -): Result; +): Result; export function tryCanonicalizeSplitPathToDestination( sp: AnySplitPathInsideLibrary, policy: ChangePolicy, intent: RenameIntent | undefined, // undefined = not rename codecs: Codecs, ): Result< - CanonicalSplitPathInsideLibraryOf | CanonicalSplitPathInsideLibrary, + CanonicalSplitPathInsideLibraryOf | AnyCanonicalSplitPathInsideLibrary, string > { const effectivePolicy = @@ -60,11 +59,9 @@ export function tryCanonicalizeSplitPathToDestination( // --- PathKing: pathParts are source of truth, build canonical from them if (effectivePolicy === ChangePolicy.PathKing) { // Convert to split path with separated suffix (validates NodeNames) - const withSeparatedSuffixResult = adaptCodecResult( - codecs.splitPathWithSeparatedSuffix.splitPathInsideLibraryToWithSeparatedSuffix( - sp, - ), - ); + const withSeparatedSuffixResult = codecs.splitPathWithSeparatedSuffix + .splitPathInsideLibraryToWithSeparatedSuffix(sp) + .mapErr((error) => error.message); if (withSeparatedSuffixResult.isErr()) { return err(withSeparatedSuffixResult.error); } @@ -76,9 +73,9 @@ export function tryCanonicalizeSplitPathToDestination( const { cleanBasename, marker } = extractDuplicateMarker(sp.basename); if (marker) { // Re-parse without duplicate marker to get clean coreName - const cleanSepRes = adaptCodecResult( - codecs.suffix.parseSeparatedSuffix(cleanBasename), - ); + const cleanSepRes = codecs.suffix + .parseSeparatedSuffix(cleanBasename) + .mapErr((error) => error.message); if (cleanSepRes.isErr()) return err(cleanSepRes.error); // Re-attach marker to coreName const coreNameWithMarker = (cleanSepRes.value.coreName + @@ -121,18 +118,16 @@ export function tryCanonicalizeSplitPathToDestination( return ok( canonizedResult.value as unknown as | CanonicalSplitPathInsideLibraryOf - | CanonicalSplitPathInsideLibrary, + | AnyCanonicalSplitPathInsideLibrary, ); } // --- NameKing: interpret basename as path intent, then OUTPUT PathKing-canonical split path // Convert to split path with separated suffix (validates NodeNames) - const withSeparatedSuffixResult = adaptCodecResult( - codecs.splitPathWithSeparatedSuffix.splitPathInsideLibraryToWithSeparatedSuffix( - sp, - ), - ); + const withSeparatedSuffixResult = codecs.splitPathWithSeparatedSuffix + .splitPathInsideLibraryToWithSeparatedSuffix(sp) + .mapErr((error) => error.message); if (withSeparatedSuffixResult.isErr()) { return err(withSeparatedSuffixResult.error); } @@ -142,9 +137,9 @@ export function tryCanonicalizeSplitPathToDestination( // Handle duplicate markers (business logic) const { cleanBasename, marker } = extractDuplicateMarker(sp.basename); if (marker) { - const cleanSepRes = adaptCodecResult( - codecs.suffix.parseSeparatedSuffix(cleanBasename), - ); + const cleanSepRes = codecs.suffix + .parseSeparatedSuffix(cleanBasename) + .mapErr((error) => error.message); if (cleanSepRes.isErr()) return err(cleanSepRes.error); const coreNameWithMarker = (cleanSepRes.value.coreName + marker) as NodeName; @@ -230,6 +225,6 @@ export function tryCanonicalizeSplitPathToDestination( return ok( canonizedResult.value as unknown as | CanonicalSplitPathInsideLibraryOf - | CanonicalSplitPathInsideLibrary, + | AnyCanonicalSplitPathInsideLibrary, ); } diff --git a/src/commanders/librarian/healer/library-tree/utils/section-chain-utils.ts b/src/commanders/librarian/healer/library-tree/utils/section-chain-utils.ts index d59f8c70b..b29405a2f 100644 --- a/src/commanders/librarian/healer/library-tree/utils/section-chain-utils.ts +++ b/src/commanders/librarian/healer/library-tree/utils/section-chain-utils.ts @@ -1,10 +1,10 @@ /** * Section chain utilities for the library tree. * - * NOTE: For new code, prefer importing from `src/commanders/librarian-new/paths/path-computer.ts` - * which consolidates all path computation logic. This file is kept for backward compatibility. + * NOTE: For new code, prefer `src/commanders/librarian/paths/path-finder.ts` + * to keep section-chain/path logic centralized. * - * @see PathFinder.parseSectionChainToNodeNames in `src/commanders/librarian-new/paths/path-computer.ts` + * @see PathFinder.parseSectionChainToNodeNames in `src/commanders/librarian/paths/path-finder.ts` */ import { err, ok, type Result } from "neverthrow"; import type { CodecError } from "../../../codecs/errors"; diff --git a/src/commanders/librarian/healer/library-tree/utils/split-path-utils.ts b/src/commanders/librarian/healer/library-tree/utils/split-path-utils.ts index af685a5b8..a6a1cf5a5 100644 --- a/src/commanders/librarian/healer/library-tree/utils/split-path-utils.ts +++ b/src/commanders/librarian/healer/library-tree/utils/split-path-utils.ts @@ -1,10 +1,10 @@ /** * Split path utilities for the library tree. * - * NOTE: For new code, prefer importing from `src/commanders/librarian-new/paths/path-computer.ts` - * which consolidates all path computation logic. This file is kept for backward compatibility. + * NOTE: For new code, prefer `src/commanders/librarian/paths/path-finder.ts` + * to keep path logic centralized. * - * @see PathFinder in `src/commanders/librarian-new/paths/path-computer.ts` + * @see PathFinder in `src/commanders/librarian/paths/path-finder.ts` */ import { ok, type Result } from "neverthrow"; import { MD } from "../../../../../managers/obsidian/vault-action-manager/types/literals"; diff --git a/src/commanders/librarian/librarian-init/build-initial-actions.ts b/src/commanders/librarian/librarian-init/build-initial-actions.ts index 895e1c2fc..73de096a9 100644 --- a/src/commanders/librarian/librarian-init/build-initial-actions.ts +++ b/src/commanders/librarian/librarian-init/build-initial-actions.ts @@ -1,32 +1,16 @@ import { z } from "zod"; import type { MD } from "../../../managers/obsidian/vault-action-manager/types/literals"; -import type { - SplitPathToMdFile, - SplitPathWithReader, -} from "../../../managers/obsidian/vault-action-manager/types/split-path"; +import type { SplitPathWithReader } from "../../../managers/obsidian/vault-action-manager/types/split-path"; import { SplitPathKind } from "../../../managers/obsidian/vault-action-manager/types/split-path"; -import type { VaultAction } from "../../../managers/obsidian/vault-action-manager/types/vault-action"; -import { VaultActionKind } from "../../../managers/obsidian/vault-action-manager/types/vault-action"; import { noteMetadataHelper } from "../../../stateless-helpers/note-metadata"; -import { parseFrontmatter } from "../../../stateless-helpers/note-metadata/internal/frontmatter"; -import { readJsonSection } from "../../../stateless-helpers/note-metadata/internal/json-section"; -import { - addFrontmatter, - migrateFrontmatter, - migrateToFrontmatter, -} from "../../../stateless-helpers/note-metadata/internal/migration"; import { logger } from "../../../utils/logger"; import type { AnySplitPathInsideLibrary, CodecRules, Codecs, - SplitPathToMdFileInsideLibrary, } from "../codecs"; import { isCodexSplitPath } from "../healer/library-tree/codex/helpers"; -import { - makeVaultScopedSplitPath, - tryParseAsInsideLibrarySplitPath, -} from "../healer/library-tree/tree-action/bulk-vault-action-adapter/layers/library-scope/codecs/split-path-inside-the-library"; +import { tryParseAsInsideLibrarySplitPath } from "../healer/library-tree/tree-action/bulk-vault-action-adapter/layers/library-scope/codecs/split-path-inside-the-library"; import { inferCreatePolicy } from "../healer/library-tree/tree-action/bulk-vault-action-adapter/layers/translate-material-event/policy-and-intent/policy/infer-create"; import type { CreateTreeLeafAction } from "../healer/library-tree/tree-action/types/tree-action"; import { tryCanonicalizeSplitPathToDestination } from "../healer/library-tree/tree-action/utils/canonical-naming/canonicalize-to-destination"; @@ -45,18 +29,16 @@ const ScrollMetadataSchema = z export type BuildInitialActionsResult = { createActions: CreateTreeLeafAction[]; - migrationActions: VaultAction[]; }; /** * Build CreateTreeLeafAction for each file in the library. * Applies policy (NameKing for root, PathKing for nested) to determine canonical location. * Reads status from md file metadata or YAML frontmatter. - * Returns migration actions for files that need format conversion. * * @param files - Files from vault with readers * @param codecs - Codec API - * @param rules - Codec rules (includes hideMetadata setting) + * @param rules - Codec rules */ export async function buildInitialCreateActions( files: SplitPathWithReader[], @@ -64,7 +46,6 @@ export async function buildInitialCreateActions( rules: CodecRules, ): Promise { const createActions: CreateTreeLeafAction[] = []; - const migrationActions: VaultAction[] = []; for (const file of files) { // Skip codex files (basename starts with __) @@ -132,61 +113,6 @@ export async function buildInitialCreateActions( if (meta?.status === "Done") { status = TreeNodeStatus.Done; } - - // Check which formats exist for migration - const hasInternal = - readJsonSection(content, ScrollMetadataSchema) !== null; - const hasFrontmatter = parseFrontmatter(content) !== null; - - // Cast observedPath to MdFile since we're inside file.kind === MdFile check - const mdPath = observedPath as SplitPathToMdFileInsideLibrary; - const vaultMdPath = makeVaultScopedSplitPath( - mdPath, - rules, - ) as SplitPathToMdFile; - - if (rules.hideMetadata) { - // Want internal format - migrate YAML to internal if needed - if (hasFrontmatter && !hasInternal) { - migrationActions.push({ - kind: VaultActionKind.ProcessMdFile, - payload: { - splitPath: vaultMdPath, - transform: migrateFrontmatter({ - stripYaml: true, - }), - }, - }); - } - } else { - // Want YAML format - if (hasInternal && meta) { - // Convert internal to YAML - migrationActions.push({ - kind: VaultActionKind.ProcessMdFile, - payload: { - splitPath: vaultMdPath, - transform: migrateToFrontmatter(meta), - }, - }); - } else if (!hasFrontmatter) { - // No metadata at all - add YAML with current status - const statusValue = - status === TreeNodeStatus.Done - ? TreeNodeStatus.Done - : TreeNodeStatus.NotStarted; - - migrationActions.push({ - kind: VaultActionKind.ProcessMdFile, - payload: { - splitPath: vaultMdPath, - transform: addFrontmatter({ - status: statusValue, - }), - }, - }); - } - } } } @@ -212,5 +138,5 @@ export async function buildInitialCreateActions( } } - return { createActions, migrationActions }; + return { createActions }; } diff --git a/src/commanders/librarian/librarian.ts b/src/commanders/librarian/librarian.ts index bcba264d7..df824f579 100644 --- a/src/commanders/librarian/librarian.ts +++ b/src/commanders/librarian/librarian.ts @@ -75,7 +75,6 @@ import { getNextPage as getNextPageImpl, getPrevPage as getPrevPageImpl, } from "./page-navigation"; -import { buildPageNavMigrationActions } from "./section-healing/page-nav-migration"; import { triggerSectionHealing as triggerSectionHealingImpl } from "./section-healing/section-healing-coordinator"; import { PREFIX_OF_CODEX } from "./types/consts/literals"; import type { NodeName } from "./types/schemas/node-name"; @@ -146,12 +145,11 @@ export class Librarian { const allFiles = allFilesResult.value; // Build Create actions for each file - const { createActions, migrationActions } = - await buildInitialCreateActions( - allFiles, - this.codecs, - this.rules, - ); + const { createActions } = await buildInitialCreateActions( + allFiles, + this.codecs, + this.rules, + ); // Apply all create actions via HealingTransaction const tx = new HealingTransaction(this.healer); @@ -210,13 +208,7 @@ export class Librarian { ); allHealingActions.push(...deletionHealingActions); - // Build page navigation migration actions - const pageNavMigrationActions = await buildPageNavMigrationActions( - allFiles, - this.rules, - ); - - // Combine all actions and dispatch once (healing → codex → backlink → migration) + // Combine all actions and dispatch once (healing → codex → backlink) const allVaultActions = [ ...assembleVaultActions( allHealingActions, @@ -229,8 +221,6 @@ export class Librarian { this.codecs, this.rules, ), - ...migrationActions, // Convert YAML frontmatter to internal format - ...pageNavMigrationActions, // Add missing page navigation indices ]; if (allVaultActions.length > 0) { diff --git a/src/commanders/librarian/list-commands-executable.ts b/src/commanders/librarian/list-commands-executable.ts index bc7d87247..13ab9a7f1 100644 --- a/src/commanders/librarian/list-commands-executable.ts +++ b/src/commanders/librarian/list-commands-executable.ts @@ -58,7 +58,6 @@ export function listCommandsExecutableIn( } else { // Scroll commands: split into pages commands.push(CommandKind.SplitToPages); - commands.push(CommandKind.MakeText); } // Selection-dependent commands available for any library file diff --git a/src/commanders/librarian/section-healing/page-nav-migration.ts b/src/commanders/librarian/section-healing/page-nav-migration.ts deleted file mode 100644 index 8547bf89c..000000000 --- a/src/commanders/librarian/section-healing/page-nav-migration.ts +++ /dev/null @@ -1,166 +0,0 @@ -/** - * Migration for page navigation indices. - * - * Detects Page notes missing prevPageIdx/nextPageIdx and generates - * ProcessMdFile actions to add the indices based on sibling pages. - */ - -import { z } from "zod"; -import type { - SplitPathToMdFile, - SplitPathWithReader, -} from "../../../managers/obsidian/vault-action-manager/types/split-path"; -import { SplitPathKind } from "../../../managers/obsidian/vault-action-manager/types/split-path"; -import type { VaultAction } from "../../../managers/obsidian/vault-action-manager/types/vault-action"; -import { VaultActionKind } from "../../../managers/obsidian/vault-action-manager/types/vault-action"; -import { noteMetadataHelper } from "../../../stateless-helpers/note-metadata"; -import { parsePageIndex } from "../bookkeeper/page-codec"; -import { parseSeparatedSuffix } from "../codecs/internal/suffix/parse"; -import type { CodecRules } from "../codecs/rules"; - -/** Schema for reading page metadata */ -const PageMetadataSchema = z - .object({ - nextPageIdx: z.number().optional(), - noteKind: z.string().optional(), - prevPageIdx: z.number().optional(), - status: z.string().optional(), - }) - .passthrough(); - -type PageInfo = { - splitPath: SplitPathToMdFile; - pageIndex: number; - coreName: string; - hasPrevIdx: boolean; - hasNextIdx: boolean; - needsNoteKind: boolean; -}; - -/** - * Build migration actions for pages missing navigation indices. - * - * @param files - All files in the library with readers - * @param rules - Codec rules for building vault-scoped paths - * @returns VaultActions to add missing indices - */ -export async function buildPageNavMigrationActions( - files: SplitPathWithReader[], - rules: CodecRules, -): Promise { - const migrationActions: VaultAction[] = []; - - // Group pages by their parent folder + coreName - const pageGroups = new Map(); - - for (const file of files) { - if (file.kind !== SplitPathKind.MdFile || !("read" in file)) continue; - - // Parse suffix to extract coreName before checking page pattern - const suffixResult = parseSeparatedSuffix(rules, file.basename); - if (suffixResult.isErr()) continue; - const { coreName } = suffixResult.value; - - // Check if this is a page file by coreName pattern - const parseResult = parsePageIndex(coreName); - if (!parseResult.isPage) continue; - - // Read content to check metadata - const contentResult = await file.read(); - if (contentResult.isErr()) continue; - - const content = contentResult.value; - const metadata = noteMetadataHelper.read(content, PageMetadataSchema); - - // Page files identified by filename pattern, not by noteKind - // Initialize missing/partial metadata - const normalizedMeta = metadata ?? {}; - - // Build group key: folder path + page coreName (from parsePageIndex) - const folderPath = file.pathParts.join("/"); - const groupKey = `${folderPath}/${parseResult.coreName}`; - - // Build vault-scoped split path - const vaultPath: SplitPathToMdFile = { - basename: file.basename, - extension: file.extension, - kind: SplitPathKind.MdFile, - pathParts: [...rules.libraryRootPathParts, ...file.pathParts], - }; - - const pageInfo: PageInfo = { - coreName: parseResult.coreName, - hasNextIdx: normalizedMeta.nextPageIdx !== undefined, - hasPrevIdx: normalizedMeta.prevPageIdx !== undefined, - needsNoteKind: normalizedMeta.noteKind !== "Page", - pageIndex: parseResult.pageIndex, - splitPath: vaultPath, - }; - - if (!pageGroups.has(groupKey)) { - pageGroups.set(groupKey, []); - } - pageGroups.get(groupKey)?.push(pageInfo); - } - - // Process each group to compute and add missing indices - for (const [_groupKey, pages] of pageGroups) { - // Sort by page index - pages.sort((a, b) => a.pageIndex - b.pageIndex); - - const totalPages = pages.length; - if (totalPages === 0) continue; - - // Check each page for missing indices - for (let i = 0; i < totalPages; i++) { - const page = pages[i]; - if (!page) continue; - - // Compute expected indices - const expectedPrevIdx = i > 0 ? pages[i - 1]?.pageIndex : undefined; - const expectedNextIdx = - i < totalPages - 1 ? pages[i + 1]?.pageIndex : undefined; - - // Check if migration is needed - const needsPrev = expectedPrevIdx !== undefined && !page.hasPrevIdx; - const needsNext = expectedNextIdx !== undefined && !page.hasNextIdx; - const needsNoteKind = page.needsNoteKind; - - if (needsPrev || needsNext || needsNoteKind) { - migrationActions.push({ - kind: VaultActionKind.ProcessMdFile, - payload: { - splitPath: page.splitPath, - transform: (content: string) => { - // Read existing metadata (may be null/partial) - const existing = - noteMetadataHelper.read( - content, - PageMetadataSchema, - ) ?? {}; - - // Build updated metadata with noteKind always set - const updated: Record = { - ...existing, - noteKind: "Page", // Always ensure noteKind is set - }; - if (needsPrev && expectedPrevIdx !== undefined) { - updated.prevPageIdx = expectedPrevIdx; - } - if (needsNext && expectedNextIdx !== undefined) { - updated.nextPageIdx = expectedNextIdx; - } - - // Apply metadata update - return noteMetadataHelper.upsert(updated)( - content, - ) as string; - }, - }, - }); - } - } - } - - return migrationActions; -} diff --git a/src/commanders/textfresser/commands/generate/generate-command.ts b/src/commanders/textfresser/commands/generate/generate-command.ts index c7963f671..ba0338f31 100644 --- a/src/commanders/textfresser/commands/generate/generate-command.ts +++ b/src/commanders/textfresser/commands/generate/generate-command.ts @@ -7,6 +7,7 @@ import { checkAttestation } from "./steps/check-attestation"; import { checkEligibility } from "./steps/check-eligibility"; import { checkLemmaResult } from "./steps/check-lemma-result"; import { generateSections } from "./steps/generate-sections"; +import { maintainClosedSetSurfaceHub } from "./steps/maintain-closed-set-surface-hub"; import { moveToWorter } from "./steps/move-to-worter"; import { propagateGeneratedSections } from "./steps/propagate-generated-sections"; import { resolveExistingEntry } from "./steps/resolve-existing-entry"; @@ -17,8 +18,9 @@ import { serializeEntry } from "./steps/serialize-entry"; * checkAttestation → checkEligibility → checkLemmaResult * → resolveExistingEntry (parse existing entries) * → generateSections (async: LLM calls or append attestation) - * → propagateGeneratedSections (v2-only propagation + post-propagation decoration) - * → serializeEntry (includes noteKind meta) → moveToWorter(policy destination) → addWriteAction + * → propagateGeneratedSections (core propagation + post-propagation decoration) + * → serializeEntry (includes noteKind meta) → moveToWorter(policy destination) + * → maintainClosedSetSurfaceHub (closed-set ambiguous manual-link hubs) → addWriteAction */ export function generateCommand( input: CommandInput, @@ -37,6 +39,7 @@ export function generateCommand( .andThen(propagateGeneratedSections) .andThen(serializeEntry) .andThen(moveToWorter) + .andThen(maintainClosedSetSurfaceHub) .andThen((c) => { const activeFile = c.commandContext.activeFile; const writeAction = { diff --git a/src/commanders/textfresser/commands/generate/section-formatters/common/header-formatter.ts b/src/commanders/textfresser/commands/generate/section-formatters/common/header-formatter.ts index 6be120435..5425062bf 100644 --- a/src/commanders/textfresser/commands/generate/section-formatters/common/header-formatter.ts +++ b/src/commanders/textfresser/commands/generate/section-formatters/common/header-formatter.ts @@ -1,3 +1,5 @@ +import { wikilinkHelper } from "../../../../../../stateless-helpers/wikilink"; + const YOUGLISH_BASE = "https://youglish.com/pronounce"; export function buildYouglishUrl( @@ -19,7 +21,8 @@ export function formatHeaderLine( targetLanguage: string, ): string { const emoji = output.emojiDescription.join(" "); - const youglishUrl = buildYouglishUrl(lemma, targetLanguage); + const normalizedLemma = wikilinkHelper.normalizeLinkTarget(lemma); + const youglishUrl = buildYouglishUrl(normalizedLemma, targetLanguage); - return `${emoji} [[${lemma}]], [${output.ipa}](${youglishUrl})`; + return `${emoji} [[${normalizedLemma}]], [${output.ipa}](${youglishUrl})`; } diff --git a/src/commanders/textfresser/commands/generate/section-formatters/common/inflection-formatter.ts b/src/commanders/textfresser/commands/generate/section-formatters/common/inflection-formatter.ts index a3f81555f..806da487c 100644 --- a/src/commanders/textfresser/commands/generate/section-formatters/common/inflection-formatter.ts +++ b/src/commanders/textfresser/commands/generate/section-formatters/common/inflection-formatter.ts @@ -1,4 +1,5 @@ import type { AgentOutput } from "../../../../../../prompt-smith"; +import { wikilinkHelper } from "../../../../../../stateless-helpers/wikilink"; /** * Format LLM inflection output into markdown lines. @@ -13,5 +14,11 @@ import type { AgentOutput } from "../../../../../../prompt-smith"; export function formatInflectionSection( output: AgentOutput<"Inflection">, ): string { - return output.rows.map((r) => `${r.label}: ${r.forms}`).join("\n"); + return output.rows + .map((row) => { + const normalizedForms = + wikilinkHelper.normalizeWikilinkTargetsInText(row.forms); + return `${row.label}: ${normalizedForms}`; + }) + .join("\n"); } diff --git a/src/commanders/textfresser/commands/generate/section-formatters/common/inflection-propagation-helper.ts b/src/commanders/textfresser/commands/generate/section-formatters/common/inflection-propagation-helper.ts index 88f2e7430..6a0c48a26 100644 --- a/src/commanders/textfresser/commands/generate/section-formatters/common/inflection-propagation-helper.ts +++ b/src/commanders/textfresser/commands/generate/section-formatters/common/inflection-propagation-helper.ts @@ -23,7 +23,6 @@ const CASE_ORDER_INDEX = new Map( ); const INFLECTION_TAG_RE = /^#([^/\s]+)\/([^/\s]+)$/; -const LEGACY_HEADER_RE = /^#([^/\s]+)\/([^/\s]+) for: \[\[(.+)\]\]$/; export type ParsedInflectionTag = { caseValue: CaseValue; @@ -146,27 +145,6 @@ export function parseLocalizedInflectionTag( }; } -export function parseLegacyInflectionHeaderTag( - header: string, - lemma: string, - targetLanguage: TargetLanguage, -): string | null { - const match = header.match(LEGACY_HEADER_RE); - if (!match) return null; - - const caseLabel = match[1]; - const numberLabel = match[2]; - const headerLemma = match[3]; - if (!caseLabel || !numberLabel || !headerLemma) return null; - if (headerLemma !== lemma) return null; - - const caseValue = caseValueFromLocalizedLabel(caseLabel); - const numberValue = numberValueFromLocalizedLabel(numberLabel); - if (!caseValue || !numberValue) return null; - - return buildLocalizedInflectionTag(caseValue, numberValue, targetLanguage); -} - import { extractHashTags } from "../../../../../../utils/text-utils"; export { extractHashTags }; diff --git a/src/commanders/textfresser/commands/generate/section-formatters/common/relation-formatter.ts b/src/commanders/textfresser/commands/generate/section-formatters/common/relation-formatter.ts index f008eb3cd..c1e1361dc 100644 --- a/src/commanders/textfresser/commands/generate/section-formatters/common/relation-formatter.ts +++ b/src/commanders/textfresser/commands/generate/section-formatters/common/relation-formatter.ts @@ -1,5 +1,6 @@ import type { AgentOutput } from "../../../../../../prompt-smith"; import type { RelationSubKind } from "../../../../../../prompt-smith/schemas/relation"; +import { wikilinkHelper } from "../../../../../../stateless-helpers/wikilink"; const SYMBOL_FOR_KIND: Record = { Antonym: "≠", @@ -19,7 +20,11 @@ export function formatRelationSection(output: AgentOutput<"Relation">): string { return output.relations .map((r) => { const symbol = SYMBOL_FOR_KIND[r.kind]; - const words = r.words.map((w) => `[[${w}]]`).join(", "); + const words = r.words + .map( + (word) => `[[${wikilinkHelper.normalizeLinkTarget(word)}]]`, + ) + .join(", "); return `${symbol} ${words}`; }) .join("\n"); diff --git a/src/commanders/textfresser/commands/generate/section-formatters/de/lexem/noun/header-formatter.ts b/src/commanders/textfresser/commands/generate/section-formatters/de/lexem/noun/header-formatter.ts index 5fa46b0b8..12d2386ae 100644 --- a/src/commanders/textfresser/commands/generate/section-formatters/de/lexem/noun/header-formatter.ts +++ b/src/commanders/textfresser/commands/generate/section-formatters/de/lexem/noun/header-formatter.ts @@ -2,6 +2,7 @@ import { articleFromGenus, type GermanGenus, } from "../../../../../../../../linguistics/de/lexem/noun/features"; +import { wikilinkHelper } from "../../../../../../../../stateless-helpers/wikilink"; import { buildYouglishUrl } from "../../../common/header-formatter"; /** @@ -17,7 +18,8 @@ export function formatHeaderLine( ): string { const emoji = output.emojiDescription.join(" "); const article = articleFromGenus[genus]; - const youglishUrl = buildYouglishUrl(lemma, targetLanguage); + const normalizedLemma = wikilinkHelper.normalizeLinkTarget(lemma); + const youglishUrl = buildYouglishUrl(normalizedLemma, targetLanguage); - return `${emoji} ${article} [[${lemma}]], [${output.ipa}](${youglishUrl})`; + return `${emoji} ${article} [[${normalizedLemma}]], [${output.ipa}](${youglishUrl})`; } diff --git a/src/commanders/textfresser/commands/generate/section-formatters/de/lexem/noun/inflection-formatter.ts b/src/commanders/textfresser/commands/generate/section-formatters/de/lexem/noun/inflection-formatter.ts index 29333c697..37e927f60 100644 --- a/src/commanders/textfresser/commands/generate/section-formatters/de/lexem/noun/inflection-formatter.ts +++ b/src/commanders/textfresser/commands/generate/section-formatters/de/lexem/noun/inflection-formatter.ts @@ -5,6 +5,7 @@ import { type NounInflectionCell, } from "../../../../../../../../linguistics/de/lexem/noun"; import type { AgentOutput } from "../../../../../../../../prompt-smith"; +import { wikilinkHelper } from "../../../../../../../../stateless-helpers/wikilink"; /** * Group cells by case, then format each case line as: @@ -19,7 +20,7 @@ export function formatInflection(output: AgentOutput<"NounInflection">): { const cells: NounInflectionCell[] = output.cells.map((c) => ({ article: c.article, case: c.case, - form: c.form, + form: wikilinkHelper.normalizeLinkTarget(c.form), number: c.number, })); @@ -37,7 +38,9 @@ export function formatInflection(output: AgentOutput<"NounInflection">): { if (!group || group.length === 0) continue; const label = CASE_SHORT_LABEL[caseVal]; - const forms = group.map((c) => `${c.article} [[${c.form}]]`).join(", "); + const forms = group + .map((cell) => `${cell.article} [[${cell.form}]]`) + .join(", "); lines.push(`${label}: ${forms}`); } diff --git a/src/commanders/textfresser/commands/generate/steps/generate-sections.ts b/src/commanders/textfresser/commands/generate/steps/generate-sections.ts index 6d5e25be1..1fabf4097 100644 --- a/src/commanders/textfresser/commands/generate/steps/generate-sections.ts +++ b/src/commanders/textfresser/commands/generate/steps/generate-sections.ts @@ -226,20 +226,12 @@ export function generateSections( generated.enrichmentOutput, generated.featuresOutput, ); - const emojiDescription = - entity?.emojiDescription ?? - lemmaResult.precomputedEmojiDescription ?? - generated.enrichmentOutput.emojiDescription; - const senseGloss = - entity?.senseGloss ?? generated.enrichmentOutput.senseGloss; const newEntry: DictEntry = { headerContent: generated.headerContent, id: generated.entryId, meta: { ...(entity ? { entity } : {}), - emojiDescription, - ...(senseGloss ? { senseGloss } : {}), ...(verbEntryIdentity ? { verbEntryIdentity } : {}), ...(linguisticUnit ? { linguisticUnit } : {}), }, diff --git a/src/commanders/textfresser/commands/generate/steps/maintain-closed-set-surface-hub.ts b/src/commanders/textfresser/commands/generate/steps/maintain-closed-set-surface-hub.ts new file mode 100644 index 000000000..7794ac8d6 --- /dev/null +++ b/src/commanders/textfresser/commands/generate/steps/maintain-closed-set-surface-hub.ts @@ -0,0 +1,36 @@ +import { ok, type Result } from "neverthrow"; +import { buildClosedSetSurfaceHubActions } from "../../../common/closed-set-surface-hub"; +import { isClosedSetPos } from "../../../common/lemma-link-routing"; +import type { CommandError, CommandStateWithLemma } from "../../types"; + +export function maintainClosedSetSurfaceHub( + ctx: CommandStateWithLemma, +): Result { + const lemmaResult = ctx.textfresserState.latestLemmaResult; + if (lemmaResult.linguisticUnit !== "Lexem") { + return ok(ctx); + } + if (!isClosedSetPos(lemmaResult.posLikeKind)) { + return ok(ctx); + } + if (!ctx.textfresserState.isLibraryLookupAvailable) { + return ok(ctx); + } + + const actions = buildClosedSetSurfaceHubActions({ + currentClosedSetTarget: ctx.commandContext.activeFile.splitPath, + lookupInLibrary: ctx.textfresserState.lookupInLibrary, + surface: lemmaResult.attestation.target.surface, + targetLanguage: ctx.textfresserState.languages.target, + vam: ctx.textfresserState.vam, + }); + + if (actions.length === 0) { + return ok(ctx); + } + + return ok({ + ...ctx, + actions: [...ctx.actions, ...actions], + }); +} diff --git a/src/commanders/textfresser/commands/generate/steps/morphology-utils.ts b/src/commanders/textfresser/commands/generate/steps/morphology-utils.ts index bccc06be5..8ce7800c5 100644 --- a/src/commanders/textfresser/commands/generate/steps/morphology-utils.ts +++ b/src/commanders/textfresser/commands/generate/steps/morphology-utils.ts @@ -1,6 +1,12 @@ +import { wikilinkHelper } from "../../../../../stateless-helpers/wikilink"; + export function normalizeLemma(raw: string | null | undefined): string | null { const trimmed = raw?.trim(); - return trimmed && trimmed.length > 0 ? trimmed : null; + if (!trimmed || trimmed.length === 0) { + return null; + } + const normalized = wikilinkHelper.normalizeLinkTarget(trimmed); + return normalized.length > 0 ? normalized : null; } export function normalizeMorphologyKey( diff --git a/src/commanders/textfresser/commands/generate/steps/propagate-v2.ts b/src/commanders/textfresser/commands/generate/steps/propagate-core.ts similarity index 96% rename from src/commanders/textfresser/commands/generate/steps/propagate-v2.ts rename to src/commanders/textfresser/commands/generate/steps/propagate-core.ts index 8c5e5d2cf..aef2c7b19 100644 --- a/src/commanders/textfresser/commands/generate/steps/propagate-v2.ts +++ b/src/commanders/textfresser/commands/generate/steps/propagate-core.ts @@ -14,7 +14,7 @@ import { propagateRelations } from "./propagate-relations"; type ProcessTransform = (content: string) => string | Promise; -function buildV2Error(reason: string): CommandError { +function buildPropagationError(reason: string): CommandError { return { kind: "ApiError", reason, @@ -136,8 +136,8 @@ export function foldScopedActionsToSingleWritePerTarget( } return err( - buildV2Error( - `[propagateV2] Unsupported scoped action kind: ${action.kind}`, + buildPropagationError( + `[propagateCore] Unsupported scoped action kind: ${action.kind}`, ), ); } @@ -174,7 +174,7 @@ function collectScopedActions( .map((result) => result.actions); } -export function propagateV2( +export function propagateCore( ctx: GenerateSectionsResult, ): Result { return collectScopedActions(ctx) diff --git a/src/commanders/textfresser/commands/generate/steps/propagate-generated-sections.ts b/src/commanders/textfresser/commands/generate/steps/propagate-generated-sections.ts index b29cc8805..777d26480 100644 --- a/src/commanders/textfresser/commands/generate/steps/propagate-generated-sections.ts +++ b/src/commanders/textfresser/commands/generate/steps/propagate-generated-sections.ts @@ -2,10 +2,10 @@ import type { Result } from "neverthrow"; import type { CommandError } from "../../types"; import { decorateAttestationSeparability } from "./decorate-attestation-separability"; import type { GenerateSectionsResult } from "./generate-sections"; -import { propagateV2 } from "./propagate-v2"; +import { propagateCore } from "./propagate-core"; export function propagateGeneratedSections( ctx: GenerateSectionsResult, ): Result { - return propagateV2(ctx).andThen(decorateAttestationSeparability); + return propagateCore(ctx).andThen(decorateAttestationSeparability); } diff --git a/src/commanders/textfresser/commands/generate/steps/propagate-inflections.ts b/src/commanders/textfresser/commands/generate/steps/propagate-inflections.ts index b6d63907e..737bd500a 100644 --- a/src/commanders/textfresser/commands/generate/steps/propagate-inflections.ts +++ b/src/commanders/textfresser/commands/generate/steps/propagate-inflections.ts @@ -23,7 +23,6 @@ import { extractHashTags, isNounInflectionPropagationHeaderForLemma, mergeLocalizedInflectionTags, - parseLegacyInflectionHeaderTag, } from "../section-formatters/common/inflection-propagation-helper"; import type { GenerateSectionsResult } from "./generate-sections"; @@ -62,7 +61,6 @@ function findTagsSection(entry: DictEntry): EntrySection | undefined { * * For each inflected form, creates or updates one inflection entry per lemma/POS * and stores all case/number variants as tags in a tags section. - * Also auto-collapses legacy per-cell entries into the single-entry format. */ export function propagateInflections( ctx: GenerateSectionsResult, @@ -111,9 +109,7 @@ export function propagateInflections( const transform = (content: string) => { const existingEntries = dictNoteHelper.parse(content); - const legacyEntryIndexes: number[] = []; const matchingEntryIndexes: number[] = []; - const tagsFromLegacyHeaders: string[] = []; const tagsFromExistingMatches: string[] = []; for (let index = 0; index < existingEntries.length; index++) { @@ -133,24 +129,12 @@ export function propagateInflections( ...extractHashTags(tagsSection.content), ); } - continue; - } - - const legacyTag = parseLegacyInflectionHeaderTag( - entry.headerContent, - lemma, - targetLanguage, - ); - if (legacyTag) { - legacyEntryIndexes.push(index); - tagsFromLegacyHeaders.push(legacyTag); } } - const indexesToRemove = new Set([ - ...legacyEntryIndexes, - ...matchingEntryIndexes.slice(1), - ]); + const indexesToRemove = new Set( + matchingEntryIndexes.slice(1), + ); const compactedEntries = existingEntries.filter( (_, index) => !indexesToRemove.has(index), ); @@ -164,11 +148,7 @@ export function propagateInflections( } } - const tagsToMerge = [ - ...tagsFromCells, - ...tagsFromLegacyHeaders, - ...tagsFromExistingMatches, - ]; + const tagsToMerge = [...tagsFromCells, ...tagsFromExistingMatches]; if (!targetEntry) { const existingIds = compactedEntries.map((entry) => entry.id); diff --git a/src/commanders/textfresser/commands/generate/steps/propagate-morphology-relations.ts b/src/commanders/textfresser/commands/generate/steps/propagate-morphology-relations.ts index 8efa58bbe..0d1e68558 100644 --- a/src/commanders/textfresser/commands/generate/steps/propagate-morphology-relations.ts +++ b/src/commanders/textfresser/commands/generate/steps/propagate-morphology-relations.ts @@ -3,6 +3,7 @@ import { SurfaceKind } from "../../../../../linguistics/common/enums/core"; import type { VaultAction } from "../../../../../managers/obsidian/vault-action-manager"; import { morphologyRelationHelper } from "../../../../../stateless-helpers/morphology-relation"; import { wikilinkHelper } from "../../../../../stateless-helpers/wikilink"; +import { canonicalizeTargetForComparison } from "../../../common/target-comparison"; import { buildPropagationActionPair, resolveMorphemePath, @@ -75,7 +76,7 @@ function hasEquivalentEquationLine( function extractLineTargetSignature(line: string): string[] { return wikilinkHelper .parse(line) - .map((wikilink) => wikilinkHelper.normalizeTarget(wikilink.target)) + .map((wikilink) => canonicalizeTargetForComparison(wikilink.target)) .filter((target) => target.length > 0); } @@ -111,8 +112,8 @@ function buildTargets(ctx: GenerateSectionsResult): MorphologyTarget[] { const derivedFrom = normalizeLemma(morphology.derivedFromLemma); if ( derivedFrom && - wikilinkHelper.normalizeTarget(derivedFrom) !== - wikilinkHelper.normalizeTarget(sourceLemma) + canonicalizeTargetForComparison(derivedFrom) !== + canonicalizeTargetForComparison(sourceLemma) ) { targets.push({ kind: "UsedIn", @@ -126,8 +127,8 @@ function buildTargets(ctx: GenerateSectionsResult): MorphologyTarget[] { const normalized = normalizeLemma(lemma); if (!normalized) continue; if ( - wikilinkHelper.normalizeTarget(normalized) === - wikilinkHelper.normalizeTarget(sourceLemma) + canonicalizeTargetForComparison(normalized) === + canonicalizeTargetForComparison(sourceLemma) ) { continue; } @@ -145,8 +146,8 @@ function buildTargets(ctx: GenerateSectionsResult): MorphologyTarget[] { ); if ( targetWord && - wikilinkHelper.normalizeTarget(targetWord) !== - wikilinkHelper.normalizeTarget(sourceLemma) + canonicalizeTargetForComparison(targetWord) !== + canonicalizeTargetForComparison(sourceLemma) ) { const targetKey = normalizeMorphologyKey(targetWord); if (targetKey) { @@ -194,8 +195,8 @@ function groupTargets( for (const target of targets) { const key = target.kind === "Equation" - ? `${target.kind}::${wikilinkHelper.normalizeTarget(target.targetWord)}::${target.targetHeader}` - : `${target.kind}::${wikilinkHelper.normalizeTarget(target.targetWord)}`; + ? `${target.kind}::${canonicalizeTargetForComparison(target.targetWord)}::${target.targetHeader}` + : `${target.kind}::${canonicalizeTargetForComparison(target.targetWord)}`; const existing = grouped.get(key); if (!existing) { grouped.set(key, target); diff --git a/src/commanders/textfresser/commands/generate/steps/propagation-line-append.ts b/src/commanders/textfresser/commands/generate/steps/propagation-line-append.ts index 28d325d4c..9d788e157 100644 --- a/src/commanders/textfresser/commands/generate/steps/propagation-line-append.ts +++ b/src/commanders/textfresser/commands/generate/steps/propagation-line-append.ts @@ -1,5 +1,6 @@ import { morphologyRelationHelper } from "../../../../../stateless-helpers/morphology-relation"; import { wikilinkHelper } from "../../../../../stateless-helpers/wikilink"; +import { canonicalizeTargetForComparison } from "../../../common/target-comparison"; export type PropagationResult = { changed: boolean; content: string }; @@ -130,12 +131,12 @@ export function blockHasWikilinkTarget( blockContent: string, target: string, ): boolean { - const normalizedTarget = wikilinkHelper.normalizeTarget(target); + const normalizedTarget = canonicalizeTargetForComparison(target); return wikilinkHelper .parse(blockContent) .some( (link) => - wikilinkHelper.normalizeTarget(link.target) === + canonicalizeTargetForComparison(link.target) === normalizedTarget, ); } diff --git a/src/commanders/textfresser/commands/generate/steps/propagation-v2-ports-adapter.ts b/src/commanders/textfresser/commands/generate/steps/propagation-ports-adapter.ts similarity index 95% rename from src/commanders/textfresser/commands/generate/steps/propagation-v2-ports-adapter.ts rename to src/commanders/textfresser/commands/generate/steps/propagation-ports-adapter.ts index 5a72cd381..753769d1d 100644 --- a/src/commanders/textfresser/commands/generate/steps/propagation-v2-ports-adapter.ts +++ b/src/commanders/textfresser/commands/generate/steps/propagation-ports-adapter.ts @@ -24,19 +24,19 @@ type VamPortDependency = Pick< "exists" | "findByBasename" | "readContent" >; -export type CreatePropagationV2PortsAdapterParams = { +export type CreatePropagationPortsAdapterParams = { vam: VamPortDependency; lookupInLibraryByCoreName: PathLookupFn; }; -export type PropagationV2PortsAdapter = { +export type PropagationPortsAdapter = { vault: PropagationVaultPort; libraryLookup: PropagationLibraryLookupPort; }; -export function createPropagationV2PortsAdapter( - params: CreatePropagationV2PortsAdapterParams, -): PropagationV2PortsAdapter { +export function createPropagationPortsAdapter( + params: CreatePropagationPortsAdapterParams, +): PropagationPortsAdapter { const libraryLookup = createPropagationLibraryLookupPort( params.lookupInLibraryByCoreName, ); diff --git a/src/commanders/textfresser/commands/generate/steps/section-generators/morphology-section-generator.ts b/src/commanders/textfresser/commands/generate/steps/section-generators/morphology-section-generator.ts index 24725610c..d9b8da518 100644 --- a/src/commanders/textfresser/commands/generate/steps/section-generators/morphology-section-generator.ts +++ b/src/commanders/textfresser/commands/generate/steps/section-generators/morphology-section-generator.ts @@ -1,5 +1,6 @@ import { morphologyRelationHelper } from "../../../../../../stateless-helpers/morphology-relation"; import { wikilinkHelper } from "../../../../../../stateless-helpers/wikilink"; +import { canonicalizeTargetForComparison } from "../../../../common/target-comparison"; import type { EntrySection } from "../../../../domain/dict-note/types"; import { type MorphemeItem, @@ -136,10 +137,10 @@ function buildPrefixEquationLine( function haveSameWikilinkTargets(left: string, right: string): boolean { const leftTargets = wikilinkHelper .parse(left) - .map((wikilink) => wikilinkHelper.normalizeTarget(wikilink.target)); + .map((wikilink) => canonicalizeTargetForComparison(wikilink.target)); const rightTargets = wikilinkHelper .parse(right) - .map((wikilink) => wikilinkHelper.normalizeTarget(wikilink.target)); + .map((wikilink) => canonicalizeTargetForComparison(wikilink.target)); if (leftTargets.length !== rightTargets.length) return false; return leftTargets.every((target, index) => { diff --git a/src/commanders/textfresser/commands/generate/steps/section-generators/relation-section-generator.ts b/src/commanders/textfresser/commands/generate/steps/section-generators/relation-section-generator.ts index 3eeaac82f..40d628bd2 100644 --- a/src/commanders/textfresser/commands/generate/steps/section-generators/relation-section-generator.ts +++ b/src/commanders/textfresser/commands/generate/steps/section-generators/relation-section-generator.ts @@ -1,3 +1,4 @@ +import { wikilinkHelper } from "../../../../../../stateless-helpers/wikilink"; import type { EntrySection } from "../../../../domain/dict-note/types"; import { cssSuffixFor } from "../../../../targets/de/sections/section-css-kind"; import { @@ -24,7 +25,9 @@ export type RelationSectionResult = { function toParsedRelations(output: RelationOutput): ParsedRelation[] { return output.relations.map((relation) => ({ kind: relation.kind, - words: relation.words, + words: relation.words.map((word) => + wikilinkHelper.normalizeLinkTarget(word), + ), })); } diff --git a/src/commanders/textfresser/commands/lemma/steps/disambiguate-sense.ts b/src/commanders/textfresser/commands/lemma/steps/disambiguate-sense.ts index 70d65a07b..d88ff7cb5 100644 --- a/src/commanders/textfresser/commands/lemma/steps/disambiguate-sense.ts +++ b/src/commanders/textfresser/commands/lemma/steps/disambiguate-sense.ts @@ -184,52 +184,22 @@ export function disambiguateSense( Array.isArray(entity?.emojiDescription) && entity.emojiDescription.length > 0 ? entity.emojiDescription - : e.meta.emojiDescription; - const ipaFromLegacyMeta = e.meta.ipa; + : null; const ipa = typeof entity?.ipa === "string" && entity.ipa.length > 0 ? entity.ipa - : typeof ipaFromLegacyMeta === "string" && - ipaFromLegacyMeta.length > 0 - ? ipaFromLegacyMeta - : extractIpaFromHeaderContent(e.headerContent); + : extractIpaFromHeaderContent(e.headerContent); const senseGlossFromEntity = extractSenseGlossFromEntity(entity); - const senseGlossFromLegacyMeta = - typeof e.meta.senseGloss === "string" && - e.meta.senseGloss.length > 0 - ? e.meta.senseGloss - : undefined; const senseGloss = senseGlossFromEntity ?? - senseGlossFromLegacyMeta ?? extractSenseGlossFromTranslationSection(e); - - let genus = extractGenusFromEntity(entity); - let phrasemeKind = extractPhrasemeKindFromEntity(entity); - - // Extract genus/phraseme kind from legacy linguisticUnit metadata if needed. - const lu = e.meta.linguisticUnit; - if (lu?.kind === "Lexem") { - const features = lu.surface.features; - if ( - !genus && - features.pos === "Noun" && - "genus" in features - ) { - genus = features.genus; - } - } else if (!phrasemeKind && lu?.kind === "Phrasem") { - phrasemeKind = lu.surface.features.phrasemeKind; - } return { - emojiDescription: Array.isArray(emojiDescription) - ? emojiDescription - : null, - genus, + emojiDescription, + genus: extractGenusFromEntity(entity), index: parsed.index, ipa, - phrasemeKind, + phrasemeKind: extractPhrasemeKindFromEntity(entity), pos: parsed.pos, senseGloss, unitKind: parsed.unitKind, @@ -251,19 +221,19 @@ export function disambiguateSense( ); } - // Edge case: all matching entries lack emojiDescription (V2 legacy) - // → backward compat: treat as re-encounter of the first match + // Hard cutover: if all matches are missing emojiDescription, + // we cannot disambiguate reliably, so treat this as a new sense. const sensesWithEmoji = senses.filter( (s) => s.emojiDescription !== null, ) as Array<(typeof senses)[number] & { emojiDescription: string[] }>; if (sensesWithEmoji.length === 0) { logger.info( - "[disambiguate] V2 legacy path — no emojiDescription on any sense", + "[disambiguate] Missing emojiDescription on all senses; treating as new sense", ); return ResultAsync.fromSafePromise( Promise.resolve({ - matchedIndex: senses[0]?.index, + matchedIndex: null, } as DisambiguationResult), ); } diff --git a/src/commanders/textfresser/common/closed-set-surface-hub.ts b/src/commanders/textfresser/common/closed-set-surface-hub.ts new file mode 100644 index 000000000..cf9a2a2c2 --- /dev/null +++ b/src/commanders/textfresser/common/closed-set-surface-hub.ts @@ -0,0 +1,478 @@ +import { err, ok, type Result } from "neverthrow"; +import { LANGUAGE_ISO_CODE } from "../../../linguistics/common/enums/core"; +import type { DeLexemPos } from "../../../linguistics/de"; +import type { VaultActionManager } from "../../../managers/obsidian/vault-action-manager"; +import { + type AnySplitPath, + SplitPathKind, + type SplitPathToFolder, + type SplitPathToMdFile, +} from "../../../managers/obsidian/vault-action-manager/types/split-path"; +import { + type VaultAction, + VaultActionKind, +} from "../../../managers/obsidian/vault-action-manager/types/vault-action"; +import type { TargetLanguage } from "../../../types"; +import type { PathLookupFn } from "./target-path-resolver"; + +export const CLOSED_SET_HUB_FOLDER = "closed-set-hub"; + +const CLOSED_SET_POS: ReadonlyArray = [ + "Pronoun", + "Article", + "Preposition", + "Conjunction", + "Particle", + "InteractionalUnit", +]; + +const CLOSED_SET_POS_KEBABS: ReadonlySet = new Set( + CLOSED_SET_POS.map((pos) => toPosKebab(pos)), +); + +const POS_LABEL_BY_KEBAB: ReadonlyMap = new Map( + CLOSED_SET_POS.map((pos) => [toPosKebab(pos), pos]), +); + +type HubActionsParams = { + surface: string; + targetLanguage: TargetLanguage; + lookupInLibrary: PathLookupFn; + vam: Pick; + currentClosedSetTarget?: SplitPathToMdFile | null; +}; + +type BackfillParams = { + targetLanguage: TargetLanguage; + lookupInLibrary: PathLookupFn; + vam: Pick< + VaultActionManager, + "exists" | "list" | "listAllFilesWithMdReaders" + >; +}; + +export function buildClosedSetSurfaceHubSplitPath( + surface: string, + targetLanguage: TargetLanguage, +): SplitPathToMdFile { + const langCode = LANGUAGE_ISO_CODE[targetLanguage]; + const normalizedSurface = normalizeSurface(surface); + + return { + basename: normalizedSurface, + extension: "md", + kind: SplitPathKind.MdFile, + pathParts: ["Worter", langCode, CLOSED_SET_HUB_FOLDER], + }; +} + +export function isClosedSetLibraryTarget( + splitPath: SplitPathToMdFile, + targetLanguage: TargetLanguage, +): boolean { + const langCode = LANGUAGE_ISO_CODE[targetLanguage]; + const root = splitPath.pathParts[0]; + const lang = splitPath.pathParts[1]; + const posSegment = splitPath.pathParts[2]; + + return ( + root === "Library" && + lang === langCode && + typeof posSegment === "string" && + CLOSED_SET_POS_KEBABS.has(posSegment) + ); +} + +export function extractSurfaceFromClosedSetTarget( + splitPath: SplitPathToMdFile, + targetLanguage: TargetLanguage, +): string | null { + if (!isClosedSetLibraryTarget(splitPath, targetLanguage)) { + return null; + } + + const basename = splitPath.basename.trim(); + if (basename.length === 0) { + return null; + } + + const posSegment = splitPath.pathParts[2]; + if (typeof posSegment !== "string") { + return null; + } + + const langCode = LANGUAGE_ISO_CODE[targetLanguage]; + const suffix = `-${posSegment}-${langCode}`; + const lowerBasename = basename.toLocaleLowerCase("de-DE"); + + if (lowerBasename.endsWith(suffix)) { + const surfaceCandidate = basename.slice(0, -suffix.length).trim(); + if (surfaceCandidate.length > 0) { + return normalizeSurface(surfaceCandidate); + } + } + + // Fallback is intentional for non-standard/manual basenames: + // treat the full basename as a surface candidate instead of dropping the entry. + return normalizeSurface(basename); +} + +export function buildClosedSetSurfaceHubContent(params: { + surface: string; + targetLanguage: TargetLanguage; + targets: ReadonlyArray; +}): string { + const normalizedSurface = normalizeSurface(params.surface); + const sortedTargets = [...params.targets].sort((left, right) => + toPathString(left).localeCompare(toPathString(right), "en"), + ); + + const targetLines = sortedTargets.map((target) => { + const label = formatPosLabel(target); + const targetPath = toPathString(target); + return `- [[${targetPath}|${normalizedSurface} (${label})]]`; + }); + + const disambiguationSection = + targetLines.length > 0 + ? `Possible closed-set targets:\n${targetLines.join("\n")}` + : "No closed-set targets available."; + + return [ + "---", + "textfresser:", + " kind: closed-set-surface-hub", + ` surface: ${normalizedSurface}`, + "---", + `# ${normalizedSurface}`, + "", + disambiguationSection, + "", + ].join("\n"); +} + +export function buildClosedSetSurfaceHubActions( + params: HubActionsParams, +): VaultAction[] { + const normalizedSurface = normalizeSurface(params.surface); + if (normalizedSurface.length === 0) { + return []; + } + + const hubPath = buildClosedSetSurfaceHubSplitPath( + normalizedSurface, + params.targetLanguage, + ); + const hubExists = params.vam.exists(hubPath); + + const targets = collectClosedSetTargetsForSurface({ + currentClosedSetTarget: params.currentClosedSetTarget, + lookupInLibrary: params.lookupInLibrary, + surface: normalizedSurface, + targetLanguage: params.targetLanguage, + }); + + if (targets.length === 0) { + if (!hubExists) { + return []; + } + return [ + { + kind: VaultActionKind.TrashMdFile, + payload: { splitPath: hubPath }, + }, + ]; + } + + if (targets.length === 1 && !hubExists) { + return []; + } + + const content = buildClosedSetSurfaceHubContent({ + surface: normalizedSurface, + targetLanguage: params.targetLanguage, + targets, + }); + + return [ + { + kind: VaultActionKind.UpsertMdFile, + payload: { + content, + splitPath: hubPath, + }, + }, + ]; +} + +export async function buildClosedSetSurfaceHubBackfillActions( + params: BackfillParams, +): Promise> { + const librarySurfacesResult = collectClosedSetSurfacesFromLibrary( + params.vam, + params.targetLanguage, + ); + if (librarySurfacesResult.isErr()) { + return err(librarySurfacesResult.error); + } + + const existingHubSurfacesResult = collectExistingHubSurfaces( + params.vam, + params.targetLanguage, + ); + if (existingHubSurfacesResult.isErr()) { + return err(existingHubSurfacesResult.error); + } + + const allSurfaces = dedupeStrings([ + ...librarySurfacesResult.value, + ...existingHubSurfacesResult.value, + ]); + const existingHubContentResult = await collectExistingHubContentByPath( + params.vam, + params.targetLanguage, + ); + if (existingHubContentResult.isErr()) { + return err(existingHubContentResult.error); + } + + const actions: VaultAction[] = []; + for (const surface of allSurfaces) { + actions.push( + ...buildClosedSetSurfaceHubActions({ + lookupInLibrary: params.lookupInLibrary, + surface, + targetLanguage: params.targetLanguage, + vam: params.vam, + }), + ); + } + + const idempotentActions = actions.filter((action) => { + if (action.kind !== VaultActionKind.UpsertMdFile) { + return true; + } + + const splitPath = action.payload.splitPath; + const existingContent = existingHubContentResult.value.get( + toPathString(splitPath), + ); + return existingContent !== action.payload.content; + }); + + return ok(idempotentActions); +} + +function collectClosedSetSurfacesFromLibrary( + vam: Pick, + targetLanguage: TargetLanguage, +): Result { + const folder = buildLibraryLanguageFolder(targetLanguage); + if (!vam.exists(folder)) { + return ok([]); + } + + const listed = vam.listAllFilesWithMdReaders(folder); + if (listed.isErr()) { + return err(listed.error); + } + + const surfaces: string[] = []; + for (const splitPath of listed.value) { + if (splitPath.kind !== SplitPathKind.MdFile) { + continue; + } + if (!isClosedSetLibraryTarget(splitPath, targetLanguage)) { + continue; + } + const surface = extractSurfaceFromClosedSetTarget( + splitPath, + targetLanguage, + ); + if (!surface) { + continue; + } + surfaces.push(surface); + } + + return ok(dedupeStrings(surfaces)); +} + +function collectExistingHubSurfaces( + vam: Pick, + targetLanguage: TargetLanguage, +): Result { + const folder = buildClosedSetHubFolder(targetLanguage); + if (!vam.exists(folder)) { + return ok([]); + } + + const listed = vam.list(folder); + if (listed.isErr()) { + return err(listed.error); + } + + const surfaces: string[] = []; + for (const splitPath of listed.value) { + if (splitPath.kind !== SplitPathKind.MdFile) { + continue; + } + surfaces.push(normalizeSurface(splitPath.basename)); + } + + return ok(dedupeStrings(surfaces)); +} + +function buildLibraryLanguageFolder( + targetLanguage: TargetLanguage, +): SplitPathToFolder { + return { + basename: LANGUAGE_ISO_CODE[targetLanguage], + kind: SplitPathKind.Folder, + pathParts: ["Library"], + }; +} + +function buildClosedSetHubFolder( + targetLanguage: TargetLanguage, +): SplitPathToFolder { + return { + basename: CLOSED_SET_HUB_FOLDER, + kind: SplitPathKind.Folder, + pathParts: ["Worter", LANGUAGE_ISO_CODE[targetLanguage]], + }; +} + +function collectClosedSetTargetsForSurface(params: { + surface: string; + targetLanguage: TargetLanguage; + lookupInLibrary: PathLookupFn; + currentClosedSetTarget?: SplitPathToMdFile | null; +}): SplitPathToMdFile[] { + const normalizedSurface = normalizeSurface(params.surface); + if (normalizedSurface.length === 0) { + return []; + } + + const candidates = buildSurfaceLookupCandidates(params.surface); + const fromLookup: SplitPathToMdFile[] = []; + for (const candidate of candidates) { + fromLookup.push(...params.lookupInLibrary(candidate)); + } + + if (params.currentClosedSetTarget) { + fromLookup.push(params.currentClosedSetTarget); + } + + const filtered = fromLookup.filter((splitPath) => { + const surface = extractSurfaceFromClosedSetTarget( + splitPath, + params.targetLanguage, + ); + return surface === normalizedSurface; + }); + + return dedupeSplitPaths(filtered); +} + +function dedupeSplitPaths( + splitPaths: ReadonlyArray, +): SplitPathToMdFile[] { + const unique: SplitPathToMdFile[] = []; + const seen = new Set(); + for (const splitPath of splitPaths) { + const key = toPathString(splitPath); + if (seen.has(key)) continue; + seen.add(key); + unique.push(splitPath); + } + return unique; +} + +async function collectExistingHubContentByPath( + vam: Pick, + targetLanguage: TargetLanguage, +): Promise, string>> { + const folder = buildClosedSetHubFolder(targetLanguage); + if (!vam.exists(folder)) { + return ok(new Map()); + } + + const listed = vam.listAllFilesWithMdReaders(folder); + if (listed.isErr()) { + return err(listed.error); + } + + const contentByPath = new Map(); + for (const splitPath of listed.value) { + if (splitPath.kind !== SplitPathKind.MdFile) { + continue; + } + + const readResult = await splitPath.read(); + if (readResult.isErr()) { + return err( + `Failed to read existing hub ${toPathString(splitPath)}: ${readResult.error.reason}`, + ); + } + + contentByPath.set(toPathString(splitPath), readResult.value); + } + + return ok(contentByPath); +} + +function dedupeStrings(values: ReadonlyArray): string[] { + const out: string[] = []; + const seen = new Set(); + for (const value of values) { + const trimmed = value.trim(); + if (trimmed.length === 0) { + continue; + } + if (seen.has(trimmed)) { + continue; + } + seen.add(trimmed); + out.push(trimmed); + } + return out.sort((left, right) => left.localeCompare(right, "en")); +} + +function buildSurfaceLookupCandidates(surface: string): string[] { + const trimmed = surface.trim(); + const normalized = normalizeSurface(surface); + const capitalized = capitalizeFirst(normalized); + return dedupeStrings([trimmed, normalized, capitalized]); +} + +function formatPosLabel(splitPath: SplitPathToMdFile): string { + const posSegment = splitPath.pathParts[2]; + if (typeof posSegment !== "string") { + return "Unknown"; + } + const mapped = POS_LABEL_BY_KEBAB.get(posSegment); + return mapped ?? posSegment; +} + +function toPathString( + splitPath: Pick, +): string { + return [...splitPath.pathParts, splitPath.basename].join("/"); +} + +function normalizeSurface(surface: string): string { + return surface.trim().toLocaleLowerCase("de-DE"); +} + +function capitalizeFirst(value: string): string { + const first = value.charAt(0); + if (first.length === 0) { + return value; + } + return `${first.toLocaleUpperCase("de-DE")}${value.slice(1)}`; +} + +function toPosKebab(pos: DeLexemPos): string { + return pos.replace(/([a-z0-9])([A-Z])/g, "$1-$2").toLowerCase(); +} diff --git a/src/commanders/textfresser/common/lemma-link-routing.ts b/src/commanders/textfresser/common/lemma-link-routing.ts index 92553173a..c825a0c09 100644 --- a/src/commanders/textfresser/common/lemma-link-routing.ts +++ b/src/commanders/textfresser/common/lemma-link-routing.ts @@ -125,7 +125,7 @@ export function formatLinkTarget( splitPath: SplitPathToMdFile, opts?: FormatLinkTargetOpts, ): string { - const libraryTargetStyle = opts?.libraryTargetStyle ?? "full-path"; + const libraryTargetStyle = opts?.libraryTargetStyle ?? "basename"; if (isLibraryPath(splitPath) && libraryTargetStyle === "full-path") { return [...splitPath.pathParts, splitPath.basename].join("/"); @@ -134,6 +134,37 @@ export function formatLinkTarget( return splitPath.basename; } +function splitPathKey(splitPath: SplitPathToMdFile): string { + return [...splitPath.pathParts, splitPath.basename].join("/"); +} + +function resolveLibraryRenderStyle(params: { + findByBasename: (basename: string) => SplitPathToMdFile[]; + splitPath: SplitPathToMdFile; +}): "full-path" | "basename" { + if (!isLibraryPath(params.splitPath)) { + return "basename"; + } + + const matches = params.findByBasename(params.splitPath.basename); + const uniqueMatchKeys = new Set( + matches.map((match) => splitPathKey(match)), + ); + if (uniqueMatchKeys.size <= 1) { + return "basename"; + } + + return "full-path"; +} + +function formatResolvedTarget(params: { + findByBasename: (basename: string) => SplitPathToMdFile[]; + splitPath: SplitPathToMdFile; +}): string { + const libraryTargetStyle = resolveLibraryRenderStyle(params); + return formatLinkTarget(params.splitPath, { libraryTargetStyle }); +} + export function computePrePromptTarget( params: PrePromptTargetParams, ): ComputedLinkTarget { @@ -148,7 +179,10 @@ export function computePrePromptTarget( const fromResolver = resolveLinkpathDest(surface, sourcePath); if (fromResolver) { return { - linkTarget: formatLinkTarget(fromResolver), + linkTarget: formatResolvedTarget({ + findByBasename: lookupInLibrary, + splitPath: fromResolver, + }), shouldCreatePlaceholder: false, splitPath: fromResolver, }; @@ -157,7 +191,10 @@ export function computePrePromptTarget( const libraryMatch = lookupInLibrary(surface)[0]; if (libraryMatch) { return { - linkTarget: formatLinkTarget(libraryMatch), + linkTarget: formatResolvedTarget({ + findByBasename: lookupInLibrary, + splitPath: libraryMatch, + }), shouldCreatePlaceholder: false, splitPath: libraryMatch, }; @@ -201,7 +238,10 @@ export function computeFinalTarget( const libraryMatch = existingMatches.find(isLibraryPath); if (libraryMatch) { return { - linkTarget: formatLinkTarget(libraryMatch), + linkTarget: formatResolvedTarget({ + findByBasename, + splitPath: libraryMatch, + }), shouldCreatePlaceholder: false, splitPath: libraryMatch, }; @@ -210,7 +250,10 @@ export function computeFinalTarget( const fromLibraryLookup = lookupInLibrary(lemma)[0]; if (fromLibraryLookup) { return { - linkTarget: formatLinkTarget(fromLibraryLookup), + linkTarget: formatResolvedTarget({ + findByBasename, + splitPath: fromLibraryLookup, + }), shouldCreatePlaceholder: false, splitPath: fromLibraryLookup, }; @@ -237,7 +280,10 @@ export function computeFinalTarget( }); return { - linkTarget: formatLinkTarget(computed), + linkTarget: formatResolvedTarget({ + findByBasename, + splitPath: computed, + }), shouldCreatePlaceholder: false, splitPath: computed, }; diff --git a/src/commanders/textfresser/common/morpheme-link-target.ts b/src/commanders/textfresser/common/morpheme-link-target.ts index 701efd675..5cd93c29e 100644 --- a/src/commanders/textfresser/common/morpheme-link-target.ts +++ b/src/commanders/textfresser/common/morpheme-link-target.ts @@ -6,23 +6,60 @@ */ import type { LlmMorpheme } from "../../../prompt-smith/schemas/morphem"; +import { wikilinkHelper } from "../../../stateless-helpers/wikilink"; import type { TargetLanguage } from "../../../types"; import type { MorphemeItem } from "../domain/morpheme/morpheme-formatter"; +const PREFIX_TARGET_SUFFIX = "-prefix-de"; + +function stripAnchor(value: string): string { + const anchorIndex = value.indexOf("#"); + return anchorIndex >= 0 ? value.slice(0, anchorIndex) : value; +} + +function stripPrefixTargetSuffix(value: string): string { + const lower = value.toLowerCase(); + if (!lower.endsWith(PREFIX_TARGET_SUFFIX)) { + return value; + } + return value.slice(0, -PREFIX_TARGET_SUFFIX.length); +} + +function buildPrefixTarget(value: string): string { + const lower = value.toLowerCase(); + return lower.endsWith(PREFIX_TARGET_SUFFIX) + ? lower + : `${lower}${PREFIX_TARGET_SUFFIX}`; +} + export function resolveMorphemeItems( morphemes: LlmMorpheme[], targetLang: TargetLanguage, ): MorphemeItem[] { return morphemes.map((m): MorphemeItem => { + const normalizedRawSurf = stripAnchor( + wikilinkHelper.normalizeLinkTarget(m.surf), + ); + const normalizedSurf = + m.kind === "Prefix" && targetLang === "German" + ? stripPrefixTargetSuffix(normalizedRawSurf) + : normalizedRawSurf; + const normalizedLemma = + typeof m.lemma === "string" + ? stripAnchor(wikilinkHelper.normalizeLinkTarget(m.lemma)) + : undefined; const base: MorphemeItem = { kind: m.kind, - surf: m.surf, - ...(m.lemma != null && { lemma: m.lemma }), + surf: normalizedSurf, + ...(normalizedLemma != null && { lemma: normalizedLemma }), ...(m.separability != null && { separability: m.separability }), }; if (m.kind === "Prefix" && targetLang === "German") { - return { ...base, linkTarget: `${m.surf.toLowerCase()}-prefix-de` }; + return { + ...base, + linkTarget: buildPrefixTarget(normalizedRawSurf), + }; } return base; diff --git a/src/commanders/textfresser/common/target-comparison.ts b/src/commanders/textfresser/common/target-comparison.ts new file mode 100644 index 000000000..ea8de615b --- /dev/null +++ b/src/commanders/textfresser/common/target-comparison.ts @@ -0,0 +1,6 @@ +/** + * Canonicalize link-like targets for case-insensitive comparison in Textfresser domain logic. + */ +export function canonicalizeTargetForComparison(target: string): string { + return target.trim().toLowerCase(); +} diff --git a/src/commanders/textfresser/domain/dict-note/internal/constants.ts b/src/commanders/textfresser/domain/dict-note/internal/constants.ts index efc66bf20..3e615d693 100644 --- a/src/commanders/textfresser/domain/dict-note/internal/constants.ts +++ b/src/commanders/textfresser/domain/dict-note/internal/constants.ts @@ -1,7 +1,7 @@ export const ENTRY_SEPARATOR = "\n\n\n---\n---\n\n\n"; -/** Splits on both old (`\n---\n---\n---\n`) and new (`\n\n---\n---\n\n`) separators. */ -export const ENTRY_SEPARATOR_RE = /\n+---\n---(?:\n---)*\n+/; +/** Splits only on the canonical DictEntry separator. */ +export const ENTRY_SEPARATOR_RE = /\n\n\n---\n---\n\n\n/; /** Legacy CSS class name — parsed as-is from note markup */ export const ENTRY_SECTION_CSS_CLASS = "entry_section_title"; diff --git a/src/commanders/textfresser/domain/dict-note/types.ts b/src/commanders/textfresser/domain/dict-note/types.ts index 34a0be395..30fa6f00d 100644 --- a/src/commanders/textfresser/domain/dict-note/types.ts +++ b/src/commanders/textfresser/domain/dict-note/types.ts @@ -12,7 +12,6 @@ export type EntrySection = { export type DictEntryMeta = { entity?: DeEntity; linguisticUnit?: GermanLinguisticUnit; - emojiDescription?: string[]; } & Record; export type DictEntry = { diff --git a/src/commanders/textfresser/domain/morpheme/morpheme-formatter.ts b/src/commanders/textfresser/domain/morpheme/morpheme-formatter.ts index 23878872b..4035b8f21 100644 --- a/src/commanders/textfresser/domain/morpheme/morpheme-formatter.ts +++ b/src/commanders/textfresser/domain/morpheme/morpheme-formatter.ts @@ -4,6 +4,7 @@ */ import type { MorphemeKind } from "../../../../linguistics/common/enums/linguistic-units/morphem/morpheme-kind"; +import { wikilinkHelper } from "../../../../stateless-helpers/wikilink"; import type { TargetLanguage } from "../../../../types"; export type MorphemeItem = { @@ -50,9 +51,10 @@ function formatAsWikilink( _targetLang: TargetLanguage, ): string { const display = item.surf; - const target = + const target = wikilinkHelper.normalizeLinkTarget( item.linkTarget ?? - (item.lemma && item.lemma !== item.surf ? item.lemma : item.surf); + (item.lemma && item.lemma !== item.surf ? item.lemma : item.surf), + ); if (target.toLowerCase() === display.toLowerCase()) return `[[${target}]]`; return `[[${target}|${display}]]`; diff --git a/src/commanders/textfresser/domain/propagation/note-adapter.ts b/src/commanders/textfresser/domain/propagation/note-adapter.ts index 7daef63c0..36859394c 100644 --- a/src/commanders/textfresser/domain/propagation/note-adapter.ts +++ b/src/commanders/textfresser/domain/propagation/note-adapter.ts @@ -247,7 +247,7 @@ export function serializeWikilinkDto(link: WikilinkDto): string { logSampledWarning({ context: { link }, message: - "[propagation-v2-note-adapter] Serializing wikilink with empty target", + "[propagation-note-adapter] Serializing wikilink with empty target", sampleKey: "empty-target", }); return "[[]]"; @@ -280,7 +280,7 @@ function extractWikilinkTokensFromText(text: string): string[] { wikilink: typeof fullMatch === "string" ? fullMatch : "", }, message: - "[propagation-v2-note-adapter] Skipping embedded wikilink during target extraction", + "[propagation-note-adapter] Skipping embedded wikilink during target extraction", sampleKey: fullMatch.trim(), }); continue; @@ -335,7 +335,7 @@ function serializePreservedWikilink(raw: string): string { raw, }, message: - "[propagation-v2-note-adapter] Failed to serialize preserved wikilink; emitting empty marker", + "[propagation-note-adapter] Failed to serialize preserved wikilink; emitting empty marker", sampleKey: raw.trim(), }); return "[[]]"; @@ -378,7 +378,7 @@ function parseRelationToken( token: trimmed, }, message: - "[propagation-v2-note-adapter] Preserving unsupported relation wikilink token", + "[propagation-note-adapter] Preserving unsupported relation wikilink token", sampleKey: `${relationKind}:${trimmed}`, }); return { @@ -539,7 +539,7 @@ function parseMorphologySection(rawContent: string): MorphologySectionDto { relationType: activeRelationType, }, message: - "[propagation-v2-note-adapter] Skipping unparseable morphology backlink line", + "[propagation-note-adapter] Skipping unparseable morphology backlink line", sampleKey: `${activeRelationType}:${trimmed}`, }); continue; @@ -645,6 +645,13 @@ function parseSectionsForEntry(sectionText: string): PropagationSection[] { title: marker.title, }; } + + return { + cssKind: marker.cssKind, + kind: "Raw", + rawBlock, + title: marker.title, + }; }); } diff --git a/src/commanders/textfresser/orchestration/background/background-generate-coordinator.ts b/src/commanders/textfresser/orchestration/background/background-generate-coordinator.ts index 293226ad7..8b21b18b1 100644 --- a/src/commanders/textfresser/orchestration/background/background-generate-coordinator.ts +++ b/src/commanders/textfresser/orchestration/background/background-generate-coordinator.ts @@ -121,12 +121,17 @@ export function createBackgroundGenerateCoordinator(params: { ): Promise { const targetExistedBefore = vam.exists(targetPath); + async function readTargetContent() { + // Single retry layer lives in VaultReader/TFileHelper. + return vam.readContent(targetPath); + } + async function cleanupIfEmpty(): Promise { const shouldCleanup = targetOwnedByInvocation || !targetExistedBefore; if (!shouldCleanup) return "skipped"; - const currentContent = await vam.readContent(targetPath); + const currentContent = await readTargetContent(); if (currentContent.isErr()) return "gone"; if (currentContent.value.trim().length > 0) return "has-content"; @@ -149,7 +154,7 @@ export function createBackgroundGenerateCoordinator(params: { return "deleted"; } - const contentResult = await vam.readContent(targetPath); + const contentResult = await readTargetContent(); const content = contentResult.isOk() ? contentResult.value : ""; const stateSnapshot: TextfresserState = { @@ -193,7 +198,7 @@ export function createBackgroundGenerateCoordinator(params: { ); } - const finalContentResult = await vam.readContent(targetPath); + const finalContentResult = await readTargetContent(); if (finalContentResult.isErr()) { throw new Error( "Background generate finished but target note could not be read", diff --git a/src/commanders/textfresser/orchestration/lemma/execute-lemma-flow.ts b/src/commanders/textfresser/orchestration/lemma/execute-lemma-flow.ts index f545636f0..6b8a68737 100644 --- a/src/commanders/textfresser/orchestration/lemma/execute-lemma-flow.ts +++ b/src/commanders/textfresser/orchestration/lemma/execute-lemma-flow.ts @@ -71,7 +71,7 @@ export function executeLemmaFlow(params: { .map(() => { const lemma = state.latestLemmaResult; if (!lemma) { - return; + return undefined; } state.latestLemmaInvocationCache = { @@ -98,6 +98,7 @@ export function executeLemmaFlow(params: { : ""; notify(`✓ ${lemma.lemma}${pos}`); requestBackgroundGenerate(notify); + return undefined; }) .mapErr((error) => { const reason = diff --git a/src/commanders/textfresser/orchestration/lemma/lemma-output-guardrails.ts b/src/commanders/textfresser/orchestration/lemma/lemma-output-guardrails.ts index b63ab1ae0..0a6b04c45 100644 --- a/src/commanders/textfresser/orchestration/lemma/lemma-output-guardrails.ts +++ b/src/commanders/textfresser/orchestration/lemma/lemma-output-guardrails.ts @@ -1,4 +1,5 @@ import { multiSpanHelper } from "../../../../stateless-helpers/multi-span"; +import { wikilinkHelper } from "../../../../stateless-helpers/wikilink"; import type { PromptOutput } from "../../llm/prompt-catalog"; type LemmaPromptOutput = PromptOutput<"Lemma">; @@ -88,6 +89,23 @@ function contextWithLinkedPartsMatches( ); } +function sanitizeLemmaOutput(output: LemmaPromptOutput): LemmaPromptOutput { + const normalizedLemma = wikilinkHelper.normalizeLinkTarget(output.lemma); + const normalizedContextWithLinkedParts = + typeof output.contextWithLinkedParts === "string" + ? wikilinkHelper.normalizeWikilinkTargetsInText( + output.contextWithLinkedParts, + ) + : undefined; + + return { + ...output, + contextWithLinkedParts: normalizedContextWithLinkedParts, + lemma: + normalizedLemma.length > 0 ? normalizedLemma : output.lemma.trim(), + }; +} + export function evaluateLemmaOutputGuardrails(params: { context: string; output: LemmaPromptOutput; @@ -96,31 +114,31 @@ export function evaluateLemmaOutputGuardrails(params: { const { context, output, surface } = params; const coreIssues: string[] = []; - let sanitizedOutput: LemmaPromptOutput = output; + let sanitizedOutput: LemmaPromptOutput = sanitizeLemmaOutput(output); let droppedContextWithLinkedParts = false; - const linkedParts = output.contextWithLinkedParts; + const linkedParts = sanitizedOutput.contextWithLinkedParts; if ( typeof linkedParts === "string" && linkedParts.length > 0 && !contextWithLinkedPartsMatches(context, linkedParts) ) { sanitizedOutput = { - ...output, + ...sanitizedOutput, contextWithLinkedParts: undefined, }; droppedContextWithLinkedParts = true; } - const normalizedLemma = normalizeGermanToken(output.lemma); + const normalizedLemma = normalizeGermanToken(sanitizedOutput.lemma); const normalizedSurface = normalizeGermanToken(surface); const hasSameSurfaceLemma = normalizedLemma === normalizedSurface; if ( hasSameSurfaceLemma && - output.linguisticUnit === "Lexem" && - output.posLikeKind === "Verb" && - output.surfaceKind === "Inflected" && + sanitizedOutput.linguisticUnit === "Lexem" && + sanitizedOutput.posLikeKind === "Verb" && + sanitizedOutput.surfaceKind === "Inflected" && hasSeparableVerbEvidence(context, surface) ) { coreIssues.push( @@ -130,9 +148,9 @@ export function evaluateLemmaOutputGuardrails(params: { if ( hasSameSurfaceLemma && - output.linguisticUnit === "Lexem" && - output.posLikeKind === "Adjective" && - output.surfaceKind === "Inflected" && + sanitizedOutput.linguisticUnit === "Lexem" && + sanitizedOutput.posLikeKind === "Adjective" && + sanitizedOutput.surfaceKind === "Inflected" && looksComparativeOrSuperlative(surface) ) { coreIssues.push( diff --git a/src/commanders/textfresser/orchestration/shared/dispatch-actions.ts b/src/commanders/textfresser/orchestration/shared/dispatch-actions.ts index 486a7bd90..ce2994772 100644 --- a/src/commanders/textfresser/orchestration/shared/dispatch-actions.ts +++ b/src/commanders/textfresser/orchestration/shared/dispatch-actions.ts @@ -6,6 +6,28 @@ import type { import type { CommandError } from "../../commands/types"; import { CommandErrorKind } from "../../errors"; +function stringifySplitPath(splitPath: { + pathParts: string[]; + basename: string; +}): string { + return [...splitPath.pathParts, splitPath.basename].join("/"); +} + +function formatDispatchFailure(failure: { + action: VaultAction; + error: string; +}): string { + const action = failure.action; + switch (action.kind) { + case "RenameFolder": + case "RenameFile": + case "RenameMdFile": + return `${action.kind}(${stringifySplitPath(action.payload.from)} -> ${stringifySplitPath(action.payload.to)}): ${failure.error}`; + default: + return `${action.kind}(${stringifySplitPath(action.payload.splitPath)}): ${failure.error}`; + } +} + export function dispatchActions( vam: VaultActionManager, actions: VaultAction[], @@ -14,7 +36,7 @@ export function dispatchActions( vam.dispatch(actions).then((dispatchResult) => { if (dispatchResult.isErr()) { const reason = dispatchResult.error - .map((e) => e.error) + .map((failure) => formatDispatchFailure(failure)) .join(", "); return err({ kind: CommandErrorKind.DispatchFailed, diff --git a/src/commanders/textfresser/state/textfresser-state.ts b/src/commanders/textfresser/state/textfresser-state.ts index 67a5566ec..053f1948d 100644 --- a/src/commanders/textfresser/state/textfresser-state.ts +++ b/src/commanders/textfresser/state/textfresser-state.ts @@ -42,6 +42,7 @@ export type TextfresserState = { inFlightGenerate: InFlightGenerate | null; pendingGenerate: PendingGenerate | null; languages: LanguagesConfig; + isLibraryLookupAvailable: boolean; lookupInLibrary: PathLookupFn; promptRunner: PromptRunner; vam: VaultActionManager; @@ -57,6 +58,7 @@ export function createInitialTextfresserState(params: { return { attestationForLatestNavigated: null, inFlightGenerate: null, + isLibraryLookupAvailable: false, languages, latestFailedSections: [], latestLemmaInvocationCache: null, diff --git a/src/commanders/textfresser/textfresser.ts b/src/commanders/textfresser/textfresser.ts index 3e1514961..941419d92 100644 --- a/src/commanders/textfresser/textfresser.ts +++ b/src/commanders/textfresser/textfresser.ts @@ -103,6 +103,7 @@ export class Textfresser { } this.scrollToTargetBlock(); } + return undefined; }) .mapErr((error) => { const reason = @@ -130,6 +131,12 @@ export class Textfresser { setLibrarianLookup(fn: PathLookupFn): void { this.state.lookupInLibrary = fn; + this.state.isLibraryLookupAvailable = true; + } + + clearLibrarianLookup(): void { + this.state.lookupInLibrary = () => []; + this.state.isLibraryLookupAvailable = false; } private scrollToTargetBlock(): void { diff --git a/src/documentaion/book-of-work/book-of-work.md b/src/documentaion/book-of-work/book-of-work.md new file mode 100644 index 000000000..6d36d63b4 --- /dev/null +++ b/src/documentaion/book-of-work/book-of-work.md @@ -0,0 +1,172 @@ +# Book Of Work + +## Compatibility Policy (Dev Mode, 2026-02-20) + +- Textfresser is treated as green-field. Breaking changes are allowed; no backward-compatibility guarantees for Textfresser note formats, schemas, or intermediate contracts. +- Librarian and VAM are stability-critical infrastructure. Changes there require conservative rollout, migration planning when persisted contracts change, and explicit regression coverage. + +## Pre-Extension Stabilization Backlog (Audit 2026-02-19) + +Source: local health-check run before new Textfresser feature work. + +### P0 (blockers) + +#### 0.1) Fix order-dependent unit tests caused by leaked module mocks +- Symptom: tests pass in isolation but fail when specific files are run together. +- Repro: + - `bun test --preload ./tests/unit/setup.ts tests/unit/textfresser/steps/propagate-generated-sections.test.ts tests/unit/textfresser/steps/generate-reencounter-sections.test.ts` + - `bun test --preload ./tests/unit/setup.ts tests/unit/textfresser/steps/propagate-generated-sections.test.ts tests/unit/textfresser/steps/decorate-attestation-separability.test.ts` +- Root cause: top-level `mock.module(...)` in `/Users/annagorelova/work/Textfresser_vault/.obsidian/plugins/textfresser/tests/unit/textfresser/steps/propagate-generated-sections.test.ts` shadows real modules across the process. +- Follow-up: + - Isolate mocks per test (or use scoped dynamic imports + restore discipline). + - Audit other top-level `mock.module(...)` users for similar bleed. + +#### 0.2) Restore `src/` TypeScript compile health +- Current state: `bun x tsc --noEmit -skipLibCheck` reports production-code errors in `src/` (not only tests). +- High-priority breakpoints: + - `/Users/annagorelova/work/Textfresser_vault/.obsidian/plugins/textfresser/src/commanders/librarian/codecs/locator/internal/from.ts` + - `/Users/annagorelova/work/Textfresser_vault/.obsidian/plugins/textfresser/src/commanders/librarian/commands/split-in-blocks.ts` + - `/Users/annagorelova/work/Textfresser_vault/.obsidian/plugins/textfresser/src/commanders/librarian/healer/library-tree/tree-action/bulk-vault-action-adapter/layers/library-scope/codecs/events/make-event-vault-scoped.ts` + - `/Users/annagorelova/work/Textfresser_vault/.obsidian/plugins/textfresser/src/managers/obsidian/behavior-manager/checkbox-behavior.ts` + - `/Users/annagorelova/work/Textfresser_vault/.obsidian/plugins/textfresser/src/managers/overlay-manager/index.ts` + +### P1 (stability + correctness) + +#### 1.1) Resolve stale separability-decoration tests vs current contract +- Current implementation says no visual alias markers are added for multi-span separable verbs: + - `/Users/annagorelova/work/Textfresser_vault/.obsidian/plugins/textfresser/src/commanders/textfresser/commands/generate/steps/decorate-attestation-separability.ts` +- Existing tests still expect `>` / `<` decorations: + - `/Users/annagorelova/work/Textfresser_vault/.obsidian/plugins/textfresser/tests/unit/textfresser/steps/decorate-attestation-separability.test.ts` +- Decision needed: reintroduce marker behavior or update tests to the current no-marker design. + +#### 1.2) Make `typecheck:changed` script deterministic and actionable +- Current script: + - `/Users/annagorelova/work/Textfresser_vault/.obsidian/plugins/textfresser/scripts/typecheck-changed.sh` +- Issue: final `grep` can produce exit code 1 with no useful output (false-fail behavior for "no matched diagnostics"). +- Follow-up: preserve non-empty diagnostics and avoid grep-only exit semantics as the pass/fail source. +- Update (2026-02-19): Completed. + - Replaced grep-driven pass/fail with `tsc` exit-status + filtered diagnostics for changed files only. + - Added bash-3-compatible implementation (no `mapfile`) and explicit "no relevant changed-file errors" output path. + +#### 1.3) Return lint to green +- Current state: `bun run lint` fails. +- Notable error/warning hotspots: + - `/Users/annagorelova/work/Textfresser_vault/.obsidian/plugins/textfresser/src/commanders/textfresser/domain/propagation/note-adapter.ts` + - `/Users/annagorelova/work/Textfresser_vault/.obsidian/plugins/textfresser/src/commanders/textfresser/orchestration/lemma/execute-lemma-flow.ts` + - `/Users/annagorelova/work/Textfresser_vault/.obsidian/plugins/textfresser/src/commanders/textfresser/textfresser.ts` + - `/Users/annagorelova/work/Textfresser_vault/.obsidian/plugins/textfresser/src/stateless-helpers/wikilink.ts` + +### P2 (maintainability / debt) + +#### 2.1) Decompose large multi-responsibility modules +- Largest active hotspots in runtime code: + - `/Users/annagorelova/work/Textfresser_vault/.obsidian/plugins/textfresser/src/commanders/textfresser/domain/propagation/note-adapter.ts` + - `/Users/annagorelova/work/Textfresser_vault/.obsidian/plugins/textfresser/src/commanders/textfresser/commands/generate/steps/generate-new-entry-sections.ts` + - `/Users/annagorelova/work/Textfresser_vault/.obsidian/plugins/textfresser/src/main.ts` +- Follow-up: split parser/serializer/normalization/section-dispatch concerns into focused modules with targeted tests. + +#### 2.2) Reduce risky cast footprint in critical pathways +- Health-check snapshot found high cast volume (`as any`, `as unknown as`, `@ts-expect-error`) across runtime and tests. +- Start with runtime modules touched by P0/P1 fixes, then tighten tests. + +## Deferred Follow-Ups (Morphological Relations) + +### 1) Prefix derivations: avoid redundant `` with equation +- Status: Deferred by request. +- Current behavior: prefix cases can render both: + - `` `[[base]]` + - `[[prefix|decorated]] + [[base]] = [[source]] *(gloss)*` +- Follow-up decision to implement later: for inferred prefix derivations, render only the equation and skip ``. + +### 2) Architecture doc table sync for Lexem POS section coverage +- Status: Deferred by request. +- Gap: `sectionsForLexemPos` in code includes `Morphology` for all Lexem POS, but the table in `/Users/annagorelova/work/Textfresser_vault/.obsidian/plugins/textfresser/src/documentaion/linguistics-and-prompt-smith-architecture.md` is only partially updated. +- Follow-up to implement later: update all POS rows in the table so docs exactly match `section-config.ts`. + +--- + +## Post-Migration Cleanup (from PR review, 2026-02-19) + +Source: ideas extracted from open PRs #6, #7, #13, #15, #16, #17, #22 in clockblocker/filler-de. + +### Immediate (low effort) + +#### 3) Rename `src/documentaion/` → `src/documentation/` +- Source: PR #6 ideas (#10) +- Typo directory still exists alongside correctly-spelled `documentation/` at repo root. Bulk rename of 6+ files + CLAUDE.md refs. Standalone commit. + +#### 4) Healer `as any` elimination +- Source: PR #6 ideas (#8) +- Two `makeNodeSegmentId(node as any)` casts in `healer.ts` and `leaf-move-healing.ts`. Fix with overloads on `makeNodeSegmentId`. + +#### 5) Placeholder command cleanup in `main.ts` +- Source: PR #6 ideas (#7) +- `fill-template` and `duplicate-selection` permanently return `false`. `check-ru-de-translation` and `check-schriben` are TODO stubs. Remove or gate behind dev flag. + +#### 6) Fold `serializeDictNote()` into `dictNoteHelper` facade +- Source: PR #7 ideas +- `serialize-dict-note.ts` is a thin wrapper. Add `dictNoteHelper.serializeToString()`, delete the file, update 5 imports. + +#### 7) Add API timeout to `generate()` call +- Source: PR #22 +- `client.chat.completions.parse()` in `api-service.ts` has no timeout. Wrap in `Promise.race` with 30-60s timeout. +- Update (2026-02-19): Completed. + - Added `GENERATE_TIMEOUT_MS = 45_000` and wrapped `client.chat.completions.parse()` in a timeout helper. + - Timeout errors are marked retryable in existing `withRetry` flow. + +#### 8) `literals.ts` — consider killing or splitting by domain +- Source: PR #6 ideas (#4) +- 760-line flat file. Originally needed for Zod v4 considerations — may no longer be necessary. Either kill it or split into `types/literals/linguistic.ts`, `ui.ts`, `commands.ts` + barrel re-export. + +#### 9) Error type consolidation +- Source: PR #6 ideas (#2) +- `TextfresserCommandError` and `LibrarianCommandError` are independent types with similar patterns. Shared `BaseCommandError` enables unified error reporting. + +#### 10) Add `@generated` header to prompt-smith codegen output +- Source: PR #6 ideas (#9) +- `src/prompt-smith/index.ts` is generated but committed alongside hand-written code with no marker. + +### Bug fixes to verify against current code + +#### 11) Unsafe `error.message` access in catch blocks +- Source: PR #6 fix +- Catch blocks accessing `error.message` without `instanceof Error` narrowing. Check `src/main.ts` and `tfile-helper.ts`. +- Update (2026-02-19): Verified for scoped files. + - Audited `src/main.ts` and `src/managers/obsidian/vault-action-manager/file-services/background/helpers/tfile-helper.ts`. + - No unsafe `catch` access of `error.message` remains in those two files; no code change required. + +#### 12) Event listener leak in `whenMetadataResolved()` +- Source: PR #6 fix +- `this.app.metadataCache.off("resolved", () => null)` passes anonymous fn that never matches. Listener never removed. Check if still present in `src/main.ts`. + +### Needs separate discussion + +#### 13) Extract propagation action collection helper +- Source: PR #7 ideas +- The `push healingActions + push buildPropagationActionPair + return ok(ctx)` epilogue is copy-pasted across all 4 propagation steps (~32 lines). Extract into shared function. + +#### 14) Audit in-progress loading indicator +- Source: PR #22 +- The `notify` callback communicates results/errors but may not show "loading..." state during the API call itself. Worth auditing whether Lemma/Generate flows show in-progress UI. + +#### 15) Codec factory `createEventCodec()` +- Source: PR #6 ideas (#3) +- 8 event codecs in `user-event-interceptor/events/` reimplementing similar encode/decode. A factory would reduce boilerplate. Marginal — works fine as-is. + +### Longer-term + +#### 16) `generate-sections.ts` decomposition +- Source: PR #6 ideas (#5) +- Was 716 lines. V2 pipeline already extracted section formatters — check current size. If still large, split per-section-kind generators. + +#### 17) `splitPath` as-cast reduction +- Source: PR #6 ideas (#6) +- 96 `as` casts in split-path codec chains. Use overloads + discriminated unions. High effort but aligned with CLAUDE.md's explicit pattern. + +#### 18) Unit test coverage expansion +- Source: PR #6 ideas (#1) +- Pure-logic modules first: codecs, vault-action-queue, section-healing. No Obsidian runtime deps needed. + +#### 19) Textfresser wikilink resolution (spec-driven) +- Detailed design is maintained in `/Users/annagorelova/work/Textfresser_vault/.obsidian/plugins/textfresser/src/documentaion/specs/textfresser-wikilink-resolution-spec.md`. +- Keep BOW at pointer level only; implementation decisions and edge-case policy live in the spec. diff --git a/src/documentaion/book-of-work/ideas-to-consider.md b/src/documentaion/book-of-work/ideas-to-consider.md new file mode 100644 index 000000000..7150c8a6c --- /dev/null +++ b/src/documentaion/book-of-work/ideas-to-consider.md @@ -0,0 +1,12 @@ +--- + Reviewed: 2026-02-19 (from open PRs #6, #7, #13, #15, #16, #17, #22) + Accepted items moved to: documentation/book-of-work.md + + Rejected: + + #: 2 + Item: Remove vestigial apiProvider setting + Source: PR #22 + What: apiProvider: "google" is a typed literal with one option. Settings tab renders a pointless + dropdown. Remove from types.ts, settings-tab.ts, api-service.ts, and locales. + Decision: Rejected. apiProvider will be expanded later diff --git a/src/documentaion/commands-and-behaviors-architecture.md b/src/documentaion/commands-and-behaviors-architecture.md index 2f21c57b3..06b1645cb 100644 --- a/src/documentaion/commands-and-behaviors-architecture.md +++ b/src/documentaion/commands-and-behaviors-architecture.md @@ -1,6 +1,10 @@ # Commands & Behaviors — Architecture > **Scope**: This document covers `command-executor/` and `behavior-manager/` — the layer between raw user events and the two commanders (Librarian, Textfresser). For the event detection layer, see `UserEventInterceptor`. For the file system layer, see `vam-architecture.md`. +> +> **Compatibility Policy (Dev Mode, 2026-02-20)**: +> - Textfresser is treated as green-field. Breaking changes are allowed; no backward-compatibility guarantees for Textfresser note formats, schemas, or intermediate contracts. +> - Librarian and VAM are stability-critical infrastructure. Changes there require conservative rollout, migration planning when persisted contracts change, and explicit regression coverage. --- @@ -50,7 +54,7 @@ src/managers/obsidian/ `CommandKind` is a Zod enum merging all Librarian and Textfresser command kind strings: ``` -Librarian: GoToPrevPage, GoToNextPage, MakeText, SplitToPages, SplitInBlocks +Librarian: GoToPrevPage, GoToNextPage, SplitToPages, SplitInBlocks Textfresser: TranslateSelection, Generate, Lemma ``` @@ -107,7 +111,7 @@ Three possible outcomes: `Handled` (consumed), `Passthrough` (native behavior), |----------|-------------|---------------------|------------| | **codex-checkbox** | `CheckboxClicked` | `librarian.isCodexInsideLibrary(splitPath)` | `librarian.handleCodexCheckboxClick(payload)` | | **checkbox-frontmatter** | `CheckboxInFrontmatterClicked` | Always applies | `librarian.handlePropertyCheckboxClick(payload)` | -| **wikilink-completion** | `WikilinkCompleted` | Always applies | Three-step resolution: suffix alias → Obsidian resolve → corename tree lookup (uses `pickClosestLeaf` for disambiguation) | +| **wikilink-completion** | `WikilinkCompleted` | Always applies | Three-step resolution: suffix alias → Obsidian resolve → corename tree lookup. On a single match, auto-resolves via `pickClosestLeaf`; on ambiguous matches, returns passthrough (no silent auto-pick). | ### 4.4 Textfresser Behavior diff --git a/src/documentaion/e2e-architecture.md b/src/documentaion/e2e-architecture.md index 367cf290a..ab22ed55b 100644 --- a/src/documentaion/e2e-architecture.md +++ b/src/documentaion/e2e-architecture.md @@ -1,6 +1,10 @@ # E2E Testing — Architecture (CLI-Based) > **Scope**: This document covers the CLI-based end-to-end testing infrastructure for the Textfresser plugin. For the file system abstraction layer, see `vam-architecture.md`. For vocabulary/dictionary commands, see `textfresser-architecture.md`. +> +> **Compatibility Policy (Dev Mode, 2026-02-20)**: +> - Textfresser is treated as green-field. Breaking changes are allowed; no backward-compatibility guarantees for Textfresser note formats, schemas, or intermediate contracts. +> - Librarian and VAM are stability-critical infrastructure. Changes there require conservative rollout, migration planning when persisted contracts change, and explicit regression coverage. --- diff --git a/src/documentaion/error-contract-book-of-work.md b/src/documentaion/error-contract-book-of-work.md index 18060e29c..77ef29609 100644 --- a/src/documentaion/error-contract-book-of-work.md +++ b/src/documentaion/error-contract-book-of-work.md @@ -2,6 +2,11 @@ Track error-shape debt where APIs currently return unstable string messages and callers must infer semantics from text. +## Compatibility Policy (Dev Mode, 2026-02-20) + +- Textfresser is treated as green-field. Breaking changes are allowed; no backward-compatibility guarantees for Textfresser note formats, schemas, or intermediate contracts. +- Librarian and VAM are stability-critical infrastructure. Changes there require conservative rollout, migration planning when persisted contracts change, and explicit regression coverage. + ## Cases | ID | Area | Current Contract | Current Workaround | Desired Contract | Priority | Status | @@ -12,13 +17,13 @@ Track error-shape debt where APIs currently return unstable string messages and Primary landing points: -- `src/commanders/textfresser/commands/generate/steps/propagation-v2-ports-adapter.ts` +- `src/commanders/textfresser/commands/generate/steps/propagation-ports-adapter.ts` - `src/managers/obsidian/vault-action-manager/impl/vault-reader.ts` Rationale: - String-message matching was fragile (localization/wording drift). -- Propagation v2 keeps race-safe behavior (`exists` true, then vanished file) by mapping typed `FileNotFound` reads to `Missing`. +- Propagation keeps race-safe behavior (`exists` true, then vanished file) by mapping typed `FileNotFound` reads to `Missing`. Landed shape: diff --git a/src/documentaion/librarian-architecture.md b/src/documentaion/librarian-architecture.md index 7529793c4..5dc0c4498 100644 --- a/src/documentaion/librarian-architecture.md +++ b/src/documentaion/librarian-architecture.md @@ -1,6 +1,10 @@ # Librarian Tree & Healing System — Architecture > **Scope**: This document covers the tree/healing/codex half of the plugin (the "Librarian" commander). For the vocabulary/dictionary half, see `textfresser-architecture.md`. For E2E testing, see `e2e-architecture.md`. +> +> **Compatibility Policy (Dev Mode, 2026-02-20)**: +> - Textfresser is treated as green-field. Breaking changes are allowed; no backward-compatibility guarantees for Textfresser note formats, schemas, or intermediate contracts. +> - Librarian and VAM are stability-critical infrastructure. Changes there require conservative rollout, migration planning when persisted contracts change, and explicit regression coverage. --- @@ -527,7 +531,7 @@ The system supports two metadata storage formats: ``` Placed at the bottom of the file with 20 newlines of padding (invisible in Obsidian). -**YAML Frontmatter** (visible, legacy): +**YAML Frontmatter** (visible): ```yaml --- status: Done @@ -539,7 +543,6 @@ prevPageIdx: 42 - **Read**: tries JSON first, falls back to YAML - **Write**: respects `rules.hideMetadata` setting (JSON if true, YAML if false) -- **Migration**: at init, converts between formats based on settings ### 11.3 Public API @@ -635,17 +638,15 @@ On plugin load, the Librarian rebuilds the entire tree from vault state: d. Canonicalize to destination e. Build locator → CreateTreeLeafAction f. Read metadata → extract status (Done/NotStarted) - g. Detect metadata format → generate migration action if needed 5. Apply all CreateTreeLeafActions via HealingTransaction 6. Find invalid codex files → delete actions 7. Scan for orphaned codexes → cleanup actions 8. Subscribe to vault events (BEFORE dispatch — catches cascading events) 9. Process codex impacts → deletions + recreations -10. Build page navigation migration actions -11. Assemble: healing + codex + backlink + migration actions -12. Single vam.dispatch(allVaultActions) -13. Wait for queue to drain -14. Commit transaction +10. Assemble: healing + codex + backlink actions +11. Single vam.dispatch(allVaultActions) +12. Wait for queue to drain +13. Commit transaction ``` **Source**: `src/commanders/librarian/librarian.ts:init()`, `src/commanders/librarian/librarian-init/` diff --git a/src/documentaion/linguistics-and-prompt-smith-architecture.md b/src/documentaion/linguistics-and-prompt-smith-architecture.md index ee7d49853..dcec2812c 100644 --- a/src/documentaion/linguistics-and-prompt-smith-architecture.md +++ b/src/documentaion/linguistics-and-prompt-smith-architecture.md @@ -1,6 +1,10 @@ # Linguistics & Prompt-Smith — Architecture > **Scope**: This document covers the **linguistics type system** (`src/linguistics/`) and the **prompt management layer** (`src/prompt-smith/`). These two modules form the foundational layer that both the Lemma and Generate commands depend on. For the command pipeline itself, see `textfresser-architecture.md`. For FS dispatch, see `vam-architecture.md`. +> +> **Compatibility Policy (Dev Mode, 2026-02-20)**: +> - Textfresser is treated as green-field. Breaking changes are allowed; no backward-compatibility guarantees for Textfresser note formats, schemas, or intermediate contracts. +> - Librarian and VAM are stability-critical infrastructure. Changes there require conservative rollout, migration planning when persisted contracts change, and explicit regression coverage. --- @@ -256,7 +260,7 @@ GermanPrefixFullFeaturesSchema = z.object({ `de/lexem/verb/features.ts` defines structured lemma-only verb features: ```typescript -GermanVerbConjugation = "Irregular" | "Rregular" +GermanVerbConjugation = "Irregular" | "Regular" GermanVerbSeparability = "Separable" | "Inseparable" | "None" GermanVerbReflexivity = "NonReflexive" | "ReflexiveOnly" | "OptionalReflexive" @@ -649,7 +653,7 @@ type AgentOutput = z.inferMorphologische Relationen - -[[Zusammenarbeit]] *(cooperation)* -``` - -Subsequently, when the user ran lemma for "Arbeit", the system: -1. Correctly resolved "Arbeit" as a lemma -2. Found the existing `Arbeit.md` file -3. Treated it as a re-encounter → skipped LLM generation -4. Did NOT append attestation or enrich the entry - -**Result**: `Arbeit.md` remains a bare propagation stub — no header, no IPA, no article, no inflection table, no translation, no attestation. - -**Expected**: Either: -- (a) Recognize that the existing file is a propagation stub (no `textfresser_meta` section) and run full generation anyway, OR -- (b) The re-encounter path should detect that the entry is incomplete and trigger enrichment - -**Impact**: Any word that appears first as a propagation target and then as a direct user lookup will be permanently stuck as a stub. This affects common root words (Arbeit, Bau, Arbeiter, etc.) that are likely to be propagation targets before they're directly looked up. - -**Possible fix**: In `resolve-existing-entry.ts`, check for the presence of `textfresser_meta` section. If absent, treat the file as "no existing entry" and proceed with full generation, merging in any existing propagation content. - -### ISSUE-4 (informational): Same entries from different derivation paths - -Not a bug, but notable: "Arbeiter" was created as a propagation target from "Bauarbeiter" with just semantic + morphological relations. If the user later looks up "Arbeiter" directly, it will hit the same ISSUE-3 (stub blocks full generation). - -Similarly, propagation targets like "Bau", "Zusammen", "Neu", "vorstellen", "schreiben" are all stubs that would need full generation if the user encounters them. - -## Cross-Sentence Observations - -1. **Second lemma in same sentence**: Works correctly. After "Fahrer" inserts `[[Fahrer]]`, the "Abfahrt" lemma correctly finds the surface in the modified content. -2. **Compound morpheme propagation**: Working well. Bauarbeiter propagates to Bau, Arbeiter, en. Zusammenarbeit propagates to Zusammen, Arbeit. Neubau propagates to Neu, Bau (+ spurious b). -3. **Semantic relation propagation**: Working correctly and bidirectionally. All synonym/antonym/hypernym/hyponym targets get backlink entries. -4. **Inflection propagation**: Working. Plural/genitive forms get proper inflection entries (e.g., Abfahrten, Bauarbeiters, Neubauten, Vorstellungen, Zusammenarbeiten). - -## Verdict - -The pipeline is **functionally working** post-migration. All lemma commands succeed, wikilinks are inserted correctly, entries are generated with full content, and propagation creates bidirectional cross-references. - -**Critical fix needed**: ISSUE-3 (propagation stubs blocking full generation) affects the most common use case — root words that are compound parts. This should be fixed before heavy use. - -**Nice-to-fix**: ISSUE-1 (Unterschrift classification) is an LLM prompt quality issue that will affect other derived nouns (Bewegung, Erfahrung, Bedeutung, etc.). - -## Runner Script - -The smoke test runner is at `tests/cli-e2e/textfresser/smoke-test-runner.ts`. Can be re-run with: - -```bash -CLI_E2E_VAULT=cli-e2e-test-vault CLI_E2E_VAULT_PATH=/Users/annagorelova/work/obsidian/cli-e2e-test-vault \ - bun run tests/cli-e2e/textfresser/smoke-test-runner.ts -``` diff --git a/src/documentaion/propagation-migration/propagation-v2-architecture.md b/src/documentaion/propagation-migration/propagation-v2-architecture.md deleted file mode 100644 index 3aec3c6f6..000000000 --- a/src/documentaion/propagation-migration/propagation-v2-architecture.md +++ /dev/null @@ -1,795 +0,0 @@ -# Textfresser Propagation V2 - Architecture - -> Scope: This document defines the v2 propagation architecture for Textfresser Generate. It covers propagation only (from a newly generated source entry to referenced target notes). It does not change Lemma routing, section generation prompts, or VAM dispatch internals. - ---- - -## 1. Status - -Design status: locked for v1 implementation. - -This document is the source of truth for: - -- Pipeline shape and phase boundaries -- IO vs pure algebra responsibilities -- Matching identity precedence -- Intent model and merge semantics -- Migration plan from the current propagation step - ---- - -## 2. Goals and Non-Goals - -### 2.1 Goals - -1. Keep propagation deterministic, idempotent, and one-hop. -2. Keep section algebra pure (no Obsidian API, no VAM calls, no file reads/writes). -3. Support per-`` specialization while preserving one shared pipeline. -4. Apply all updates with one write action per target note. -5. Make propagation testable with fast DTO-only unit tests. - -### 2.2 Non-Goals (v1) - -1. No recursive/transitive propagation. -2. No back-propagation from mutated target notes. -3. No global refactor of all section parsers in one shot. -4. No requirement to fully type every section on day one. - ---- - -## 3. Core Invariants - -1. One-hop only: -- A source entry generates intents. -- Intents are applied to targets. -- The run ends. No second-wave propagation. - -2. Algebra purity: -- `src/linguistics/**` propagator logic is pure DTO in/DTO out. -- Only orchestrator/adapters may do IO and VAM action creation. - -3. Identity precedence: -- `byStableId -> byDeterministicKey -> byHeader (fallback + warn)`. - -4. Idempotency: -- Running the same propagation twice with same source/targets produces no changes on second run. - -5. One target write: -- All intents for a target note are merged and applied before one final `VaultAction`. - ---- - -## 4. Architectural Split - -Propagation is one phase with three internal subparts: - -1. Hydrate (IO): -- Resolve unresolved semantic targets to concrete note paths. -- Read target notes. -- Parse notes into DTO form. - -2. Algebra (Pure): -- Build propagation intents per section. -- Match/create target entries in parsed DTO. -- Apply section mutations via deterministic merge rules. - -3. Materialize (IO-bound output): -- Serialize canonical markdown. -- Build VAM dispatch actions (plus path-healing actions when needed). - -Important: "pure" means no file/path API calls inside propagator algebra. - -Stage mapping: - -1. Algebra: Stage 1 (intent collection from source sections). -2. Hydrate: Stage 2 (path resolution) and Stage 6.1 (read/parse target note). -3. Materialize: Stage 6.3 (serialize) and Stage 7 (emit actions). - -Stages 3-5 (validation, dedupe, grouping) are orchestration control stages spanning the phase. - ---- - -## 5. Pipeline - -```text -Source DictEntry - -> Collect unresolved intents by section (pure) - -> Resolve target paths (IO) - -> Validate intents - -> Dedupe intents by intentKey - -> Group intents by targetPath - -> For each target: - read + parse target note - match/create target entry - apply section mutations - serialize canonical note - emit one VaultAction - -> Dispatch via VAM -``` - -### 5.1 Stage 1: Intent Collection - -For each source section: - -- `resolveTargets(section) -> UnresolvedTargetRef[]` -- `buildIntents(section, unresolvedTargetRef) -> UnresolvedPropagationIntent[]` - -Stage 1 is pure algebra. No path lookup or vault calls are allowed. - -### 5.2 Stage 2: Path Resolution (IO) - -Orchestrator resolves each unresolved target ref into a concrete `targetPath`. - -- Path resolution uses existing resolver policy (library lookup, sharding fallback, healing-aware path helpers). -- Only orchestrator/adapters perform this step. -- If resolver cannot produce a valid dictionary target path, the intent is rejected at validation. - -### 5.3 Stage 3: Validation - -Reject intent when: - -- `targetPath` is malformed -- Target is outside dictionary scope -- Self-targeting is forbidden by section policy and detected - -### 5.4 Stage 4: Dedupe - -Deduplicate intents by deterministic `intentKey`. - -### 5.5 Stage 5: Group by Target - -`Map`. - -### 5.6 Stage 6: Apply - -For each group: - -1. Read and parse target note. -- If target note does not exist but resolved path is valid dictionary scope, treat as empty content and create-on-write during emit. -2. For each intent: -- Resolve target entry via matching cascade. -- Create entry when strategy says create and none matches. -- Apply mutation with section merge policy. -3. Serialize canonical note body/meta. - -A single propagator may emit multiple intents for the same target note across different sections (for example morpheme propagation emitting both `Morphology:addBacklink` and `Tags:addTags`). - -### 5.7 Stage 7: Emit - -Emission is all-or-nothing: - -1. Build all target updates in memory first. -2. If any target fails during apply/serialize, return `Err` and emit zero propagation actions. -3. Append propagation actions to command action list only after all targets succeed. - ---- - -## 6. Type Contracts (v1) - -The following types are canonical architecture contracts. Concrete file placement may vary. - -Branded codes are compile-time safety guards. Runtime adapters still parse/normalize raw strings into these branded forms at architecture boundaries. - -```ts -export type StableEntryId = string; -export type IntentKey = string; -export type CreationKey = string; - -export type LangCode = string & { readonly __brand: "LangCode" }; -export type UnitKindCode = string & { readonly __brand: "UnitKindCode" }; -export type SurfaceKindCode = string & { readonly __brand: "SurfaceKindCode" }; -export type PosCode = string & { readonly __brand: "PosCode" }; - -export type SourceEntryKey = { - notePath: string; - stableId: StableEntryId; - lemma: string; - lang: LangCode; - unit: UnitKindCode; - surface: SurfaceKindCode; - pos: PosCode; -}; - -export type UnresolvedTargetRef = { - targetLemma: string; - lang: TLang; - unit: TUnit; - surface: TSurface; - pos: TPos; -}; - -export type TargetRef = { - targetPath: string; - lang: TLang; - unit: TUnit; - surface: TSurface; - pos: TPos; -}; - -export type RelationItemDto = { - relationKind: string; - targetLemma: string; - targetWikilink: string; -}; - -export type MorphologyBacklinkDto = { - relationType: "derived_from" | "compounded_from" | "used_in"; - value: string; -}; - -export type MorphologyEquationDto = { - lhsParts: ReadonlyArray; - rhs: string; -}; - -export type InflectionItemDto = { - form: string; - tags: ReadonlyArray; -}; - -export type RelationSectionDto = { kind: "Relation"; items: ReadonlyArray }; -export type MorphologySectionDto = { - kind: "Morphology"; - backlinks: ReadonlyArray; - equations: ReadonlyArray; -}; -export type InflectionSectionDto = { kind: "Inflection"; items: ReadonlyArray }; -export type TagsSectionDto = { kind: "Tags"; tags: ReadonlyArray }; - -export type SectionPayloadByKind = { - Relation: RelationSectionDto; - Morphology: MorphologySectionDto; - Inflection: InflectionSectionDto; - Tags: TagsSectionDto; -}; - -export type EntryMatchCriteria = - | { strategy: "byStableId"; stableId: StableEntryId } - | { - strategy: "byDeterministicKey"; - lang: LangCode; - unit: UnitKindCode; - surface: SurfaceKindCode; - pos: PosCode; - lemma: string; - } - | { strategy: "byHeader"; normalizedHeader: string } - | { strategy: "createNew"; creationKey: CreationKey; template: NewEntryTemplate }; - -export type SectionMutation = - | { - sectionKind: "Relation"; - op: "addRelation"; - relationKind: string; - targetLemma: string; - targetWikilink: string; - } - | { - sectionKind: "Morphology"; - op: "addBacklink"; - backlinkWikilink: string; - relationType: "derived_from" | "compounded_from" | "used_in"; - } - | { - sectionKind: "Morphology"; - op: "addEquation"; - lhsParts: ReadonlyArray; - rhs: string; - } - | { - sectionKind: "Inflection"; - op: "upsertInflection"; - tags: string[]; - headerTemplate: string; - } - | { - sectionKind: "Tags"; - op: "addTags"; - tags: string[]; - }; - -export type PropagationIntent = { - targetPath: string; - entryMatch: EntryMatchCriteria; - mutation: SectionMutation; - - sourceStableId: StableEntryId; - sourceSection: string; - sourceNotePath: string; - - creationKey?: CreationKey; - intentKey: IntentKey; -}; - -export type UnresolvedPropagationIntent = Omit & { - target: UnresolvedTargetRef; -}; -``` - ---- - -## 7. Propagator Contracts - -`Propagator` is explicit and typed. - -```ts -export type TargetRefOf = TgtEntry extends { - lang: infer L extends string; - unit: infer U extends string; - surface: infer S extends string; - pos: infer P extends string; -} - ? TargetRef - : never; - -export type UnresolvedTargetRefOf = TgtEntry extends { - lang: infer L extends string; - unit: infer U extends string; - surface: infer S extends string; - pos: infer P extends string; -} - ? UnresolvedTargetRef - : never; - -export interface Propagator< - SrcEntry, - TgtEntry, - K extends keyof SectionPayloadByKind, -> { - sectionKind: K; - resolveTargets(input: { - source: SrcEntry; - section: SectionPayloadByKind[K]; - }): ReadonlyArray>; - buildIntents(input: { - source: SrcEntry; - section: SectionPayloadByKind[K]; - target: UnresolvedTargetRefOf; - }): ReadonlyArray; -} -``` - -### 7.1 Why explicit `` - -1. Compile-time protection from nonsense source-target pairs. -2. Clear declaration of same-shape vs cross-shape propagation. -3. Easier registration and test coverage per linguistic slice. - ---- - -## 8. Matching Engine - -Matching is centralized in pipeline code, not inside propagators. - -### 8.1 Precedence - -1. `byStableId` -2. `byDeterministicKey` -3. `byHeader` (fallback only, log warning) - -### 8.2 Deterministic Key - -Key fields must be canonical and normalized: - -- `lang` -- `unit` -- `surface` -- `pos` -- `lemma` (from canonical metadata/DTO, not from rendered header text) - -Header text is never primary identity. - -`byDeterministicKey` is a fallback only when stable ID matching is unavailable. - -If lemma mutability is introduced in the future, deterministic-key matching must be revised with an explicit migration/rekey strategy. - -### 8.3 Create-New Flow with `creationKey` - -For create intents: - -1. Intent carries `creationKey` and creation template. -2. Apply stage parses note and checks if same creation intent already resolved. -3. If missing, allocator picks next note-local index for the required ID prefix. -4. Stable entry id is minted during apply, not during intent creation. - -This avoids pre-allocation races and still allows deterministic dedupe. - -Concurrency assumption in v1: - -- Apply + emit runs under a single Generate writer lock (single-flight coordinator) in one process. -- Without this serialization guarantee, concurrent runs can allocate colliding next indices for the same target note. -- If this assumption changes, allocator must add collision retry against latest note state before final emit. - ---- - -## 9. Intent Key - -`intentKey` is a deterministic hash over semantic payload, not rendered markdown. - -Required input tuple: - -- `sourceStableId` -- `sourceSection` -- `targetPath` -- `entryMatch` semantic identity fields -- `mutation` semantic fields -- `creationKey` (when present) - -Never hash formatted line strings. - ---- - -## 10. Propagator Registry and Selection - -Registration is explicit and deterministic. - -1. Registry key: `(lang, unit, surface, pos, sectionKind)`. -2. Multiple propagators for the same `sectionKind` are allowed only when source-shape keys differ. -3. If no propagator is registered for a source section, behavior is explicit no-op. -4. Execution order is deterministic by registry order to keep intent logs stable. - ---- - -## 11. Merge Policies - -Merge policies are explicit per section operation. - -| Section | Operation | Policy | Notes | -|---|---|---|---| -| Relation | `addRelation` | Set-union | Deduplicate by `(relationKind, targetLemma)` semantic key. | -| Morphology | `addBacklink` | Set-union | Deduplicate by backlink target + relation type. | -| Morphology | `addEquation` | Set-union with semantic dedupe | Deduplicate by normalized equation signature. | -| Inflection | `upsertInflection` | Upsert | Match target entry first, then merge tags + normalize header. | -| Tags | `addTags` | Set-union | Deduplicate normalized tag tokens. | - -Upsert conflict rules must be deterministic and order-independent. - ---- - -### 11.1 DTO Identity Keys for Set-Union - -DTO collections use `ReadonlyArray`, but merge semantics are set-union/upsert by semantic identity keys. - -Required dedupe keys: - -1. `RelationItemDto`: `(relationKind, targetLemma)`. -2. `MorphologyBacklinkDto`: `(relationType, value)`. -3. `MorphologyEquationDto`: normalized equation signature `(lhsParts[], rhs)`. -4. `TagsSectionDto.tags`: normalized tag token. -5. `InflectionItemDto`: item identity is `form`; nested `tags` dedupe by normalized tag token. - -Implementations must not use array position as identity. - ---- - -## 12. Parsing and Serialization - -### 12.1 Two-Tier Parsing (v1) - -1. Typed DTO parsing for sections used by propagation. -2. Raw passthrough storage for unsupported sections. -3. Typed `MorphologySectionDto` parsing must preserve both `backlinks` and `equations`. - -This allows migration without typing every section immediately. - -### 12.2 Canonical Serialization - -Serializer output is canonical and deterministic: - -1. Stable section order by configured weight. -2. Stable line/item ordering where applicable. -3. Normalized whitespace and separators. -4. Stable hashtag and wikilink formatting for typed sections. -5. Raw passthrough sections are byte-preserving when untouched by mutations. -6. Raw passthrough sections are not normalized in Phase 2 (including line endings, whitespace, and blank lines). - -Invariant: parse -> serialize -> parse yields equivalent DTO state. -Phase 2 characterization baseline for typed sections is DTO semantic equivalence plus deterministic canonical v2 serializer output, not byte-for-byte parity. - -Canonicalization is expected and accepted on first touch for typed sections. - ---- - -## 13. Validation and Error Handling - -### 13.1 Validation Rules - -Before apply: - -1. Reject malformed target paths. -2. Reject targets outside dictionary scope. -3. Reject disallowed self-targets. -4. Reject unknown section operation pairs. -5. Missing target file at a valid resolved dictionary path is not a validation error (it is created on write). - -### 13.2 Error Handling - -1. Per-target apply failures are logged with intent provenance. -2. Failure policy is fail-fast for command-level correctness in v1. -3. Action emission is all-or-nothing: on any failure, no propagation actions are appended. -4. Generate-level behavior in v1 is strict fail-fast: if propagation fails, the full Generate command returns `Err` and emits zero actions (source note actions and propagation actions are both dropped for that invocation). -5. Errors return `CommandError` through existing neverthrow pipeline. -6. On failure, logging must include: `targetPath`, `intentKey`, and normalized error reason. -7. User-facing error notice must include the failing `targetPath` to make repair actionable. - ---- - -## 14. Logging Rules - -Use `src/utils/logger` only. - -Recommended log points: - -1. Intent collection count by section. -2. Validation rejects with reason and intent key. -3. Group sizes by target note. -4. Apply summary per target: matched/created entries, changed/unchanged. -5. Apply failure detail: `targetPath`, `intentKey`, error kind/message. -6. Final emitted action count. - -Do not manually stringify objects. - ---- - -## 15. Suggested Module Layout (v1) - -This layout follows existing repository boundaries and keeps pure logic in linguistics/common modules. - -```text -src/commanders/textfresser/commands/generate/steps/ - propagate-v2.ts # orchestrator: collect/validate/group/apply/emit - propagate-v2-path-resolution.ts # unresolved target -> targetPath - propagate-v2-validation.ts # intent validation - propagate-v2-grouping.ts # dedupe + grouping helpers - propagate-v2-matcher.ts # centralized entry matching engine - propagate-v2-apply.ts # apply intents to parsed note DTO - propagator-registry.ts # source-shape + section dispatch - -src/commanders/textfresser/domain/ - propagation/ - types.ts # target refs, intents, EntryMatchCriteria, mutations - intent-key.ts # deterministic intent key helper - creation-key.ts # creation key helpers - merge-policy.ts # pure section merge policy table + dispatch - -src/linguistics/ - ... per-slice pure propagators ... -``` - -No Obsidian/VAM imports from pure propagator modules. - ---- - -## 16. Migration Plan - -Do not attempt one-shot migration. - -### 16.1 Phase 0 - Capability Audit (VAM + Ports) - -Do this before implementing propagation v2 logic. - -Status: complete (implemented on February 18, 2026). - -1. Define a minimal propagation IO contract for v2 orchestrator needs (`PropagationVaultPort` + Librarian lookup port). -2. Map each required method to existing VAM/Librarian APIs. -3. Mark each capability as `covered` or `gap`. -4. Implement missing VAM/Librarian methods only for confirmed gaps. -5. Lock Phase 0 output as a short capability matrix in this section or an appendix. - -Phase 0 output is tracked in: - -- [`propagation-v2-phase0-capability-audit.md`](./propagation-v2-phase0-capability-audit.md) - -Recommended contract surface: - -1. Read existing note content by split path (or empty when missing). -2. Bulk read resolved target note paths with explicit per-path `Found`/`Missing`/`Error` outcomes (`readManyMdFiles`). -3. Resolve candidate target note paths by basename/policy using both VAM lookups and Librarian leaf-core-name lookup. -4. Emit upsert/process actions for one target note. -5. Expose path-existence checks needed by resolver policy. - -### 16.2 Phase 1 - Contracts and Guardrails - -1. Add propagation v2 contracts/types. -2. Add lint or boundary checks blocking IO imports in propagator modules. -3. Add unit tests for intent key determinism and merge helpers. - -### 16.3 Phase 2 - Adapter Layer - -1. Build typed parser/serializer adapters needed by v2 apply. -2. Add characterization tests using this baseline: typed sections assert DTO semantic equivalence + deterministic canonical v2 serializer output; raw passthrough asserts byte-preserving output. -3. Keep Wikilink adapter scope minimal for v2 migration: support basic `WikilinkDto { target, displayText? }` only where required by parser/serializer boundaries. -4. Exotic wikilinks (anchors, embeds, alias suffix decorations) are passthrough in Phase 2; when target extraction is unparseable for intent-building, log warning and skip that specific intent (not a hard run error). -5. Explicitly defer suffix-decoration parsing/typing to Post-v1 Book of Work (TBD), unless a concrete Phase 2 blocker is discovered. -6. Canonicalize typed Morphology backlink output (normalize surrounding whitespace in wikilink values) so semantically equal backlinks serialize identically. -7. Defer semantic inflection diffing work by default; keep current Phase 1 diffing unless concrete readability/performance issues are observed. - -### 16.4 Phase 3 - Orchestrator Skeleton - -1. Introduce `propagate-v2` step behind one global feature flag: `propagationV2Enabled`. -2. Keep v1/v2 routing inside a propagation wrapper/facade (Generate entrypoint calls one propagation interface). -3. Keep v2 strict fail-fast semantics (`Err` short-circuits Generate and no actions are emitted/dispatched). -4. Wire validation, dedupe, grouping, apply, emit flow incrementally behind this flag. - -### 16.5 Phase 4 - First Vertical Slice - -Prerequisite: Phase 4 assumes Phase 1 contracts and Phase 2 adapters are already landed. - -1. Switch routing to per-slice in the propagation facade. -2. Keep `propagationV2Enabled` as a global kill-switch: when `false`, always route to `v1`. -3. Seed routing map with one migrated slice: `de/lexem/noun -> v2`; all non-migrated slices route to `v1`. - - Routing key is source-slice only: `(lang, unit, pos)`. - - `surfaceKind` is intentionally excluded from Phase 4 routing decisions. -4. Lock first-slice operation scope: - - Required: `Relation.addRelation`, `Inflection.upsertInflection`. - - Required when current noun `v1` behavior emits them: `Morphology.addBacklink`, `Morphology.addEquation`. - - Include `Tags.addTags` only if current noun `v1` propagation emits tag side-effects; do not add new tag propagation behavior in v2 slice 1. - - For noun slice parity in Phase 4, `Inflection.upsertInflection` materializes as tags updates on inflected-target entries (current `v1` behavior), not as propagated `Inflection` sections on targets. - - Keep `decorateAttestationSeparability` out of v2 slice scope (source-note formatting concern, not target-note propagation). -5. Verify parity and idempotency against current flow using the Phase 4 sign-off gate. - -### 16.6 Phase 5 - Incremental Rollout - -1. Migrate remaining section propagators using a hybrid `risk x usage` order (objective rubric below). -2. Keep tests green per slice. -3. Require full sign-off gate per slice before moving to the next slice (batching allowed only under strict criteria below). -4. Before migrating any non-verb slice, explicitly audit `decorateAttestationSeparability` as no-op/impossible for that slice and record the audit in the PR checklist. -5. BoW item 14 (shared post-propagation decoration step) is landed; verb slice migration is completed in routing/tests. -6. At 100% coverage, transition in two PRs: - - PR1: default `propagationV2Enabled=true`, keep kill-switch and v1 code as rollback path. - - PR2: remove v1 + kill-switch after soak exit criteria are met, or after an explicit soak waiver decision. - -Phase 5 runtime source of truth: - -1. `propagateGeneratedSections` in `src/commanders/textfresser/commands/generate/steps/propagate-generated-sections.ts` (v2-only wrapper route). -2. As of February 18, 2026: all `de/lexem/*` and `de/phrasem/*` slices are migrated to `v2`, and runtime routing no longer includes `v1`. -3. BoW item 14 status: landed on February 18, 2026; `decorateAttestationSeparability` runs as a shared post-propagation step after `v2` core propagation. -4. Non-verb audit outcome: decoration remains no-op for migrated non-verb slices; parity is locked by `tests/unit/textfresser/steps/propagate-v2-phase4.test.ts`. -5. Soak tracking/waiver record: GitHub issue `#20` (waiver approved for pre-release/dev-only, no external users). -6. PR2 status target: remove v1 path + kill-switch settings/state/wiring and keep v2-only runtime. - -#### 16.6.1 Risk x Usage Scoring Rubric - -Use this rubric to produce deterministic slice ordering. - -Operational measurement scope: - -1. Do not add telemetry/analytics infrastructure for migration. -2. `usageScore` is a documented manual ranking input provided by user/domain knowledge in each rollout PR. -3. Manual `usageScore` must be recorded in the PR description as a `1..5` value with a one-line rationale. - -Usage score (`1..5`): - -1. Assign relative usage rank manually (`5` highest usage, `1` lowest usage). -2. Keep ordering deterministic by documenting each slice's assigned score in the rollout PR. - -Phase 5 required gate per slice PR: - -1. Run and pass parity/idempotency/fail-fast/one-write-target tests for the slice. -2. Run `bun run typecheck:changed`. -3. Run full `bun test` for non-regression. -4. If full `bun test` baseline is red, include baseline vs PR failure counts in PR notes. - -Risk score (`1..5`): - -1. Start at `1`. -2. Add `+1` if slice writes to 3 or more distinct target mutation kinds. -3. Add `+1` if slice creates new target entries (not only updates existing entries). -4. Add `+1` if slice depends on morphology equation/backlink marker parsing. -5. Add `+1` if slice relies on path-healing actions (`RenameMdFile`) as normal flow. -6. Cap at `5`. - -Rollout priority score: - -1. `rolloutPriority = (2 * usageScore) - riskScore` -2. Sort by `rolloutPriority` descending. -3. Tie-breakers: lower `riskScore` first, then higher `usageScore`, then lexical slice key. -4. Historical override (pre-verb-migration): verb slices stayed last unless explicitly reprioritized. - -#### 16.6.2 Batching Eligibility (Exception Path) - -Default is one-slice-at-a-time gating. Batching multiple slices into one gate is allowed only if all conditions hold: - -1. Same v1 mutation-kind surface (identical set of `{Relation, MorphologyBacklink, MorphologyEquation, Inflection, Tags}` actually emitted). -2. Same target surface policy (same lemma/inflected target resolution behavior). -3. Same entry matching/creation behavior (no slice-specific matching override). -4. Same no-op audit result for `decorateAttestationSeparability`. -5. Near-isomorphic fixture topology (same fixture structure with only lexical data substitutions). -6. Explicit maintainer sign-off before batching. - -If any condition fails, do not batch; run independent per-slice sign-off. - -#### 16.6.3 PR2 Soak Exit Criteria - -Do not remove v1/kill-switch until all criteria pass (unless a waiver is explicitly recorded per 16.6.4): - -1. Minimum soak window: 14 consecutive days with PR1 deployed. -2. Minimum traffic: at least 100 successful `Generate` runs across at least 3 migrated slices during soak. -3. Stability: zero kill-switch rollback activations during soak. -4. Regression safety: zero open P0/P1 propagation regressions at cut time. -5. CI: parity/idempotency/fail-fast suites for migrated slices are green on the PR2 head commit. - -#### 16.6.4 Soak Waiver Decision (Issue #20) - -1. Decision date: February 18, 2026. -2. Decision: soak requirement waived for this rollout. -3. Reason: pre-release/dev-only environment with no external users. -4. Record: GitHub issue `#20` documents the waiver and PR2 execution plan. - -### 16.7 Coexistence Rule During Migration - -Do not mix v1 and v2 propagation writes within a single Generate invocation. - -1. Historical (Phase 3): routing was global by `propagationV2Enabled` (single engine per invocation). -2. Historical (Phase 4): routing switched to per-slice in the facade. -3. Current (post-PR2): runtime is v2-only; v1 routing path and kill-switch are removed. -4. Mixed v1/v2 propagation engines are impossible in current runtime. - ---- - -## 17. Test Strategy - -### 17.1 Unit Tests (pure) - -1. `resolveTargets` and `buildIntents` per propagator. -2. Intent key determinism and dedupe. -3. Merge policy behavior for each operation. -4. Matching precedence and fallback behavior. - -### 17.2 Integration Tests (step-level) - -1. End-to-end propagation over in-memory note strings. -2. One write action per target note invariant. -3. Validation rejections for malformed/out-of-scope targets. - -### 17.3 Regression/Property Tests - -1. Idempotency: second run produces no changes. -2. Canonical serialization stability. - -### 17.4 Phase 4 Sign-Off Gate - -Primary gate (must pass): - -1. Semantic DTO parity between `v1` and `v2` on curated noun fixtures. -2. Idempotency: second `v2` run over same inputs emits no changed-target writes (zero effective target writes). -3. Invariants: strict fail-fast/all-or-nothing emission, and one write action per target note. - -Secondary gate (should pass): - -1. Order-insensitive action-target parity for emitted mutation intents (`{ targetPath, mutationKind }` set parity vs `v1`). - -Out of scope for parity gate: - -1. Byte-for-byte markdown parity. -2. Full `VaultAction[]` structural equality. - ---- - -## 18. Performance Expectations - -1. Grouping by target note bounds writes to O(number of touched targets). -2. One parse/serialize cycle per touched target note. -3. Intent collection remains linear in number of source sections and extracted references. - ---- - -## 19. Post-v1 Book of Work - -Post-migration deferred work is tracked in: - -- [`post-migration-book-of-work.md`](./post-migration-book-of-work.md) - ---- - -## 20. Implementation Checklist - -1. Add `propagation-v2` contracts. -2. Implement `intentKey` helper and tests. -3. Implement centralized matcher and tests. -4. Implement merge policy dispatcher and tests. -5. Implement orchestrator behind flag (historical, completed). -6. Implement per-slice facade routing with global kill-switch precedence (historical, completed). -7. Migrate noun slice operations in scoped order and add parity + idempotency tests (historical, completed). -8. Roll out remaining slices (historical, completed). -9. Remove legacy step and kill-switch runtime wiring (historical, completed). diff --git a/src/documentaion/propagation-migration/propagation-v2-phase0-capability-audit.md b/src/documentaion/propagation-migration/propagation-v2-phase0-capability-audit.md deleted file mode 100644 index 450f01f38..000000000 --- a/src/documentaion/propagation-migration/propagation-v2-phase0-capability-audit.md +++ /dev/null @@ -1,103 +0,0 @@ -# Propagation V2 Phase 0 Capability Audit - -Parent architecture doc: - -- [`propagation-v2-architecture.md`](./propagation-v2-architecture.md) - -Scope: - -- Confirm whether existing Vault Action Manager (VAM) + Librarian lookup APIs can satisfy the minimum propagation IO surface for v2. -- Mark each capability as `covered` or `gap`. -- Propose only minimal adapter glue where needed. - -Status: complete (implemented on February 18, 2026). - -## Proposed Minimal Propagation IO Ports - -```ts -type PropagationVaultPort = { - readNoteOrEmpty(splitPath: SplitPathToMdFile): Promise>; - readManyMdFiles(paths: ReadonlyArray): Promise< - ReadonlyArray< - | { kind: "Found"; splitPath: SplitPathToMdFile; content: string } - | { kind: "Missing"; splitPath: SplitPathToMdFile } - | { kind: "Error"; splitPath: SplitPathToMdFile; reason: ReadContentError } - > - >; - findCandidateTargets(params: { - basename: string; - folder?: SplitPathToFolder; - }): ReadonlyArray; - exists(path: AnySplitPath): boolean; - buildTargetWriteActions(params: { - splitPath: SplitPathToMdFile; - transform: (content: string) => string; - }): readonly VaultAction[]; -}; - -type PropagationLibraryLookupPort = { - findByLeafCoreName(coreName: string): ReadonlyArray; -}; -``` - -Notes: - -- `readNoteOrEmpty` is an adapter-level behavior ("missing file means empty"), not a raw VAM primitive. -- `readNoteOrEmpty` returns `Err` only for read/IO failures. Missing file maps to `ok("")`. Parse/validation/apply failures are handled in later stages, not in this port method. -- `readManyMdFiles` is the preferred Stage 6 hydrate API for already-resolved targets (bulk parallel reads with explicit missing/error outcomes). -- `readManyMdFiles` must classify "file vanished between `exists` and `readContent`" as `Missing`, not `Error`, to preserve create-on-write behavior under concurrent vault changes. -- VAM read errors are typed (`ReadContentError`); propagation adapter classifies missing-file reads via discriminant (`FileNotFound`), not message matching. -- `buildTargetWriteActions` returns actions to append to Generate's action list; v2 should keep dispatch outside the pure/apply phase. -- `buildTargetWriteActions` intentionally narrows transform to sync `(content) => string` for v2. Even though VAM supports async transforms, v2 apply/serialize is designed as pure synchronous DTO algebra. -- `findByLeafCoreName` is required for Library-hosted closed sets (for example prefixes/particles/prepositions) where path resolution must consult Librarian's leaf-core-name index. - -## Capability Matrix - -| Capability | Port method | Existing API(s) | Status | Notes | -|---|---|---|---|---| -| Read existing note content by split path (or empty when missing) | `readNoteOrEmpty` | `vam.exists(splitPath)`, `vam.readContent(splitPath)` | `covered` | `readContent` returns typed `ReadContentError`; adapter maps `!exists` and `FileNotFound` to `ok("")`, otherwise returns `Err` with normalized reason text. | -| Bulk hydrate resolved target notes | `readManyMdFiles` | `vam.exists(splitPath)`, `vam.readContent(splitPath)` (parallelized in adapter) | `covered` | Implement in adapter with `Promise.all` over deduped target paths; return `Found/Missing/Error` per path and classify typed `FileNotFound` read failures as `Missing` to preserve deterministic create-on-write behavior. | -| Resolve candidate target note paths by basename/policy | `findCandidateTargets` + `findByLeafCoreName` | `vam.findByBasename(basename, opts?)`, `librarian.findMatchingLeavesByCoreName(coreName)` (wired into `textfresserState.lookupInLibrary`) | `covered` | Current resolver policy is composition-level (`findByBasename` + Librarian core-name lookup + sharded fallback + healing in `target-path-resolver.ts`). Closed-set Library entries rely on the Librarian lookup branch. | -| Emit upsert/process actions for one target note | `buildTargetWriteActions` | `VaultActionKind.UpsertMdFile`, `VaultActionKind.ProcessMdFile`, existing pattern in `buildPropagationActionPair` | `covered` | Existing action pair pattern is sufficient. Port contract uses sync transform only (`(content) => string`) for v2 simplicity. VAM ensure-requirements can synthesize `UpsertMdFile(content:null)` when only `ProcessMdFile` is present. | -| Expose path-existence checks needed by resolver policy | `exists` | `vam.exists(splitPath)` | `covered` | Direct match for resolver and validation checks. | - -Additional available capability (not required by the minimal v2 port): - -- `vam.listAllFilesWithMdReaders(splitPathToFolder)` is available for recursive folder scans with per-file readers. -- This is useful for future resolver/index strategies, but not required for v2 Phase 0 because current propagation resolution is basename/policy-driven. - -## Confirmed Gaps - -None at VAM/Librarian port level for v2 propagation Phase 0. - -## Required Phase 0 Deliverables (Adapter-Level Only) - -1. Add a thin `PropagationVaultPort` adapter in `propagate-v2` orchestration code that: - - Implements `readNoteOrEmpty` via `exists` + `readContent`. - - Implements `readManyMdFiles(paths)` via parallelized reads with per-path `Found/Missing/Error` outcomes. - - Treats typed `FileNotFound` read failures as `Missing` (race-safe classification), not `Error`. - - Wraps `findByBasename` (with optional folder scoping). - - Centralizes write-action construction for one target note. -2. Add a thin `PropagationLibraryLookupPort` adapter that wraps Librarian leaf-core-name lookup (`findMatchingLeavesByCoreName`) as `findByLeafCoreName`. -3. Keep policy decisions (dictionary-scope validation, sharded fallback, healing-aware path choice) outside VAM and inside propagation resolver modules. -4. Do not add new VAM/Librarian methods unless a later phase reveals a concrete, test-proven gap. - -## Evidence Snapshot (Current Code) - -- VAM interface methods: `readContent`, `exists`, `findByBasename`, `dispatch` - - `src/managers/obsidian/vault-action-manager/index.ts` -- Additional VAM inventory API: `listAllFilesWithMdReaders` - - `src/managers/obsidian/vault-action-manager/index.ts` -- Librarian core-name lookup API: `findMatchingLeavesByCoreName` - - `src/commanders/librarian/librarian.ts` -- Wiring from Librarian lookup into Textfresser state lookup (`lookupInLibrary`) - - `src/main.ts` - - `src/commanders/textfresser/textfresser.ts` -- Missing read behavior (`readContent` errors when file absent) - - `src/managers/obsidian/vault-action-manager/impl/vault-reader.ts` -- Existing propagation path policy + healing + action pair helper - - `src/commanders/textfresser/common/target-path-resolver.ts` -- Current generate propagation steps already consume these capabilities - - `src/commanders/textfresser/commands/generate/steps/propagate-relations.ts` - - `src/commanders/textfresser/commands/generate/steps/propagate-morphemes.ts` - - `src/commanders/textfresser/commands/generate/steps/propagate-inflections.ts` diff --git a/src/documentaion/specs/textfresser-wikilink-resolution-spec.md b/src/documentaion/specs/textfresser-wikilink-resolution-spec.md new file mode 100644 index 000000000..b19c3f43f --- /dev/null +++ b/src/documentaion/specs/textfresser-wikilink-resolution-spec.md @@ -0,0 +1,267 @@ +# Textfresser Wikilink Resolution Spec (Draft) + +Status: Draft +Owner: Textfresser +Last updated: 2026-02-20 + +## Compatibility Policy (Dev Mode, 2026-02-20) + +1. Textfresser is treated as green-field. Breaking changes are allowed; no backward-compatibility guarantees for Textfresser note formats, schemas, or intermediate contracts. +2. Librarian and VAM are stability-critical infrastructure. Changes there require conservative rollout, migration planning when persisted contracts change, and explicit regression coverage. + +## Why this spec exists + +Wikilink target resolution has grown across multiple helpers and call-sites. We need one explicit contract for how Textfresser resolves and rewrites link targets. + +This spec moves the in-depth discussion out of BOW and defines the behavior we want before implementation/refactor. + +## Key clarification + +Slash-based phrase targets are not a valid routing signal by themselves. + +Closed-set lexical classes are intentionally stored in `Library/...` and should be handled via explicit routing policy and known roots, not by generic "target has slashes" heuristics. + +## Goals + +1. Resolve wikilink targets deterministically and predictably. +2. Preserve meaningful user data (aliases and anchors). +3. Avoid accidental target corruption during normalization. +4. Keep Librarian and Textfresser responsibilities clear. + +## Non-goals + +1. No generic flattening rule based only on "many path segments". +2. No hidden routing side effects from arbitrary slash-separated strings. +3. No BOW-level design discussion; BOW should only point to this spec. + +## Resolution contract + +### Input forms we must support + +1. Plain target: `[[Fahren]]` +2. Target with alias: `[[Fahren|fahrt]]` +3. Vault-root path target: `[[Worter/de/.../Fahren]]`, `[[Library/de/.../auf-prefix-de]]` +4. Relative target: `[[./Fahren]]`, `[[../verbs/Fahren]]` +5. Target with anchor: `[[Fahren#^b123]]`, `[[Worter/de/.../Fahren#Kontexte]]` +6. Target with `.md`: `[[Library/de/.../Fahren.md]]` + +### Output invariants + +1. Preserve `#...` anchors exactly. +2. Preserve alias text exactly. +3. Normalize only the target path part. +4. Return a stable basename-style wikilink target when the input is an explicit known vault path. +5. Do not flatten unknown slash-structured text. + +### Known-root flattening rule + +Flatten to basename only when target explicitly matches one of these known forms: + +1. `Worter/...` +2. `Library/...` +3. `/Worter/...` +4. `/Library/...` +5. Relative forms `./...` or `../...` (resolved first, then normalized) +6. Explicit `.md` suffix path after known-root or relative resolution + +If none of these forms match, keep the target path as-is. + +## Proposed resolution pipeline + +1. Parse wikilink into `{ target, alias, anchor }`. +2. Split anchor from target once and carry it through unchanged. +3. Classify target kind: + 1. Explicit known-root path. + 2. Relative path. + 3. Plain basename or unknown structured target. +4. Resolve candidate destination: + 1. Try Obsidian native resolution from source note context. + 2. If unresolved, use Librarian alias resolution where applicable. + 3. If command policy requires deterministic destination, compute Textfresser canonical target. +5. Normalize render target: + 1. If resolved as known-root/relative path, render basename. + 2. Otherwise keep original target text. +6. Reattach anchor and alias. + +## Examples + +1. `[[Worter/de/lexem/lemma/f/fah/fahre/Fahren]]` -> `[[Fahren]]` +2. `[[Library/de/prefix/auf-prefix-de|>auf]]` -> `[[auf-prefix-de|>auf]]` +3. `[[Worter/de/x/Fahren#^abc|fahrt]]` -> `[[Fahren#^abc|fahrt]]` +4. `[[domain/schema/field]]` -> `[[domain/schema/field]]` (unchanged) +5. `[[../Worter/de/x/Fahren.md#Kontexte]]` -> `[[Fahren#Kontexte]]` (after relative resolution) + +## Responsibilities split + +1. Textfresser decides policy target family (`Library` vs `Worter`) for its commands. +2. Librarian resolves aliases/tree semantics and keeps naming invariants. +3. `wikilinkHelper` performs syntax-safe normalization only; it must not encode business routing policy. + +## Test plan requirements + +1. Unit tests for `normalizeLinkTarget` and `normalizeWikilinkTargetsInText`. +2. Cases with anchors, aliases, `.md`, known roots, relative paths, and unknown slash targets. +3. Regression tests for previously broken generated notes (full path leakage in headers/morphemes/inflections). +4. Integration tests for Lemma/Generate rewrite flow to confirm consistent target rendering. + +## Open questions + +1. Should relative paths always be resolved to absolute split paths before normalization, or only when an active file context exists? +2. When Obsidian and Librarian resolution disagree, which resolver is authoritative per command phase? +3. Do we ever need to display full paths intentionally in advanced/debug modes? + +## Alignment snapshot (2026-02-20) + +This section captures agreed baseline decisions from brainstorming. It is not the final design of every edge case. + +### Agreed baseline + +1. Split syntax and policy responsibilities: + - `wikilinkHelper` should be syntax-focused (parse/find/format-safe transforms). + - Textfresser/Librarian routing policy should live outside `wikilinkHelper`. +2. Anchor must be a first-class field in link DTOs to avoid ad-hoc split/re-attach logic. +3. We must model two different operations explicitly: + - Authoring-time normalization (sanitize LLM/text before persisting markdown). + - Read-time resolution (map a clicked/entered target to an actual destination). +4. Two-tier DTOs are preferred: + - Lightweight normalized DTO for formatter/guardrail pipelines. + - Full resolver DTO for navigation/routing when split-path/provenance is needed. +5. Migration should keep a compatibility wrapper to avoid risky big-bang refactors across all call-sites. +6. `formatLinkTarget(splitPath)` is generation-side rendering (split-path -> link target string), not raw-text normalization. It is related policy, but a different direction in the pipeline. +7. Comparison normalization (current `normalizeTarget` behavior: trim + case-fold) is domain policy, not pure wikilink syntax. + +### Locked policy: Closed-set surface hubs (2026-02-20) + +This policy is locked before Decision #10 (generation render default policy), because it defines the target structure that rendering must respect. + +1. Canonical closed-set Lexem content home: `Library` (unchanged). +2. Ambiguity handling: create/use a `Worter` surface hub when one surface maps to 2+ closed-set `Library` targets. +3. Hub creation trigger: lazy, on the second closed-set `Library` entry for the same surface. +4. Hub content: minimal disambiguation links to canonical `Library` targets; do not duplicate dictionary content. +5. LLM-confirmed attestations (Lemma/Generate): link directly to the specific `Library` note selected by confirmed POS. +6. Manual `[[surface]]`: should resolve to the `Worter` hub when it exists. +7. Completion behavior on ambiguity: stop silently picking one leaf target; do not auto-rewrite ambiguous unresolved links to a single winner. +8. Re-encounter policy: POS-confirmed re-encounters stay direct-to-`Library`; hub is for manual/unconfirmed surface entry. + +### Implementation v1 (concrete) + +This section turns the locked policy into implementable behavior for the first rollout. + +#### Hub storage model + +1. Hubs are not normal dict entries. They are dedicated surface-index notes. +2. Store hubs in a dedicated flat folder (not sharded dict-entry folders): + - `Worter/de/closed-set-hub/{surface}.md` +3. Hub notes must include a cheap marker so pipelines can detect and skip dict-entry logic: + - frontmatter marker: `textfresser.kind: closed-set-surface-hub` + +Example: + +```md +--- +textfresser: + kind: closed-set-surface-hub + surface: die +--- +# die + +Possible closed-set targets: +- [[die-pronomen-de|die (Pronoun)]] +- [[die-artikel-de|die (Article)]] +``` + +#### Hub lifecycle + +1. Create hub lazily when the second closed-set `Library` target for the same surface appears. +2. Keep hub updated when closed-set targets are added/renamed/deleted. +3. Keep hub even if it drops back to one target (do not break existing manual links). +4. Trash hub only when it has zero closed-set targets. + +#### Link routing behavior + +1. POS-confirmed Lemma/Generate attestations continue linking directly to specific `Library` note: + - example: `[[die-pronomen-de|die]]` +2. Manual/unconfirmed `[[surface]]` should land on the hub when it exists: + - example: `[[Die]]` resolves to `Worter/de/closed-set-hub/die.md` + +#### Wikilink completion guard (critical fix) + +In wikilink completion behavior, when unresolved link content has multiple Library leaf matches: + +1. Do **not** auto-pick a single leaf. +2. Return passthrough (keep user-entered target unchanged). +3. This rule applies regardless of whether a hub already exists. + +This replaces the current nearest-leaf winner behavior that can silently rewrite to a wrong closed-set target. + +#### Known gap (accepted in v1) + +If a link was auto-rewritten earlier when only one closed-set match existed, and later additional matches appear, old links are not retroactively rewritten. This is accepted for v1. + +#### Migration/backfill + +Add a maintenance command: + +1. Scan closed-set `Library` entries. +2. Build/update `Worter/de/closed-set-hub/{surface}.md` notes. +3. Preserve existing hub links where possible; ensure all currently valid closed-set targets are present. + +### Decided (2026-02-20) + +1. Comparison normalization location: + - `trim + case-fold` comparison normalization is domain policy (not syntax helper policy). + - Planned module location: + - `src/commanders/textfresser/common/target-comparison.ts` + - Migration intent: Textfresser call-sites should stop using `wikilinkHelper.normalizeTarget`. +2. Generation render default policy: + - `formatLinkTarget(splitPath)` will default to `basename`. + - `computeFinalTarget()` will auto-fallback to full-path rendering for `Library` targets when basename lookup is ambiguous (`findByBasename(...)` returns multiple unique paths). + - Explicit full-path rendering remains opt-in via `libraryTargetStyle: "full-path"`. + +### Implementation status (worktree snapshot, 2026-02-20) + +1. Implemented in current worktree: + - Closed-set hub policy module and lifecycle actions: + - `src/commanders/textfresser/common/closed-set-surface-hub.ts` + - Generate pipeline hook for hub maintenance: + - `src/commanders/textfresser/commands/generate/steps/maintain-closed-set-surface-hub.ts` + - Backfill maintenance command: + - `rebuild-closed-set-surface-hubs` in `src/main.ts` + - Includes lookup-availability guard: command aborts when Librarian lookup wiring is unavailable. + - Wikilink completion ambiguity guard (2+ matches -> passthrough): + - `src/managers/obsidian/behavior-manager/wikilink-complition-behavior.ts` + - Domain comparison normalization module + call-site migration: + - `src/commanders/textfresser/common/target-comparison.ts` + - Generation render policy update: + - `formatLinkTarget()` basename default + Library ambiguity fallback in `src/commanders/textfresser/common/lemma-link-routing.ts` + - Generate-time safety in degraded init states: + - `maintainClosedSetSurfaceHub` is now a no-op when `TextfresserState.isLibraryLookupAvailable` is false. +2. Still open (design/rollout not finished): + - Full API boundary cleanup between `wikilinkHelper` syntax responsibilities and policy modules. + - Canonical lightweight/full DTO contracts with explicit anchor field across all call-sites. + - Unified resolver precedence by command phase/intent (Obsidian vs Librarian vs policy-computed). + - Bulk rewrite parse/classify/selective-normalize/reassemble contract finalization. + - Deterministic manual `[[surface]]` preference for hub target in all contexts (not only ambiguity passthrough). +3. Notes: + - This status block is a snapshot of current branch/worktree state, not a release/merge guarantee. + +### Decision backlog (to resolve separately) + +1. Scope of first implementation slice: + - Authoring-time normalization only, or full resolver now. +2. Final API boundary: + - Exactly what stays in `wikilinkHelper` vs new policy module(s). +3. Lightweight DTO contract: + - Field names/types for normalized `{ target, anchor, alias, ... }`. +4. Full resolver DTO contract: + - `splitPath`, `source`, diagnostics, and required/optional fields. +5. Target classification model: + - Data-carrying discriminated `TargetKind` variants and required payload per variant. +6. Resolver precedence by intent/phase: + - Obsidian vs Librarian vs policy-computed ordering for Lemma/Generate/Propagation/Click. +7. Bulk text rewrite contract: + - Parse -> classify -> selective normalize -> reassemble behavior and opt-outs. +8. Naming cleanup: + - Replace ambiguous names (`normalizeTarget`, `normalizeLinkTarget`) with intent-specific names. +9. Migration/deprecation timeline: + - Call-site groups, test gates, and compatibility removal criteria. diff --git a/src/documentaion/textfresser-architecture.md b/src/documentaion/textfresser-architecture.md index d2771a4b3..9c3829362 100644 --- a/src/documentaion/textfresser-architecture.md +++ b/src/documentaion/textfresser-architecture.md @@ -1,6 +1,10 @@ # Textfresser Vocabulary System — Architecture > **Scope**: This document covers the vocabulary/dictionary half of the plugin (the "Textfresser" commander). For the tree/healing/codex half, see the Librarian docs. For E2E testing, see `e2e-architecture.md`. +> +> **Compatibility Policy (Dev Mode, 2026-02-20)**: +> - Textfresser is treated as green-field. Breaking changes are allowed; no backward-compatibility guarantees for Textfresser note formats, schemas, or intermediate contracts. +> - Librarian and VAM are stability-critical infrastructure. Changes there require conservative rollout, migration planning when persisted contracts change, and explicit regression coverage. --- @@ -42,7 +46,7 @@ User clicks the wikilink → navigates to the dictionary note User gets a tailor-made dictionary that grows with their reading ``` -> **V2 scope**: German target, 6 generated sections (Header, Morphem, Relation, Inflection, Translation, Attestation), re-encounter detection (append attestation vs new entry), cross-reference propagation for relations, noun inflection propagation in inflected-form notes, user-facing notices. +> **Core scope**: German target, 6 generated sections (Header, Morphem, Relation, Inflection, Translation, Attestation), re-encounter detection (append attestation vs new entry), cross-reference propagation for relations, noun inflection propagation in inflected-form notes, user-facing notices. > **V3 scope**: Polysemy disambiguation — new Disambiguate prompt in Lemma command, enriched note metadata per entry ID for fast lookup without note parsing, VAM API expansion (`getSplitPathsToExistingFilesWithBasename`), Lemma-side sense matching before Generate. @@ -50,7 +54,7 @@ User gets a tailor-made dictionary that grows with their reading > **V7 scope**: Polysemy quality fixes — Header emoji prompt changed to reflect the specific sense in context (not "primary/most common meaning"). Disambiguate gloss rule added: must be context-independent (e.g., "Schließvorrichtung" not "Fahrradschloss"). New polysemous examples in Header and Disambiguate prompts (Schloss castle vs lock). -> **V10 scope**: Emoji-as-semantic-differentiator — **Definition section dropped entirely** (along with `PromptKind.Semantics`). Homonym disambiguation now uses **emoji arrays** instead of text glosses. Header prompt returns `emojiDescription: string[]` (1-3 emojis capturing the sense, e.g., `["🏰"]` vs `["🔒","🔑"]` for *Schloss*). Disambiguate prompt receives `emojiDescription` + `unitKind` + `pos` + `genus` (+ optional `phrasemeKind`) per sense (richer context than the old text gloss). `meta.semantics` replaced by `meta.emojiDescription: string[]`. `LemmaResult.precomputedSemantics` replaced by `precomputedEmojiDescription: string[]`. Old entries without `emojiDescription` hit V2 legacy path (first-match fallback). CORE_SECTIONS reduced to `[Header, Translation, Attestation, FreeForm]`. +> **V10 scope**: Emoji-as-semantic-differentiator — **Definition section dropped entirely** (along with `PromptKind.Semantics`). Homonym disambiguation now uses **emoji arrays** instead of text glosses. Header prompt returns `emojiDescription: string[]` (1-3 emojis capturing the sense, e.g., `["🏰"]` vs `["🔒","🔑"]` for *Schloss*). Disambiguate prompt receives `emojiDescription` + `unitKind` + `pos` + `genus` (+ optional `phrasemeKind`) per sense (richer context than the old text gloss). `meta.semantics` replaced by canonical `meta.entity.emojiDescription`. `LemmaResult.precomputedSemantics` replaced by `precomputedEmojiDescription: string[]`. Entries without `emojiDescription` are treated as new-sense candidates. CORE_SECTIONS reduced to `[Header, Translation, Attestation, FreeForm]`. > **V11 scope**: Kill Header Prompt — `PromptKind.Header` eliminated. `emojiDescription` (1-3 emojis) and `ipa` (IPA pronunciation) moved into Lemma prompt output. Header line built from LemmaResult fields (`formatHeaderLine()` takes `{ emojiDescription, ipa }` instead of `AgentOutput<"Header">`). Header emoji display uses the full `emojiDescription` sequence in order. `genus` and article (der/die/das) dropped from header line. `buildLinguisticUnit()` removed — `meta.linguisticUnit` no longer populated during Generate. One fewer API call per new entry. @@ -58,13 +62,13 @@ User gets a tailor-made dictionary that grows with their reading > **V13 scope**: Phraseme-kind threading + linguisticUnit metadata restore — Lemma output now includes `phrasemeKind` for `linguisticUnit: "Phrasem"`. `generateSections` restores `meta.linguisticUnit` for `Lexem` and `Phrasem` entries. Disambiguate senses now forward optional `phrasemeKind` hints extracted from `meta.linguisticUnit`. -> **V14 scope**: Minimal Lemma + Generate enrichment cutover — `PromptKind.Lemma` now returns only classifier fields (`lemma`, `linguisticUnit`, `posLikeKind`, `surfaceKind`, optional `contextWithLinkedParts`). Core metadata (`emojiDescription`, `ipa`, noun-only `genus` + `nounClass`) moved to Generate via enrichment prompts. Features prompt is now POS-specific (`FeaturesNoun` ... `FeaturesInteractionalUnit`), and legacy `PromptKind.Features` is removed. Proper-noun/separable span expansion relies on `contextWithLinkedParts`; legacy `fullSurface` is removed. Runtime parsing remains backward-compatible with legacy Lemma keys (`pos` / `phrasemeKind`) by normalizing them to `posLikeKind`. Noun enrichment metadata (`genus`, `nounClass`) is treated as best-effort at parse time; header formatting first falls back to noun-inflection genus, then degrades to common header when genus is still missing. +> **V14 scope**: Minimal Lemma + Generate enrichment cutover — `PromptKind.Lemma` now returns only classifier fields (`lemma`, `linguisticUnit`, `posLikeKind`, `surfaceKind`, optional `contextWithLinkedParts`). Core metadata (`emojiDescription`, `ipa`, noun-only `genus` + `nounClass`) moved to Generate via enrichment prompts. Features prompt is now POS-specific (`FeaturesNoun` ... `FeaturesInteractionalUnit`), and legacy `PromptKind.Features` is removed. Proper-noun/separable span expansion relies on `contextWithLinkedParts`; legacy `fullSurface` is removed. Noun enrichment metadata (`genus`, `nounClass`) is treated as best-effort at parse time; header formatting first falls back to noun-inflection genus, then degrades to common header when genus is still missing. > **V15 scope**: Lemma safe-linking + deterministic target routing — Lemma now runs in two dispatch phases: pre-prompt safe link insertion (with optional Worter placeholder) and post-prompt final routing rewrite. Closed-set Lexem POS (`Pronoun`, `Article`, `Preposition`, `Conjunction`, `Particle`, `InteractionalUnit`) route to `Library///.md`; all other entries route to Worter sharded paths. Post-prompt phase can rename placeholder to final target, delete placeholder only when empty and final exists, retarget temporary links (including multi-span expansion), and navigate from placeholder to final note when needed. Background Generate now uses the latest resolved target path as its primary source of truth. > **V16 scope**: Prompt-stability + pipeline hardening — Lemma adds runtime output guardrails with one controlled retry for suspicious same-surface outputs (separable inflected verbs, comparative/superlative-like inflected adjectives). Unsafe `contextWithLinkedParts` rewrites are dropped when stripped text does not match source context. Background Generate cleanup is now ownership-aware: empty targets are auto-trashed only when invocation-owned (or truly newly created in this run). Disambiguate senses now include optional `senseGloss` (short text gloss) alongside emoji signals. -> **V17 scope**: Morphological relations v1 — Morphem output now supports optional top-level `derived_from` (single base) and `compounded_from` (immediate constituents). New DictSectionKind `Morphology` (`Morphologische Relationen`) is generated for Lexem entries (except proper nouns), ordered right after Morphem. Generate now enforces a 3-phase model (`lemma -> generation -> propagation`) by waiting for `WordTranslation` before all propagation steps. Propagation is split: `propagateMorphologyRelations` owns Lexem-side morphology backlinks (localized relation markers, e.g. German `Verwendet in:`) plus verb-prefix equations on prefix Morphem notes; `propagateMorphemes` owns bound-morpheme localized `used in` aggregation on Morphem notes (including non-verb prefixes and separable prefixes that are not covered by an equation). Relation propagation shares append/dedupe utilities and skips when source lemma is already referenced. +> **V17 scope**: Morphological relations — Morphem output now supports optional top-level `derived_from` (single base) and `compounded_from` (immediate constituents). New DictSectionKind `Morphology` (`Morphologische Relationen`) is generated for Lexem entries (except proper nouns), ordered right after Morphem. Generate now enforces a 3-phase model (`lemma -> generation -> propagation`) by waiting for `WordTranslation` before all propagation steps. Propagation is split: `propagateMorphologyRelations` owns Lexem-side morphology backlinks (localized relation markers, e.g. German `Verwendet in:`) plus verb-prefix equations on prefix Morphem notes; `propagateMorphemes` owns bound-morpheme localized `used in` aggregation on Morphem notes (including non-verb prefixes and separable prefixes that are not covered by an equation). Relation propagation shares append/dedupe utilities and skips when source lemma is already referenced. > **V9 scope**: LinguisticUnit DTO — Zod-schema-based type system as source of truth for DictEntries. German + Noun fully featured (`genus`, `nounClass`); all other POS/unit kinds have stubs. `GermanLinguisticUnit` built during Generate and stored in `meta.linguisticUnit`. Header prompt now returns `genus` ("Maskulinum"/"Femininum"/"Neutrum") instead of `article` ("der"/"die"/"das"); formatter derives article via `articleFromGenus`. New files: `surface-factory.ts`, `genus.ts`, `noun.ts`, `pos-features.ts`, `lexem-surface.ts`, `phrasem-surface.ts`, `morphem-surface.ts`, `linguistic-unit.ts`. 21 new DTO tests. @@ -311,7 +315,7 @@ D: dem [[Kohlekraftwerk]], den [[Kohlekraftwerken]] - **Header line**: emoji sequence (from `emojiDescription`) + `[[Surface]]` + pronunciation link + ` ^blockId` - **DictEntryId format** (validated by `DictEntryIdSchema`): `^{LinguisticUnitKindTag}-{SurfaceKindTag}(-{PosTag}-{index})` — the PosTag+index suffix is Lexem-only. E.g., `^lx-lm-nom-1` (Lexem, Lemma surface, Noun, 1st meaning). Final format TBD. - **DictEntrySections**: marked with `Title` -- **Multiple DictEntries** (different meanings of the same Surface) separated by `\n\n\n---\n---\n\n\n` (parser also accepts older `\n\n---\n---\n\n` and legacy `\n---\n---\n---\n`) +- **Multiple DictEntries** (different meanings of the same Surface) separated by `\n\n\n---\n---\n\n\n` ### 5.2 Parsed Representation @@ -319,7 +323,6 @@ D: dem [[Kohlekraftwerk]], den [[Kohlekraftwerken]] type DictEntryMeta = { entity?: DeEntity; // canonical DTO for generation/propagation linguisticUnit?: GermanLinguisticUnit; // V9: typed DTO (see section 15) - emojiDescription?: string[]; // legacy mirror for compatibility } & Record; type DictEntry = { @@ -344,7 +347,7 @@ Per-DictEntry metadata is stored in a hidden `

` at the bottom of the No ```html
-{"entries":{"LX-LM-NOUN-1":{"status":"Done","emojiDescription":["🏭","⚡"]},"LX-LM-NOUN-2":{"status":"NotStarted","emojiDescription":["🏰"]}}} +{"entries":{"LX-LM-NOUN-1":{"entity":{"emojiDescription":["🏭","⚡"],"ipa":"ˈkoːləˌkraftvɛɐ̯k","language":"German","lemma":"Kohlekraftwerk","linguisticUnit":"Lexem","posLikeKind":"Noun","surfaceKind":"Lemma","features":{"inflectional":{},"lexical":{"genus":"Neutrum","nounClass":"Common","pos":"Noun"}}}}}}
``` @@ -447,7 +450,7 @@ commandFn(input) → VaultAction[] → vam.dispatch(actions) |---------|--------|---------| | `Lemma` | V3 | Recon: classify word via LLM, disambiguate sense against existing entries (metadata `emojiDescription` lookup + Disambiguate prompt), wrap in wikilink, store result, notify user, fire background Generate. V5: bounds-check. V8: proper noun detection (nounClass), fullSurface expansion for multi-word proper nouns. V10: emoji-based disambiguation | | `Generate` | V3 | Build DictEntry: LLM-generated sections (Morphem, Relation, Inflection, Translation) + header from Lemma output + Attestation; re-encounter detection (via Lemma's disambiguationResult); cross-ref propagation; serialize, move to Wörter, notify user. Fires automatically in background after Lemma (user stays on source text); also callable manually. V5: scroll-to-entry (deferred via wikilink click handler when running in background) | -| `TranslateSelection` | V1 | Translate selected text via LLM | +| `TranslateSelection` | Implemented | Translate selected text via LLM | **Source**: `src/commanders/textfresser/textfresser.ts`, `src/commanders/textfresser/commands/types.ts` @@ -625,9 +628,9 @@ Filter entries by matching unitKind + POS (ignoring surfaceKind, so LX-LM-NOUN-* If no entries for this POS → disambiguationResult = null (new sense, skip Disambiguate call) ↓ Build senses: Array<{ index, emojiDescription, ipa?, senseGloss?, unitKind, pos?, genus? }> from metadata + parsed entry IDs - (prefer `meta.entity`; fallback to legacy `meta.emojiDescription` / `meta.linguisticUnit`) - (`senseGloss` fallback order: `meta.entity.senseGloss` → `meta.senseGloss` → first non-empty Translation line) - (V10: entries without emojiDescription → V2 legacy path: treat as re-encounter of first match) + (source of truth: `meta.entity`; no top-level mirror fallbacks) + (`senseGloss` fallback order: `meta.entity.senseGloss` → first non-empty Translation line) + (V10+: entries without emojiDescription are treated as new-sense candidates) ↓ Call PromptKind.Disambiguate with { lemma, context, senses } ↓ @@ -642,7 +645,7 @@ V5 bounds check: if matchedIndex is not in validIndices → treat as new sense **Key optimizations**: - The Disambiguate LLM call is skipped entirely when no note exists (first encounter) or no entries with matching POS exist (first sense for this POS) -- **V10**: When Disambiguate returns `matchedIndex: null` (new sense), it also returns an `emojiDescription` (1-3 emojis). This is stored as `LemmaResult.precomputedEmojiDescription` and used by Generate as the preferred source for `meta.emojiDescription` (falling back to `lemmaResult.emojiDescription` from Lemma LLM output). +- **V10**: When Disambiguate returns `matchedIndex: null` (new sense), it also returns an `emojiDescription` (1-3 emojis). This is stored as `LemmaResult.precomputedEmojiDescription` and used by Generate as the source for `meta.entity.emojiDescription`. - **V5**: `matchedIndex` is bounds-checked against `validIndices` — out-of-range values are treated as new sense (prevents LLM hallucinating invalid indices) The disambiguation result is stored in `LemmaResult.disambiguationResult` and consumed by Generate's `resolveExistingEntry` step, which no longer needs to re-parse or re-match. @@ -661,11 +664,20 @@ Generate fires automatically in the background after a successful Lemma command. checkAttestation → checkEligibility → checkLemmaResult → resolveExistingEntry (parse existing entries, use Lemma's disambiguationResult for re-encounter detection) → generateSections (async: LLM calls, or attestation append for re-encounters) - → propagateGeneratedSections (v2 core + source-note separability decoration) - → serializeEntry (includes noteKind + emojiDescription in single metadata upsert) → moveToWorter → addWriteAction + → propagateGeneratedSections (core propagation + source-note separability decoration) + → serializeEntry (includes noteKind + entity metadata in single metadata upsert) + → moveToWorter + → maintainClosedSetSurfaceHub (lazy Worter surface-hub management for ambiguous closed-set surfaces; no-op when library lookup is unavailable) + → addWriteAction ``` -`moveToWorter` is now policy-based: closed-set Lexem POS routes to Library, all other entries route to Worter sharded folders. It skips rename when the active file is already at destination. +`moveToWorter` is policy-based: closed-set Lexem POS routes to Library, all other entries route to Worter sharded folders. It skips rename when the active file is already at destination. + +Closed-set ambiguity support (manual links) is handled by `maintainClosedSetSurfaceHub`: + +- Canonical closed-set dict content stays in `Library`. +- For ambiguous closed-set surfaces, Textfresser maintains a minimal hub note at `Worter//closed-set-hub/.md`. +- LLM-confirmed attestation links still point directly to the specific Library note. Sync `Result` checks transition to async `ResultAsync` at `generateSections`. @@ -682,7 +694,7 @@ Matching ignores surfaceKind so that inflected encounters (e.g., "Schlosses" → Propagation-only stubs are explicitly excluded from re-encounter matching: if the matched entry has propagation sections (e.g., Morphology/Relation/Tags/Inflection) but lacks both Attestation and Translation, `resolveExistingEntry` drops that stub and forces full Generate path. This prevents propagation targets from getting stuck as permanent stubs. -#### Section Generation (V2) +#### Section Generation `generateSections` has two paths: @@ -694,7 +706,7 @@ All LLM calls are fired in parallel via `Promise.allSettled` (none depend on eac | Section | LLM? | PromptKind | Formatter | Output | |---------|------|-----------|-----------|--------| -| **Header** | No | — | `dispatchHeaderFormatter()` | `{emoji} [[lemma]], [{ipa}](youglish_url)` → `DictEntry.headerContent`. For nouns, article genus priority is: NounEnrichment genus, then noun-inflection genus fallback; when resolved, output is `{emoji} {article} [[lemma]], [{ipa}](youglish_url)` via `de/lexem/noun/header-formatter`. Dispatch routes by POS; common formatter for non-nouns or unresolved noun genus. `emoji` is rendered from the full `emojiDescription` sequence in order. No LLM call. Sense signals are stored on `meta.entity` (`emojiDescription`, `ipa`) with legacy mirror `meta.emojiDescription` for compatibility. | +| **Header** | No | — | `dispatchHeaderFormatter()` | `{emoji} [[lemma]], [{ipa}](youglish_url)` → `DictEntry.headerContent`. For nouns, article genus priority is: NounEnrichment genus, then noun-inflection genus fallback; when resolved, output is `{emoji} {article} [[lemma]], [{ipa}](youglish_url)` via `de/lexem/noun/header-formatter`. Dispatch routes by POS; common formatter for non-nouns or unresolved noun genus. `emoji` is rendered from the full `emojiDescription` sequence in order. No LLM call. Sense signals are stored on canonical `meta.entity` (`emojiDescription`, `ipa`). | | **Morphem** | Yes | `Morphem` | `morphemeFormatterHelper.formatSection()` | `[[kohle]]\|[[kraft]]\|[[werk]]` → `EntrySection` | | **Morphology** | No extra LLM call (reuses `Morphem` output) | — | `generateMorphologySection()` | Localized `derived_from`/`consists_of` markers (German: `Abgeleitet von:`, `Besteht aus:`) and verb-prefix equation lines (verbs only); `derived_from` remains explicit even when an equation is present unless it is equivalent to the equation base. Structured payload is also captured for propagation. | | **Relation** | Yes | `Relation` | `formatRelationSection()` | `= [[Synonym]], ⊃ [[Hypernym]]` → `EntrySection`. Raw output also stored for propagation. | @@ -709,7 +721,7 @@ Each `EntrySection` gets: #### Entry ID -Built via `dictEntryIdHelper.build()`. V2 uses `nextIndex` computed from existing entries: +Built via `dictEntryIdHelper.build()`. Generate uses `nextIndex` computed from existing entries: - Lexem: `LX-{SurfaceTag}-{PosTag}-{nextIndex}` (e.g., `LX-LM-NOUN-1`, `LX-LM-NOUN-2`) - Phrasem/Morphem: `{UnitTag}-{SurfaceTag}-{nextIndex}` @@ -743,13 +755,13 @@ For non-Lexem units, `pos` is passed to LLM prompts as the `linguisticUnit` name - **Multi-word selection**: Lemma handling phrasem attestations from multi-word selections - **Deviation section**: Additional LLM-generated section for irregular forms and exceptions - ~~**Scroll to latest updated entry**~~: Implemented in V5. `scrollToTargetBlock()` finds `^{blockId}` line and calls `ActiveFileService.scrollToLine()`. For background Generate: deferred via wikilink click handler — `awaitGenerateAndScroll()` waits for the in-flight promise, then scrolls if user is on the target note. -- ~~**Disambiguate prompt returning semantic info for new senses**~~: V5: returned text `semantics` gloss. V10: replaced with `emojiDescription` (1-3 emoji array). V11: `emojiDescription` moved to Lemma prompt output (Header prompt eliminated), stored in `meta.emojiDescription`. +- ~~**Disambiguate prompt returning semantic info for new senses**~~: V5: returned text `semantics` gloss. V10: replaced with `emojiDescription` (1-3 emoji array). V11: `emojiDescription` moved to Lemma prompt output (Header prompt eliminated), now persisted in canonical `meta.entity`. --- ## 9. Cross-Reference Propagation -> **Status**: V2 implemented. +> **Status**: implemented. When Generate fills DictEntrySections for a new DictEntry, the LLM output contains references to other Surfaces. Cross-reference propagation ensures those references are **bidirectional** — if A references B, then B's Note is updated to reference A back. @@ -790,9 +802,9 @@ Not all DictEntrySections participate in cross-reference propagation: **Source**: `src/commanders/textfresser/commands/generate/steps/propagate-generated-sections.ts`, `src/commanders/textfresser/commands/generate/steps/propagate-relations.ts`, `src/commanders/textfresser/commands/generate/steps/propagate-morphology-relations.ts`, `src/commanders/textfresser/commands/generate/steps/propagate-morphemes.ts`, `src/commanders/textfresser/commands/generate/steps/decorate-attestation-separability.ts`, `src/commanders/textfresser/commands/generate/steps/propagate-inflections.ts`, `src/commanders/textfresser/common/target-path-resolver.ts` -The propagation facade (`propagateGeneratedSections`) runs after `generateSections` in the Generate pipeline. Runtime path is v2-only: `propagateV2`, then `decorateAttestationSeparability` as a shared post-propagation source-note step. `propagateRelations` uses the raw `relations` output captured during section generation (not re-parsed from markdown). +The propagation facade (`propagateGeneratedSections`) runs after `generateSections` in the Generate pipeline. Runtime path is core-only: `propagateCore`, then `decorateAttestationSeparability` as a shared post-propagation source-note step. `propagateRelations` uses the raw `relations` output captured during section generation (not re-parsed from markdown). -`propagateV2` folds all scoped propagation actions to one write per target note. The fold contract accepts `ProcessMdFile` in both payload shapes (`transform` and `before/after`) and accepts non-null `UpsertMdFile` content as deterministic transform input, preserving original action order per target path. +`propagateCore` folds all scoped propagation actions to one write per target note. The fold contract accepts `ProcessMdFile` in both payload shapes (`transform` and `before/after`) and accepts non-null `UpsertMdFile` content as deterministic transform input, preserving original action order per target path. Propagation note-adapter warning logs are sampled (`first-N + periodic`) for repeated cases (embedded/unparseable wikilinks) to keep logs actionable on large notes. @@ -809,7 +821,7 @@ resolveTargetPath(word, desiredSurfaceKind, vamLookup, librarianLookup): If existing is in lemma/ but desired is inflected → use as-is (lemma files can hold both) ``` -The `librarianLookup` callback is wired in `main.ts` after Librarian init via `Textfresser.setLibrarianLookup()`, converting `LeafMatch[]` → `SplitPathToMdFile[]`. Before Librarian init, defaults to `() => []`. +The `librarianLookup` callback is wired in `main.ts` after Librarian init via `Textfresser.setLibrarianLookup()`, converting `LeafMatch[]` → `SplitPathToMdFile[]`. Before Librarian init it defaults to `() => []` and `TextfresserState.isLibraryLookupAvailable = false`; hub-maintenance and hub-backfill flows are guarded in this degraded mode to avoid destructive rewrites. ``` generateSections captures raw relation output (ParsedRelation[]) @@ -861,8 +873,7 @@ propagateInflections: 3. buildPropagationActionPair(splitPath, transform) → [UpsertMdFile, ProcessMdFile] transform: a. merge tags into existing new-format entry (same header) - b. auto-collapse legacy per-cell stubs: "#/ for: [[lemma]]" - c. ensure/update Tags section with normalized tag line + b. ensure/update Tags section with normalized tag line ↓ Propagation VaultActions (including healing) added to ctx.actions ``` @@ -878,7 +889,6 @@ Propagation VaultActions (including healing) added to ctx.actions - **One entry per form/POS**: All case/number combos are represented as tags in one entry - **Genus source + fallback**: propagation prefers genus from `NounInflection` output, falls back to `NounEnrichment` when needed, and still degrades to `#Inflection/Noun for: [[lemma]]` if both are missing - **Same-note skip**: When form === lemma, cells are skipped entirely (the main entry already covers this note) -- **Legacy migration in-place**: old per-cell stubs are collapsed into the new entry format when a target note is touched - Deterministic tag ordering: case order + number order, with dedup + localization normalization - Same UpsertMdFile + ProcessMdFile pattern as relation propagation - Skipped for re-encounters and non-noun POS @@ -1049,7 +1059,7 @@ bun run codegen:prompts - `Lemma` delegates to `executeLemmaFlow(...)` - `Generate` / `TranslateSelection` delegate to `actionCommandFnForCommandKind` - `createHandler()` delegates to `createWikilinkClickHandler(...)` -- `getState()` + `setLibrarianLookup(...)` +- `getState()` + `setLibrarianLookup(...)` + `clearLibrarianLookup()` - private `scrollToTargetBlock()` UX helper State is owned by `TextfresserState` in `state/textfresser-state.ts`: @@ -1062,6 +1072,7 @@ State is owned by `TextfresserState` in `state/textfresser-state.ts`: - `inFlightGenerate` / `pendingGenerate` - `targetBlockId` - `latestFailedSections` +- `isLibraryLookupAvailable` ### 11.2 Unified Lemma Path @@ -1143,8 +1154,8 @@ To add support for a new target language (e.g., Japanese): | File | Purpose | |------|---------| | **Textfresser Commander** | | -| `src/commanders/textfresser/textfresser.ts` | Thin public orchestrator: constructor wiring, command delegation, handler delegation, `setLibrarianLookup()`, and `scrollToTargetBlock()` | -| `src/commanders/textfresser/state/textfresser-state.ts` | TextfresserState + `InFlightGenerate` / `PendingGenerate` / `LemmaInvocationCache` + `createInitialTextfresserState()` | +| `src/commanders/textfresser/textfresser.ts` | Thin public orchestrator: constructor wiring, command delegation, handler delegation, lookup wiring (`setLibrarianLookup()` / `clearLibrarianLookup()`), and `scrollToTargetBlock()` | +| `src/commanders/textfresser/state/textfresser-state.ts` | TextfresserState + `InFlightGenerate` / `PendingGenerate` / `LemmaInvocationCache` + lookup-availability guard flag + `createInitialTextfresserState()` | | `src/commanders/textfresser/orchestration/lemma/execute-lemma-flow.ts` | Unified Lemma execution path (cache check, two-phase run, notifications, cache persistence, background trigger) | | `src/commanders/textfresser/orchestration/lemma/run-lemma-two-phase.ts` | Phase A/Phase B Lemma routing and source rewrite orchestration | | `src/commanders/textfresser/orchestration/lemma/lemma-output-guardrails.ts` | Lemma output guardrails: separable-verb/adjective checks, `contextWithLinkedParts` stripped-text validation, best-effort retry selection | @@ -1157,21 +1168,24 @@ To add support for a new target language (e.g., Japanese): | `src/commanders/textfresser/errors.ts` | TextfresserCommandError (extends BaseCommandError), AttestationParsingError | | **Commands** | | | `src/commanders/textfresser/commands/lemma/lemma-command.ts` | Lemma rewrite helpers: attestation resolution, wikilink formatting, robust two-pass source rewrite (`temporary → final`, block-id fallback, multi-span retargeting) | -| `src/commanders/textfresser/commands/lemma/steps/disambiguate-sense.ts` | V3: look up existing note, match entries by unitKind+POS (ignoring surfaceKind), call Disambiguate prompt. V5: bounds-check matchedIndex, log parse failures. V10+: emoji-based senses with emojiDescription+unitKind+pos+genus, plus optional phrasemeKind hints from `meta.linguisticUnit`. V16: adds optional `senseGloss` per sense (`entity`/legacy/meta+translation fallback) for homonym quality. Returns precomputedEmojiDescription for new-sense path. | +| `src/commanders/textfresser/commands/lemma/steps/disambiguate-sense.ts` | V3: look up existing note, match entries by unitKind+POS (ignoring surfaceKind), call Disambiguate prompt. V5: bounds-check matchedIndex, log parse failures. V10+: emoji-based senses with emojiDescription+unitKind+pos+genus. V16: adds optional `senseGloss` per sense (`entity` + translation fallback) for homonym quality. Returns precomputedEmojiDescription for new-sense path. | | `src/commanders/textfresser/commands/lemma/types.ts` | LemmaResult type (V3: disambiguationResult, V10: precomputedEmojiDescription, V8: nounClass) | | `src/commanders/textfresser/commands/generate/generate-command.ts` | Generate pipeline orchestrator | | `src/commanders/textfresser/commands/generate/steps/check-attestation.ts` | Sync check: attestation available | | `src/commanders/textfresser/commands/generate/steps/check-lemma-result.ts` | Sync check: lemma result available | | `src/commanders/textfresser/commands/generate/steps/resolve-existing-entry.ts` | Parse existing entries, use Lemma's disambiguationResult for re-encounter detection | -| `src/commanders/textfresser/commands/generate/steps/generate-sections.ts` | Async: LLM calls per section (or append attestation for re-encounters). V12: Features prompt + Tags section, article in header for nouns. V13: `meta.linguisticUnit` restored for Lexem/Phrasem entries (`Morphem` still out of scope). Stores `meta.emojiDescription` from precomputedEmojiDescription or lemmaResult.emojiDescription. Header built from LemmaResult fields. Sets targetBlockId | +| `src/commanders/textfresser/commands/generate/steps/generate-sections.ts` | Async: LLM calls per section (or append attestation for re-encounters). V12: Features prompt + Tags section, article in header for nouns. V13: `meta.linguisticUnit` restored for Lexem/Phrasem entries (`Morphem` still out of scope). Persists canonical `meta.entity` (including emoji/ipa/sense). Header built from LemmaResult fields. Sets targetBlockId | | `src/commanders/textfresser/commands/generate/steps/propagate-relations.ts` | Cross-ref: compute inverse relations, resolve target paths via shared resolver, generate actions for target notes | | `src/commanders/textfresser/commands/generate/steps/propagate-morphology-relations.ts` | Morphology propagation: Lexem localized `used in` backlinks for derivation/compounding + verb-prefix equation propagation on decorated prefix Morphem entries (with non-dict fallback append path) | | `src/commanders/textfresser/commands/generate/steps/propagate-morphemes.ts` | Morpheme propagation: bound-morpheme localized `used in` aggregation on Morphem entries (Suffix/Interfix/etc + non-verb Prefix), with Root/Suffixoid fallback when morphology payload is incomplete | -| `src/commanders/textfresser/commands/generate/steps/propagate-inflections.ts` | Noun inflection propagation: resolve target paths via shared resolver, create/update one inflection entry per form, merge tags, collapse legacy per-cell stubs | -| `src/commanders/textfresser/commands/generate/steps/propagate-v2.ts` | V2 propagation orchestration + fold-to-single-write contract per target note | -| `src/commanders/textfresser/commands/generate/steps/propagation-v2-ports-adapter.ts` | V2 IO adapter ports (`readManyMdFiles`, typed missing/error classification, write-action construction) | +| `src/commanders/textfresser/commands/generate/steps/propagate-inflections.ts` | Noun inflection propagation: resolve target paths via shared resolver, create/update one inflection entry per form, merge tags | +| `src/commanders/textfresser/commands/generate/steps/propagate-core.ts` | Core propagation orchestration + fold-to-single-write contract per target note | +| `src/commanders/textfresser/commands/generate/steps/propagation-ports-adapter.ts` | Propagation IO adapter ports (`readManyMdFiles`, typed missing/error classification, write-action construction) | | `src/commanders/textfresser/commands/generate/steps/move-to-worter.ts` | Final destination policy step: closed-set Lexem POS → Library path, others → Worter sharded path, rename skipped when already at destination | -| `src/commanders/textfresser/common/lemma-link-routing.ts` | Link-target policy helper: closed-set detection, pre-prompt target resolution, final target resolution, link-target formatting | +| `src/commanders/textfresser/commands/generate/steps/maintain-closed-set-surface-hub.ts` | Closed-set hub maintenance step: lazily create/update/trash `Worter//closed-set-hub/.md` for ambiguous closed-set surfaces; guarded by `isLibraryLookupAvailable` | +| `src/commanders/textfresser/common/lemma-link-routing.ts` | Link-target policy helper: closed-set detection, pre-prompt target resolution, final target resolution, and basename-default rendering with Library ambiguity fallback to full-path | +| `src/commanders/textfresser/common/closed-set-surface-hub.ts` | Closed-set surface hub policy helpers: target detection, hub content rendering, action planning, backfill action builder | +| `src/commanders/textfresser/common/target-comparison.ts` | Domain comparison canonicalization (`trim + case-fold`) for target equality checks outside `wikilinkHelper` | | `src/commanders/textfresser/common/target-path-resolver.ts` | Shared path resolution for propagation: two-source lookup (VAM → Librarian → computed sharded path), inflected→lemma healing, shared morpheme path resolver for prefix Library fallback, `buildPropagationActionPair` helper | | `src/commanders/textfresser/common/sharded-path.ts` | Sharded path computation for Worter entries; exports `SURFACE_KIND_PATH_INDEX` for healing checks | | `src/commanders/textfresser/domain/propagation/note-adapter.ts` | Typed parse/serialize adapter for propagation sections with passthrough handling and sampled warning logs | @@ -1234,11 +1248,11 @@ To add support for a new target language (e.g., Japanese): | `tests/unit/textfresser/formatters/relation-formatter.test.ts` | Relation formatter: symbol notation, grouping, dedup | | `tests/unit/textfresser/formatters/inflection-formatter.test.ts` | Generic inflection formatter: label/forms rows | | `tests/unit/textfresser/formatters/de/lexem/noun/inflection-formatter.test.ts` | Noun inflection: case grouping, N/A/G/D order, cells pass-through | -| `tests/unit/textfresser/steps/disambiguate-sense.test.ts` | Disambiguate: mock VAM + PromptRunner, bounds check, precomputed emojiDescription, V2 legacy | +| `tests/unit/textfresser/steps/disambiguate-sense.test.ts` | Disambiguate: mock VAM + PromptRunner, bounds check, precomputed emojiDescription, missing-emoji fallback | | `tests/unit/textfresser/steps/propagate-relations.test.ts` | Relation propagation: inverse kinds, self-ref skip, dedup, VaultAction shapes, healing when target in inflected/ | | `tests/unit/textfresser/steps/propagate-inflections.test.ts` | Inflection propagation: form grouping, same-note skip, single-entry tags merge, legacy stub collapse, genus fallback + header upgrade, VAM path reuse | -| `tests/unit/textfresser/steps/propagate-v2-phase4.test.ts` | V2 propagation parity/idempotency/one-write invariants, fold contract behavior, phase-5 migrated slice parity | -| `tests/unit/textfresser/steps/propagation-v2-ports-adapter.test.ts` | V2 port adapter contract: dedupe reads, typed missing/error classification, candidate lookup, action-pair construction | +| `tests/unit/textfresser/steps/propagate-generated-sections.test.ts` | Wrapper routing and fail-fast propagation behavior for Generate | +| `tests/unit/textfresser/steps/propagation-ports-adapter.test.ts` | Port adapter contract: dedupe reads, typed missing/error classification, candidate lookup, action-pair construction | | `tests/unit/textfresser/domain/propagation/note-adapter.test.ts` | Typed section parse/serialize characterization + deterministic canonicalization + passthrough guarantees | | `tests/unit/textfresser/domain/propagation/morphology-roundtrip-corpus.test.ts` | Morphology mixed-marker regression corpus + roundtrip equivalence guards against backlink/equation reclassification | | `tests/unit/textfresser/common/target-path-resolver.test.ts` | Path resolver: VAM/librarian lookup, computed fallback, inflected→lemma healing, no-heal cases, `buildPropagationActionPair` | @@ -1257,9 +1271,9 @@ To add support for a new target language (e.g., Japanese): - Dropped `DictSectionKind.Definition` and `PromptKind.Semantics` entirely - V11: Header prompt eliminated — `emojiDescription` and `ipa` moved to Lemma prompt output - Disambiguate prompt uses emoji-based senses (emojiDescription + unitKind + pos + genus) -- `meta.emojiDescription: string[]` replaces `meta.semantics: string` +- `meta.entity.emojiDescription: string[]` replaces `meta.semantics: string` - `LemmaResult.precomputedEmojiDescription` replaces `precomputedSemantics` -- V2 legacy path: entries without `emojiDescription` fall back to first-match +- Entries without `emojiDescription` are treated as new-sense candidates ### Deferred Items @@ -1272,7 +1286,7 @@ To add support for a new target language (e.g., Japanese): ## 15. LinguisticUnit DTO — Source of Truth Type System -> **Status**: V9+ — implemented (German + Noun/Verb/Adjective full features; remaining POS/unit kinds are stubs). Canonical DTO is `DictEntry.meta.entity` (`Entity` with `features.lexical` + `features.inflectional`), while `meta.linguisticUnit` remains as a compatibility mirror. +> **Status**: V9+ — implemented (German + Noun/Verb/Adjective full features; remaining POS/unit kinds are stubs). Canonical DTO is `DictEntry.meta.entity` (`Entity` with `features.lexical` + `features.inflectional`). `meta.linguisticUnit` is auxiliary typed surface metadata. ### 15.1 Problem @@ -1448,7 +1462,7 @@ const newEntry: DictEntry = { headerContent, id: entryId, meta: { - emojiDescription: lemmaResult.precomputedEmojiDescription ?? lemmaResult.emojiDescription, + ...(entity ? { entity } : {}), ...(linguisticUnit ? { linguisticUnit } : {}), }, sections, diff --git a/src/documentaion/vam-architecture.md b/src/documentaion/vam-architecture.md index 50f904179..f4159c7fc 100644 --- a/src/documentaion/vam-architecture.md +++ b/src/documentaion/vam-architecture.md @@ -1,6 +1,10 @@ # VaultActionManager — Architecture > **Scope**: This document covers the VaultActionManager (VAM) — the file system abstraction layer used by all other managers and commanders. For the vocabulary/dictionary subsystem, see `textfresser-architecture.md`. For tree/healing/codex, see the Librarian docs. For E2E testing, see `e2e-architecture.md`. +> +> **Compatibility Policy (Dev Mode, 2026-02-20)**: +> - Textfresser is treated as green-field. Breaking changes are allowed; no backward-compatibility guarantees for Textfresser note formats, schemas, or intermediate contracts. +> - Librarian and VAM are stability-critical infrastructure. Changes there require conservative rollout, migration planning when persisted contracts change, and explicit regression coverage. --- diff --git a/src/global-state/parsed-settings.ts b/src/global-state/parsed-settings.ts index e96e1aa49..b0038061b 100644 --- a/src/global-state/parsed-settings.ts +++ b/src/global-state/parsed-settings.ts @@ -8,8 +8,6 @@ import type { Prettify } from "../types/helpers"; import { buildCanonicalDelimiter, buildFlexibleDelimiterPattern, - isSuffixDelimiterConfig, - migrateStringDelimiter, } from "../utils/delimiter"; export type ParsedUserSettings = Prettify< @@ -29,12 +27,7 @@ export function parseSettings(settings: TextEaterSettings): ParsedUserSettings { ); } - // Migrate old string format to new config format - const suffixDelimiterConfig = isSuffixDelimiterConfig( - settings.suffixDelimiter, - ) - ? settings.suffixDelimiter - : migrateStringDelimiter(settings.suffixDelimiter as unknown as string); + const suffixDelimiterConfig = settings.suffixDelimiter; const { libraryRoot: _, suffixDelimiter: __, ...rest } = settings; diff --git a/src/linguistics/de/lemma/de-lemma-result.ts b/src/linguistics/de/lemma/de-lemma-result.ts index 25b9cb5d9..428315bda 100644 --- a/src/linguistics/de/lemma/de-lemma-result.ts +++ b/src/linguistics/de/lemma/de-lemma-result.ts @@ -37,166 +37,17 @@ export const DePhrasemLemmaResultSchema = deLemmaResultBaseSchema.extend({ posLikeKind: PhrasemeKindSchema, }); -const deLemmaResultCompatInputSchema = deLemmaResultBaseSchema.extend({ - linguisticUnit: DeLemmaLinguisticUnitSchema, - phrasemeKind: PhrasemeKindSchema.nullable().optional(), - pos: DeLexemPosSchema.nullable().optional(), - posLikeKind: DePosLikeKindSchema.nullable().optional(), -}); - -export const DeLemmaResultSchema = deLemmaResultCompatInputSchema - .superRefine((value, ctx) => { - if (value.linguisticUnit === "Lexem") { - if ( - value.phrasemeKind !== undefined && - value.phrasemeKind !== null - ) { - ctx.addIssue({ - code: z.ZodIssueCode.custom, - message: - 'Lexem must not provide "phrasemeKind" (legacy alias is only valid for Phrasem)', - path: ["phrasemeKind"], - }); - } - - const posLikeAsLexem = DeLexemPosSchema.safeParse( - value.posLikeKind, - ); - if ( - value.posLikeKind !== undefined && - value.posLikeKind !== null && - !posLikeAsLexem.success - ) { - ctx.addIssue({ - code: z.ZodIssueCode.custom, - message: - 'Lexem "posLikeKind" must be a lexical POS (Noun, Verb, ...)', - path: ["posLikeKind"], - }); - } - - if ( - !posLikeAsLexem.success && - (value.pos === undefined || value.pos === null) - ) { - ctx.addIssue({ - code: z.ZodIssueCode.custom, - message: - 'Lexem requires either canonical "posLikeKind" or legacy "pos"', - path: ["posLikeKind"], - }); - } - - if ( - posLikeAsLexem.success && - value.pos && - value.pos !== posLikeAsLexem.data - ) { - ctx.addIssue({ - code: z.ZodIssueCode.custom, - message: - 'Conflicting Lexem POS values: "posLikeKind" and "pos" must match', - path: ["posLikeKind"], - }); - } - return; - } - - if (value.pos !== undefined && value.pos !== null) { - ctx.addIssue({ - code: z.ZodIssueCode.custom, - message: - 'Phrasem must not provide "pos" (legacy alias is only valid for Lexem)', - path: ["pos"], - }); - } - - const posLikeAsPhrasem = PhrasemeKindSchema.safeParse( - value.posLikeKind, - ); - if ( - value.posLikeKind !== undefined && - value.posLikeKind !== null && - !posLikeAsPhrasem.success - ) { - ctx.addIssue({ - code: z.ZodIssueCode.custom, - message: - 'Phrasem "posLikeKind" must be a phraseme kind (Idiom, Collocation, ...)', - path: ["posLikeKind"], - }); - } - - if ( - !posLikeAsPhrasem.success && - (value.phrasemeKind === undefined || value.phrasemeKind === null) - ) { - ctx.addIssue({ - code: z.ZodIssueCode.custom, - message: - 'Phrasem requires either canonical "posLikeKind" or legacy "phrasemeKind"', - path: ["posLikeKind"], - }); - } - - if ( - posLikeAsPhrasem.success && - value.phrasemeKind && - value.phrasemeKind !== posLikeAsPhrasem.data - ) { - ctx.addIssue({ - code: z.ZodIssueCode.custom, - message: - 'Conflicting Phrasem kind values: "posLikeKind" and "phrasemeKind" must match', - path: ["posLikeKind"], - }); - } - }) +export const DeLemmaResultSchema = z + .discriminatedUnion("linguisticUnit", [ + DeLexemLemmaResultSchema, + DePhrasemLemmaResultSchema, + ]) .transform((value) => { - const contextWithLinkedParts = - value.contextWithLinkedParts ?? undefined; - - if (value.linguisticUnit === "Lexem") { - const parsedPosLikeKind = DeLexemPosSchema.safeParse( - value.posLikeKind, - ); - const posLikeKind = parsedPosLikeKind.success - ? parsedPosLikeKind.data - : value.pos; - - if (!posLikeKind) { - throw new Error( - 'Invalid Lexem lemma result: missing both "posLikeKind" and "pos"', - ); - } - - return { - contextWithLinkedParts, - lemma: value.lemma, - linguisticUnit: "Lexem" as const, - posLikeKind, - surfaceKind: value.surfaceKind, - }; - } - - const parsedPhrasemeLikeKind = PhrasemeKindSchema.safeParse( - value.posLikeKind, - ); - const posLikeKind = parsedPhrasemeLikeKind.success - ? parsedPhrasemeLikeKind.data - : value.phrasemeKind; - - if (!posLikeKind) { - throw new Error( - 'Invalid Phrasem lemma result: missing both "posLikeKind" and "phrasemeKind"', - ); - } - return { - contextWithLinkedParts, + contextWithLinkedParts: value.contextWithLinkedParts ?? undefined, lemma: value.lemma, - linguisticUnit: "Phrasem" as const, - posLikeKind, + linguisticUnit: value.linguisticUnit, + posLikeKind: value.posLikeKind, surfaceKind: value.surfaceKind, }; }); diff --git a/src/linguistics/de/lexem/verb/features.ts b/src/linguistics/de/lexem/verb/features.ts index 4694d1835..651655926 100644 --- a/src/linguistics/de/lexem/verb/features.ts +++ b/src/linguistics/de/lexem/verb/features.ts @@ -1,7 +1,7 @@ import { z } from "zod/v3"; import type { POS } from "../../../common/enums/linguistic-units/lexem/pos"; -const germanVerbConjugationValues = ["Irregular", "Rregular"] as const; +const germanVerbConjugationValues = ["Irregular", "Regular"] as const; export const GermanVerbConjugationSchema = z.enum(germanVerbConjugationValues); export type GermanVerbConjugation = z.infer; diff --git a/src/main-stripped.ts b/src/main-stripped.ts index 04bfde07e..0acf62508 100644 --- a/src/main-stripped.ts +++ b/src/main-stripped.ts @@ -1,6 +1,6 @@ /** * Stripped-down interface for TextEaterPlugin. - * Used by settings and legacy code that doesn't need the full plugin. + * Used by settings and utility modules that don't need the full plugin. */ import type { Plugin } from "obsidian"; import type { ActiveFileService } from "./managers/obsidian/vault-action-manager/file-services/active-view/active-file-service"; @@ -8,7 +8,7 @@ import type { TextEaterSettings } from "./types"; /** * Minimal interface representing the TextEater plugin. - * Contains only the properties needed by settings and legacy file operations. + * Contains only the properties needed by settings and file operations. */ export default interface TextEaterPluginStripped extends Plugin { settings: TextEaterSettings; diff --git a/src/main.ts b/src/main.ts index a345b1228..5b2f6ae88 100644 --- a/src/main.ts +++ b/src/main.ts @@ -2,6 +2,7 @@ import { Modal, Notice, Plugin, TFile } from "obsidian"; import { DelimiterChangeService } from "./commanders/librarian/delimiter-change-service"; import { Librarian } from "./commanders/librarian/librarian"; import { cleanupDictNote } from "./commanders/textfresser/common/cleanup/cleanup-dict-note"; +import { buildClosedSetSurfaceHubBackfillActions } from "./commanders/textfresser/common/closed-set-surface-hub"; import { DICT_ENTRY_NOTE_KIND } from "./commanders/textfresser/common/metadata"; import { Textfresser } from "./commanders/textfresser/textfresser"; import { @@ -44,8 +45,6 @@ import { import { buildCanonicalDelimiter, buildFlexibleDelimiterPattern, - isSuffixDelimiterConfig, - migrateStringDelimiter, } from "./utils/delimiter"; import { getErrorMessage } from "./utils/get-error-message"; import { whenIdle as whenIdleTracker } from "./utils/idle-tracker"; @@ -250,19 +249,7 @@ export default class TextEaterPlugin extends Plugin { await this.librarian.init(); // Wire librarian corename lookup into Textfresser for propagation path resolution - if (this.textfresser) { - const lib = this.librarian; - this.textfresser.setLibrarianLookup((name) => - lib.findMatchingLeavesByCoreName(name).map( - (m): SplitPathToMdFile => ({ - basename: m.basename, - extension: "md", - kind: "MdFile", - pathParts: m.pathParts, - }), - ), - ); - } + this.wireLibrarianLookup(); // Register user event handlers after librarian is initialized const handlers = createHandlers( @@ -275,6 +262,7 @@ export default class TextEaterPlugin extends Plugin { ); } } catch (error) { + this.clearLibrarianLookup(); logger.error( "[TextEaterPlugin] Failed to initialize librarian:", getErrorMessage(error), @@ -322,16 +310,6 @@ export default class TextEaterPlugin extends Plugin { const loadedData = await this.loadData(); this.settings = Object.assign({}, DEFAULT_SETTINGS, loadedData); - // Migrate old string delimiter format to new config format - if ( - loadedData?.suffixDelimiter && - !isSuffixDelimiterConfig(loadedData.suffixDelimiter) - ) { - this.settings.suffixDelimiter = migrateStringDelimiter( - loadedData.suffixDelimiter as string, - ); - } - // Initialize global state with parsed settings initializeState(this.settings); // Store initial settings for change detection (deep copy delimiter) @@ -431,6 +409,14 @@ export default class TextEaterPlugin extends Plugin { name: "Generate dictionary entry", }); + this.addCommand({ + callback: () => { + void this.rebuildClosedSetSurfaceHubs(); + }, + id: "rebuild-closed-set-surface-hubs", + name: "Rebuild closed-set surface hubs", + }); + this.addCommand({ editorCheckCallback: (checking: boolean) => { if (!checking) { @@ -443,6 +429,58 @@ export default class TextEaterPlugin extends Plugin { }); } + private async rebuildClosedSetSurfaceHubs(): Promise { + if (!this.textfresser) { + new Notice("Textfresser is not initialized"); + return; + } + if (!this.textfresser.getState().isLibraryLookupAvailable) { + logger.warn( + "[TextEaterPlugin.rebuildClosedSetSurfaceHubs] Library lookup is unavailable; aborting to avoid destructive backfill", + ); + new Notice( + "Closed-set hub rebuild is unavailable until Librarian lookup is ready", + ); + return; + } + + logger.info( + "[TextEaterPlugin.rebuildClosedSetSurfaceHubs] Starting rebuild", + ); + new Notice("Rebuilding closed-set hubs..."); + + const result = await buildClosedSetSurfaceHubBackfillActions({ + lookupInLibrary: this.textfresser.getState().lookupInLibrary, + targetLanguage: this.textfresser.getState().languages.target, + vam: this.vam, + }); + if (result.isErr()) { + logger.warn( + "[TextEaterPlugin.rebuildClosedSetSurfaceHubs] Failed to build actions:", + result.error, + ); + new Notice(`Failed to rebuild closed-set hubs: ${result.error}`); + return; + } + + if (result.value.length === 0) { + new Notice("Closed-set hubs are already up to date"); + return; + } + + const dispatchResult = await this.vam.dispatch(result.value); + if (dispatchResult.isErr()) { + logger.warn( + "[TextEaterPlugin.rebuildClosedSetSurfaceHubs] Failed to dispatch actions:", + dispatchResult.error, + ); + new Notice("Failed to rebuild closed-set hubs"); + return; + } + + new Notice(`Updated closed-set hubs (${result.value.length} actions)`); + } + getActiveFileServiceTestingApi() { return { activeFileService: this.testingActiveFileService, @@ -679,6 +717,7 @@ export default class TextEaterPlugin extends Plugin { teardown(); } this.handlerTeardowns = []; + this.clearLibrarianLookup(); if (this.librarian) { await this.librarian.unsubscribe(); @@ -686,6 +725,7 @@ export default class TextEaterPlugin extends Plugin { this.librarian = new Librarian(this.vam); try { await this.librarian.init(); + this.wireLibrarianLookup(); // Register new handlers const handlers = createHandlers( @@ -698,12 +738,34 @@ export default class TextEaterPlugin extends Plugin { ); } } catch (error) { + this.clearLibrarianLookup(); logger.error( "[TextEaterPlugin] Failed to reinitialize librarian:", getErrorMessage(error), ); } } + + private wireLibrarianLookup(): void { + if (!this.textfresser || !this.librarian) { + return; + } + const lib = this.librarian; + this.textfresser.setLibrarianLookup((name) => + lib.findMatchingLeavesByCoreName(name).map( + (m): SplitPathToMdFile => ({ + basename: m.basename, + extension: "md", + kind: "MdFile", + pathParts: m.pathParts, + }), + ), + ); + } + + private clearLibrarianLookup(): void { + this.textfresser?.clearLibrarianLookup(); + } } /** diff --git a/src/managers/obsidian/behavior-manager/checkbox-behavior.ts b/src/managers/obsidian/behavior-manager/checkbox-behavior.ts index 2b96551a2..c41062f9c 100644 --- a/src/managers/obsidian/behavior-manager/checkbox-behavior.ts +++ b/src/managers/obsidian/behavior-manager/checkbox-behavior.ts @@ -20,7 +20,12 @@ export function createCheckboxFrontmatterHandler( return { doesApply: () => true, handle: (payload) => { - librarian.handlePropertyCheckboxClick(payload); + const maybeWithHandler = librarian as unknown as { + handlePropertyCheckboxClick?: ( + payload: CheckboxFrontmatterPayload, + ) => void; + }; + maybeWithHandler.handlePropertyCheckboxClick?.(payload); return { outcome: HandlerOutcome.Handled }; }, }; diff --git a/src/managers/obsidian/behavior-manager/wikilink-complition-behavior.ts b/src/managers/obsidian/behavior-manager/wikilink-complition-behavior.ts index 7ac7e6729..ab6bc204d 100644 --- a/src/managers/obsidian/behavior-manager/wikilink-complition-behavior.ts +++ b/src/managers/obsidian/behavior-manager/wikilink-complition-behavior.ts @@ -8,39 +8,65 @@ * 3. Corename lookup in tree → replace target with suffixed basename + alias */ -import type { Librarian } from "../../../commanders/librarian/librarian"; +import type { LeafMatch } from "../../../commanders/librarian/healer/library-tree/types/leaf-match"; import { nonEmptyArrayResult } from "../../../types/utils"; -import { - type EventHandler, - type HandlerContext, - HandlerOutcome, - type WikilinkPayload, -} from "../user-event-interceptor"; +import type { WikilinkPayload } from "../user-event-interceptor/events/wikilink/payload"; import { pickClosestLeaf } from "./pick-closest-leaf"; +type WikilinkCompletionLibrarianPort = { + resolveWikilinkAlias(linkContent: string): string | null; + findMatchingLeavesByCoreName(coreName: string): LeafMatch[]; +}; + +type MinimalWikilinkHandlerContext = { + app: { + metadataCache: { + getFirstLinkpathDest: ( + linkpath: string, + sourcePath: string, + ) => unknown | null; + }; + workspace: { + getActiveFile: () => { path: string } | null; + }; + }; +}; + +type WikilinkCompletionOutcome = + | { outcome: "Passthrough" } + | { outcome: "Modified"; data: WikilinkPayload }; + +type WikilinkCompletionHandler = { + doesApply: (payload: WikilinkPayload) => boolean; + handle: ( + payload: WikilinkPayload, + ctx: MinimalWikilinkHandlerContext, + ) => WikilinkCompletionOutcome; +}; + /** * Create a handler for wikilink completion. * Thin routing layer - delegates to librarian methods. */ export function createWikilinkCompletionHandler( - librarian: Librarian, -): EventHandler { + librarian: WikilinkCompletionLibrarianPort, +): WikilinkCompletionHandler { return { doesApply: () => true, - handle: (payload, ctx: HandlerContext) => { + handle: (payload, ctx) => { // 1. Try suffix-based alias resolution (existing behavior) const alias = librarian.resolveWikilinkAlias(payload.linkContent); if (alias !== null) { return { data: { ...payload, aliasToInsert: alias }, - outcome: HandlerOutcome.Modified, + outcome: "Modified", }; } // 2. Check if Obsidian can already resolve this link const activeFile = ctx.app.workspace.getActiveFile(); if (!activeFile) { - return { outcome: HandlerOutcome.Passthrough }; + return { outcome: "Passthrough" }; } const resolved = ctx.app.metadataCache.getFirstLinkpathDest( @@ -48,7 +74,7 @@ export function createWikilinkCompletionHandler( activeFile.path, ); if (resolved) { - return { outcome: HandlerOutcome.Passthrough }; + return { outcome: "Passthrough" }; } // 3. Look up corename in library tree @@ -57,7 +83,12 @@ export function createWikilinkCompletionHandler( ); const nonEmpty = nonEmptyArrayResult(matches); if (nonEmpty.isErr()) { - return { outcome: HandlerOutcome.Passthrough }; + return { outcome: "Passthrough" }; + } + // Ambiguous corename match: do not auto-pick one leaf target. + // Keep user's input untouched to avoid silent mislinking. + if (nonEmpty.value.length > 1) { + return { outcome: "Passthrough" }; } const currentPathParts = activeFile.path.split("/"); @@ -72,7 +103,7 @@ export function createWikilinkCompletionHandler( aliasToInsert: payload.linkContent, resolvedTarget: best.basename, }, - outcome: HandlerOutcome.Modified, + outcome: "Modified", }; }, }; diff --git a/src/managers/obsidian/command-executor/create-command-executor.ts b/src/managers/obsidian/command-executor/create-command-executor.ts index 7a5af4773..d59504823 100644 --- a/src/managers/obsidian/command-executor/create-command-executor.ts +++ b/src/managers/obsidian/command-executor/create-command-executor.ts @@ -50,7 +50,6 @@ export function createCommandExecutor(managers: CommandExecutorManagers) { switch (kind) { case CommandKind.GoToPrevPage: case CommandKind.GoToNextPage: - case CommandKind.MakeText: case CommandKind.SplitToPages: case CommandKind.SplitInBlocks: { // Delegate to librarian - codex guard handled internally diff --git a/src/managers/obsidian/user-event-interceptor/events/click/action-element/detector.ts b/src/managers/obsidian/user-event-interceptor/events/click/action-element/detector.ts index eabdfe245..0791d1c0a 100644 --- a/src/managers/obsidian/user-event-interceptor/events/click/action-element/detector.ts +++ b/src/managers/obsidian/user-event-interceptor/events/click/action-element/detector.ts @@ -7,8 +7,7 @@ * - Overflow menu items * * Note: Action elements typically don't use handlers since they - * are already handled by the action executor system. This detector - * primarily exists for compatibility and future extension. + * are already handled by the action executor system. */ import { DomSelectors } from "../../../../../../utils/dom-selectors"; diff --git a/src/managers/obsidian/vault-action-manager/file-services/background/helpers/tfile-helper.ts b/src/managers/obsidian/vault-action-manager/file-services/background/helpers/tfile-helper.ts index 0a011615f..274fd31c8 100644 --- a/src/managers/obsidian/vault-action-manager/file-services/background/helpers/tfile-helper.ts +++ b/src/managers/obsidian/vault-action-manager/file-services/background/helpers/tfile-helper.ts @@ -27,6 +27,9 @@ import { import type { Transform } from "../../../types/vault-action"; import { type CollisionStrategy, getExistingBasenamesInFolder } from "./common"; +const GET_FILE_RETRY_COUNT = 10; +const GET_FILE_RETRY_DELAY_MS = 50; + /** * Helper for TFile operations in the vault. * @@ -60,6 +63,26 @@ export class TFileHelper { return err(errorTypeMismatch("file", systemPath)); } + async getFileWithRetry( + splitPath: SPF, + maxRetries = GET_FILE_RETRY_COUNT, + ): Promise> { + let lastResult = this.getFile(splitPath); + if (lastResult.isOk()) { + return lastResult; + } + + for (let retry = 0; retry < maxRetries; retry++) { + await delay(GET_FILE_RETRY_DELAY_MS); + lastResult = this.getFile(splitPath); + if (lastResult.isOk()) { + return lastResult; + } + } + + return lastResult; + } + async upsertMdFile( file: MdFileWithContentDto, ): Promise> { @@ -136,7 +159,10 @@ export class TFileHelper { // Both missing - poll for source (Obsidian index may lag after folder renames) if (fromResult.isErr() && toResult.isErr()) { - fromResult = await this.pollForFile(from, 10); + fromResult = await this.getFileWithRetry( + from, + GET_FILE_RETRY_COUNT, + ); } if (fromResult.isErr()) { @@ -171,21 +197,6 @@ export class TFileHelper { }); } - private async pollForFile( - splitPath: SPF, - maxRetries: number, - ): Promise> { - const retryDelayMs = 50; - for (let retry = 0; retry < maxRetries; retry++) { - await delay(retryDelayMs); - const result = this.getFile(splitPath); - if (result.isOk()) { - return result; - } - } - return this.getFile(splitPath); - } - private async handleTargetCollision( fromFile: TFile, toFile: TFile, @@ -308,7 +319,17 @@ export class TFileHelper { splitPath: SplitPathToMdFile, content: string, ): Promise> { - return this.getFile(splitPath).asyncAndThen((file) => + const immediateFileResult = this.getFile(splitPath); + if (immediateFileResult.isOk()) { + return this.tryVaultModify( + immediateFileResult.value, + content, + splitPath, + ); + } + + const fileResult = await this.getFileWithRetry(splitPath); + return fileResult.asyncAndThen((file) => this.tryVaultModify(file, content, splitPath), ); } @@ -336,7 +357,17 @@ export class TFileHelper { splitPath: SplitPathToMdFile; transform: Transform; }): Promise> { - return this.getFile(splitPath).asyncAndThen((file) => + const immediateFileResult = this.getFile(splitPath); + if (immediateFileResult.isOk()) { + return this.tryReadAndTransform( + immediateFileResult.value, + transform, + splitPath, + ); + } + + const fileResult = await this.getFileWithRetry(splitPath); + return fileResult.asyncAndThen((file) => this.tryReadAndTransform(file, transform, splitPath), ); } diff --git a/src/managers/obsidian/vault-action-manager/impl/vault-reader.ts b/src/managers/obsidian/vault-action-manager/impl/vault-reader.ts index 26d3fe19d..7f8de9a4f 100644 --- a/src/managers/obsidian/vault-action-manager/impl/vault-reader.ts +++ b/src/managers/obsidian/vault-action-manager/impl/vault-reader.ts @@ -38,8 +38,11 @@ export class VaultReader { .getContent() .mapErr((reason) => classifyReadContentError(reason)); } - return this.tfileHelper - .getFile(target) + const immediateFileResult = this.tfileHelper.getFile(target); + const fileResult = immediateFileResult.isOk() + ? immediateFileResult + : await this.tfileHelper.getFileWithRetry(target); + return fileResult .mapErr((reason) => classifyReadContentError(reason)) .asyncAndThen((file) => ResultAsync.fromPromise(this.vault.read(file), (error) => diff --git a/src/managers/overlay-manager/action-definitions/types.ts b/src/managers/overlay-manager/action-definitions/types.ts index 8a7f2d444..569fb142f 100644 --- a/src/managers/overlay-manager/action-definitions/types.ts +++ b/src/managers/overlay-manager/action-definitions/types.ts @@ -16,7 +16,9 @@ const OVERLAY_ACTION_KINDS = [ "GoToNextPage", ] as const satisfies readonly (keyof typeof CommandKind)[]; -export type OverlayActionKind = (typeof OVERLAY_ACTION_KINDS)[number]; +export const OverlayActionKindSchema = z.enum(OVERLAY_ACTION_KINDS); +export type OverlayActionKind = z.infer; +export const OverlayActionKind = OverlayActionKindSchema.enum; const OVERLAY_PLACEMENT_LITERALS = [ "AboveSelection", diff --git a/src/managers/overlay-manager/bottom-toolbar/bottom-toolbar.ts b/src/managers/overlay-manager/bottom-toolbar/bottom-toolbar.ts index 84e18c6a1..4e765dfcb 100644 --- a/src/managers/overlay-manager/bottom-toolbar/bottom-toolbar.ts +++ b/src/managers/overlay-manager/bottom-toolbar/bottom-toolbar.ts @@ -111,7 +111,7 @@ export function createBottomToolbar( // Update visibility of contextual buttons const buttons = toolbarEl.querySelectorAll(".tf-contextual-btn"); - for (const button of buttons) { + for (const button of Array.from(buttons)) { if (button instanceof HTMLElement) { button.style.display = hasSelection ? "" : "none"; } diff --git a/src/managers/overlay-manager/context-menu/context-menu.ts b/src/managers/overlay-manager/context-menu/context-menu.ts index a1bbce447..3dc590043 100644 --- a/src/managers/overlay-manager/context-menu/context-menu.ts +++ b/src/managers/overlay-manager/context-menu/context-menu.ts @@ -127,7 +127,7 @@ function handleEditorMenu( .setIcon("split") .onClick(() => { if (!commandExecutor) return; - void commandExecutor(CommandKind.MakeText); + void commandExecutor(CommandKind.SplitToPages); }), ); } diff --git a/src/managers/overlay-manager/index.ts b/src/managers/overlay-manager/index.ts index a61c78c0f..a63745781 100644 --- a/src/managers/overlay-manager/index.ts +++ b/src/managers/overlay-manager/index.ts @@ -15,8 +15,8 @@ export { computeNavActions, type PageNavMetadata, } from "./action-definitions/placement-utils"; +export type { ActionDefinition } from "./action-definitions/types"; export { - type ActionDefinition, OverlayActionKind, OverlayActionKindSchema, OverlayPlacement, diff --git a/src/managers/overlay-manager/toolbar-lifecycle/manager.ts b/src/managers/overlay-manager/toolbar-lifecycle/manager.ts index c316d58ef..82903ec77 100644 --- a/src/managers/overlay-manager/toolbar-lifecycle/manager.ts +++ b/src/managers/overlay-manager/toolbar-lifecycle/manager.ts @@ -2,7 +2,7 @@ * Manager for toolbar lifecycle - create, update, and destroy toolbars. */ -import type { MarkdownView } from "obsidian"; +import { MarkdownView } from "obsidian"; import { z } from "zod"; import { noteMetadataHelper } from "../../../stateless-helpers/note-metadata"; import { @@ -56,7 +56,9 @@ export function updateToolbarVisibility( const activeLeafIds = new Set(); for (const leaf of leaves) { - const file = leaf.view?.file; + const view = leaf.view; + if (!(view instanceof MarkdownView)) continue; + const file = view.file; if (!file || file.extension !== "md") continue; // Skip codex files - no toolbars for them @@ -67,11 +69,11 @@ export function updateToolbarVisibility( if (!leafId) continue; activeLeafIds.add(leafId); - const container = leaf.view.containerEl?.querySelector(".view-content"); + const container = view.containerEl?.querySelector(".view-content"); if (!container || !(container instanceof HTMLElement)) continue; // Read page metadata for nav button state - const pageMetadata = getPageMetadata(leaf.view as MarkdownView); + const pageMetadata = getPageMetadata(view); const navActions = computeNavActions(pageMetadata); // Combine base bottom actions with nav actions @@ -103,7 +105,7 @@ export function updateToolbarVisibility( // Create/update edge zones if (!edgeZones.has(leafId)) { const zones = createEdgeZones(container); - zones.attach(container, leaf.view as MarkdownView); + zones.attach(container, view); zones.setNavActions(navActions); edgeZones.set(leafId, zones); } else { diff --git a/src/prompt-smith/codegen/generated-promts/english/english/features-verb-prompt.ts b/src/prompt-smith/codegen/generated-promts/english/english/features-verb-prompt.ts index 7b0fef53b..8e88af6f1 100644 --- a/src/prompt-smith/codegen/generated-promts/english/english/features-verb-prompt.ts +++ b/src/prompt-smith/codegen/generated-promts/english/english/features-verb-prompt.ts @@ -13,7 +13,7 @@ You receive: - context: sentence where the word occurred Return: -- conjugation: one of "Irregular" | "Rregular" +- conjugation: one of "Irregular" | "Regular" - valency: - separability: one of "Separable" | "Inseparable" | "None" - reflexivity: one of "NonReflexive" | "ReflexiveOnly" | "OptionalReflexive" @@ -32,7 +32,7 @@ Rules: {"context":"Can you open the door, please?","word":"open up"} -{"conjugation":"Rregular","valency":{"reflexivity":"NonReflexive","separability":"Separable"}} +{"conjugation":"Regular","valency":{"reflexivity":"NonReflexive","separability":"Separable"}} @@ -41,7 +41,7 @@ Rules: {"context":"She relies on her team.","word":"rely"} -{"conjugation":"Rregular","valency":{"governedPreposition":"on","reflexivity":"NonReflexive","separability":"None"}} +{"conjugation":"Regular","valency":{"governedPreposition":"on","reflexivity":"NonReflexive","separability":"None"}} `; diff --git a/src/prompt-smith/codegen/generated-promts/german/english/features-verb-prompt.ts b/src/prompt-smith/codegen/generated-promts/german/english/features-verb-prompt.ts index 1f5fc248e..5a31f3af4 100644 --- a/src/prompt-smith/codegen/generated-promts/german/english/features-verb-prompt.ts +++ b/src/prompt-smith/codegen/generated-promts/german/english/features-verb-prompt.ts @@ -13,7 +13,7 @@ You receive: - context: sentence where the word occurred Return: -- conjugation: one of "Irregular" | "Rregular" +- conjugation: one of "Irregular" | "Regular" - valency: - separability: one of "Separable" | "Inseparable" | "None" - reflexivity: one of "NonReflexive" | "ReflexiveOnly" | "OptionalReflexive" @@ -32,7 +32,7 @@ Rules: {"context":"Kannst du bitte die Tür aufmachen?","word":"aufmachen"} -{"conjugation":"Rregular","valency":{"reflexivity":"NonReflexive","separability":"Separable"}} +{"conjugation":"Regular","valency":{"reflexivity":"NonReflexive","separability":"Separable"}} @@ -41,7 +41,7 @@ Rules: {"context":"Ich kümmere mich um die Kinder.","word":"sich kümmern"} -{"conjugation":"Rregular","valency":{"governedPreposition":"um","reflexivity":"ReflexiveOnly","separability":"None"}} +{"conjugation":"Regular","valency":{"governedPreposition":"um","reflexivity":"ReflexiveOnly","separability":"None"}} `; diff --git a/src/prompt-smith/prompt-parts/english/english/features-verb/examples/to-test.ts b/src/prompt-smith/prompt-parts/english/english/features-verb/examples/to-test.ts index ff74f5a68..fb4e9bbcb 100644 --- a/src/prompt-smith/prompt-parts/english/english/features-verb/examples/to-test.ts +++ b/src/prompt-smith/prompt-parts/english/english/features-verb/examples/to-test.ts @@ -7,7 +7,7 @@ export const testExamples = [ word: "open up", }, output: { - conjugation: "Rregular", + conjugation: "Regular", valency: { reflexivity: "NonReflexive", separability: "Separable", diff --git a/src/prompt-smith/prompt-parts/english/english/features-verb/examples/to-use.ts b/src/prompt-smith/prompt-parts/english/english/features-verb/examples/to-use.ts index 55afcfdee..bdce2c920 100644 --- a/src/prompt-smith/prompt-parts/english/english/features-verb/examples/to-use.ts +++ b/src/prompt-smith/prompt-parts/english/english/features-verb/examples/to-use.ts @@ -7,7 +7,7 @@ export const examples = [ word: "open up", }, output: { - conjugation: "Rregular", + conjugation: "Regular", valency: { reflexivity: "NonReflexive", separability: "Separable", @@ -20,7 +20,7 @@ export const examples = [ word: "rely", }, output: { - conjugation: "Rregular", + conjugation: "Regular", valency: { governedPreposition: "on", reflexivity: "NonReflexive", diff --git a/src/prompt-smith/prompt-parts/english/english/features-verb/task-description.ts b/src/prompt-smith/prompt-parts/english/english/features-verb/task-description.ts index 3f0fb30fc..a65545dc4 100644 --- a/src/prompt-smith/prompt-parts/english/english/features-verb/task-description.ts +++ b/src/prompt-smith/prompt-parts/english/english/features-verb/task-description.ts @@ -5,7 +5,7 @@ You receive: - context: sentence where the word occurred Return: -- conjugation: one of "Irregular" | "Rregular" +- conjugation: one of "Irregular" | "Regular" - valency: - separability: one of "Separable" | "Inseparable" | "None" - reflexivity: one of "NonReflexive" | "ReflexiveOnly" | "OptionalReflexive" diff --git a/src/prompt-smith/prompt-parts/german/english/features-verb/examples/to-test.ts b/src/prompt-smith/prompt-parts/german/english/features-verb/examples/to-test.ts index 74a573574..47009b64b 100644 --- a/src/prompt-smith/prompt-parts/german/english/features-verb/examples/to-test.ts +++ b/src/prompt-smith/prompt-parts/german/english/features-verb/examples/to-test.ts @@ -7,7 +7,7 @@ export const testExamples = [ word: "aufmachen", }, output: { - conjugation: "Rregular", + conjugation: "Regular", valency: { reflexivity: "NonReflexive", separability: "Separable", diff --git a/src/prompt-smith/prompt-parts/german/english/features-verb/examples/to-use.ts b/src/prompt-smith/prompt-parts/german/english/features-verb/examples/to-use.ts index 44c3fd89b..e0cb92f8c 100644 --- a/src/prompt-smith/prompt-parts/german/english/features-verb/examples/to-use.ts +++ b/src/prompt-smith/prompt-parts/german/english/features-verb/examples/to-use.ts @@ -7,7 +7,7 @@ export const examples = [ word: "aufmachen", }, output: { - conjugation: "Rregular", + conjugation: "Regular", valency: { reflexivity: "NonReflexive", separability: "Separable", @@ -20,7 +20,7 @@ export const examples = [ word: "sich kümmern", }, output: { - conjugation: "Rregular", + conjugation: "Regular", valency: { governedPreposition: "um", reflexivity: "ReflexiveOnly", diff --git a/src/prompt-smith/prompt-parts/german/english/features-verb/task-description.ts b/src/prompt-smith/prompt-parts/german/english/features-verb/task-description.ts index 9171ee968..dc1b70b09 100644 --- a/src/prompt-smith/prompt-parts/german/english/features-verb/task-description.ts +++ b/src/prompt-smith/prompt-parts/german/english/features-verb/task-description.ts @@ -5,7 +5,7 @@ You receive: - context: sentence where the word occurred Return: -- conjugation: one of "Irregular" | "Rregular" +- conjugation: one of "Irregular" | "Regular" - valency: - separability: one of "Separable" | "Inseparable" | "None" - reflexivity: one of "NonReflexive" | "ReflexiveOnly" | "OptionalReflexive" diff --git a/src/prompt-smith/schemas/lemma.ts b/src/prompt-smith/schemas/lemma.ts index 01ec3d6fb..1ba241a38 100644 --- a/src/prompt-smith/schemas/lemma.ts +++ b/src/prompt-smith/schemas/lemma.ts @@ -7,8 +7,6 @@ const userInputSchema = z.object({ }); // Runtime cutover: Lemma validates against the minimal classifier contract. -// DeLemmaResultSchema accepts legacy alias keys (pos / phrasemeKind) and normalizes -// them into canonical posLikeKind output for downstream consumers. // Enrichment/feature contracts are exported below for Generate routing. const agentOutputSchema = DeLemmaResultSchema; diff --git a/src/stateless-helpers/api-service.ts b/src/stateless-helpers/api-service.ts index be47cf47e..b45cf8458 100644 --- a/src/stateless-helpers/api-service.ts +++ b/src/stateless-helpers/api-service.ts @@ -35,10 +35,19 @@ function normalizeHeaders(initHeaders?: HeadersInit): Record { // 7 days const TTL_SECONDS = 604800; +const GENERATE_TIMEOUT_MS = 45_000; + +class ApiTimeoutError extends Error { + constructor(timeoutMs: number) { + super(`API request timed out after ${timeoutMs}ms`); + this.name = "ApiTimeoutError"; + } +} export type ApiServiceError = { reason: string }; function isRetryableApiError(error: unknown): boolean { + if (error instanceof ApiTimeoutError) return true; if (error instanceof APIConnectionError) return true; if (error instanceof APIError) { return error.status === 429 || (error.status ?? 0) >= 500; @@ -197,6 +206,25 @@ export class ApiService { return null; } + private async withTimeout( + promise: Promise, + timeoutMs: number, + ): Promise { + let timeoutHandle: ReturnType | null = null; + try { + const timeoutPromise = new Promise((_, reject) => { + timeoutHandle = setTimeout(() => { + reject(new ApiTimeoutError(timeoutMs)); + }, timeoutMs); + }); + return await Promise.race([promise, timeoutPromise]); + } finally { + if (timeoutHandle) { + clearTimeout(timeoutHandle); + } + } + } + generate({ systemPrompt, userInput, @@ -237,28 +265,31 @@ export class ApiService { return withRetry( async () => { - const completion = await client.chat.completions.parse({ - messages, - model, - // Type assertion needed due to Zod version mismatch between our deps and OpenAI SDK - response_format: zodResponseFormat( - schema as unknown as Parameters< - typeof zodResponseFormat - >[0], - "data", - ), - temperature: 0, - top_p: 0.95, - ...(cachedId - ? { - extra_body: { - google: { - cached_content: cachedId, + const completion = await this.withTimeout( + client.chat.completions.parse({ + messages, + model, + // Type assertion needed due to Zod version mismatch between our deps and OpenAI SDK + response_format: zodResponseFormat( + schema as unknown as Parameters< + typeof zodResponseFormat + >[0], + "data", + ), + temperature: 0, + top_p: 0.95, + ...(cachedId + ? { + extra_body: { + google: { + cached_content: cachedId, + }, }, - }, - } - : {}), - }); + } + : {}), + }), + GENERATE_TIMEOUT_MS, + ); const parsed = completion.choices?.[0]?.message?.parsed; diff --git a/src/stateless-helpers/note-metadata/internal/frontmatter.ts b/src/stateless-helpers/note-metadata/internal/frontmatter.ts index f44271d90..cc3320883 100644 --- a/src/stateless-helpers/note-metadata/internal/frontmatter.ts +++ b/src/stateless-helpers/note-metadata/internal/frontmatter.ts @@ -162,8 +162,8 @@ export function stripOnlyFrontmatter(content: string): string { export function frontmatterToInternal( fm: Record, ): ScrollMetadataWithImport { - // Detect status from common field names and values - const statusField = fm.status ?? fm.completion ?? fm.state; + // Status is derived only from canonical `status` key. + const statusField = fm.status; const isDone = statusField === "done" || statusField === TreeNodeStatus.Done || diff --git a/src/stateless-helpers/note-metadata/internal/json-section.ts b/src/stateless-helpers/note-metadata/internal/json-section.ts index b236b6ca8..891152a97 100644 --- a/src/stateless-helpers/note-metadata/internal/json-section.ts +++ b/src/stateless-helpers/note-metadata/internal/json-section.ts @@ -15,9 +15,8 @@ const SECTION = "section"; const reEscape = (s: string) => s.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"); // Pattern to match metadata section (captures JSON content) -// Matches both id={...} (legacy) and id="..." (new) formats const PATTERN = new RegExp( - `\\n*<${SECTION}\\s+id=[\\{"]${reEscape(META_SECTION_ID)}[\\}"]>([\\s\\S]*?)<\\/${SECTION}>\\n*`, + `\\n*<${SECTION}\\s+id="${reEscape(META_SECTION_ID)}">([\\s\\S]*?)<\\/${SECTION}>\\n*`, ); /** Find the start index of the metadata section (excluding preceding whitespace), or null if none. */ diff --git a/src/stateless-helpers/note-metadata/internal/migration.ts b/src/stateless-helpers/note-metadata/internal/migration.ts deleted file mode 100644 index 2c39802bd..000000000 --- a/src/stateless-helpers/note-metadata/internal/migration.ts +++ /dev/null @@ -1,72 +0,0 @@ -/** - * Migration transforms for converting between metadata formats. - * Only imported by build-initial-actions.ts for initialization. - */ - -import type { Transform } from "../../../managers/obsidian/vault-action-manager/types/vault-action"; -import { - frontmatterToInternal, - internalToFrontmatter, - parseFrontmatter, - type ScrollMetadataWithImport, - stripOnlyFrontmatter, -} from "./frontmatter"; -import { stripJsonSection, writeJsonSection } from "./json-section"; - -// ─── Types ─── - -export type MigrateFrontmatterOptions = { - /** Whether to strip YAML frontmatter after conversion. Default: true */ - stripYaml?: boolean; -}; - -// ─── Migrations ─── - -/** - * Create transform that migrates YAML frontmatter to internal JSON format. - * @param options.stripYaml - If true (default), removes YAML frontmatter. If false, keeps it. - */ -export function migrateFrontmatter( - options?: MigrateFrontmatterOptions, -): Transform { - const stripYaml = options?.stripYaml ?? true; - - return (content: string) => { - const fm = parseFrontmatter(content); - if (!fm) return content; - - const baseContent = stripYaml ? stripOnlyFrontmatter(content) : content; - const meta = frontmatterToInternal(fm); - return writeJsonSection(meta)(baseContent); - }; -} - -/** - * Create transform that converts internal JSON metadata to YAML frontmatter. - * Strips internal section and prepends YAML frontmatter. - */ -export function migrateToFrontmatter( - meta: ScrollMetadataWithImport, -): Transform { - return (content: string) => { - const stripped = stripJsonSection(content); - const withoutFm = stripOnlyFrontmatter(stripped); - const yaml = internalToFrontmatter(meta); - return `${yaml}\n${withoutFm}`; - }; -} - -/** - * Create transform that adds YAML frontmatter with given status. - * Used when file has no metadata at all. - */ -export function addFrontmatter(meta: ScrollMetadataWithImport): Transform { - return (content: string) => { - const withoutFm = stripOnlyFrontmatter(content); - const yaml = internalToFrontmatter(meta); - return `${yaml}\n${withoutFm}`; - }; -} - -// Re-export types needed by consumers -export type { ScrollMetadataWithImport }; diff --git a/src/stateless-helpers/wikilink.ts b/src/stateless-helpers/wikilink.ts index 751c2545b..e23337772 100644 --- a/src/stateless-helpers/wikilink.ts +++ b/src/stateless-helpers/wikilink.ts @@ -23,6 +23,104 @@ export type ParsedWikilinkRange = ParsedWikilink & { /** Regex to match wikilinks: [[target]] or [[target|alias]] */ const WIKILINK_REGEX = /\[\[([^\]|]+)(?:\|([^\]]+))?\]\]/g; +function looksLikeVaultPath(target: string): boolean { + const normalized = target.replace(/\\/g, "/"); + if ( + normalized.startsWith("Worter/") || + normalized.startsWith("Library/") || + normalized.startsWith("/Worter/") || + normalized.startsWith("/Library/") + ) { + return true; + } + return false; +} + +function stripWikilinkWrapper(raw: string): string { + const trimmed = raw.trim(); + if (!(trimmed.startsWith("[[") && trimmed.endsWith("]]"))) { + return trimmed; + } + const inner = trimmed.slice(2, -2); + const pipeIndex = inner.indexOf("|"); + return pipeIndex >= 0 ? inner.slice(0, pipeIndex).trim() : inner.trim(); +} + +function extractPathBasename(pathLikeTarget: string): string { + const normalized = pathLikeTarget.replace(/\\/g, "/"); + const parts = normalized.split("/").filter((part) => part.length > 0); + const basename = parts[parts.length - 1]; + return basename ?? ""; +} + +function splitAnchor(target: string): { + anchor: string; + baseTarget: string; +} { + const anchorIndex = target.indexOf("#"); + if (anchorIndex < 0) { + return { + anchor: "", + baseTarget: target, + }; + } + return { + anchor: target.slice(anchorIndex), + baseTarget: target.slice(0, anchorIndex), + }; +} + +/** + * Normalize a raw link target into a clean, user-facing wikilink target. + * Examples: + * - "Worter/de/.../Fahren" -> "Fahren" + * - "[[Library/de/prefix/auf-prefix-de|auf]]" -> "auf-prefix-de" + * - "Haus" -> "Haus" + */ +function normalizeLinkTarget(rawTarget: string): string { + const stripped = stripWikilinkWrapper(rawTarget).trim(); + if (stripped.length === 0) { + return ""; + } + + const { anchor, baseTarget } = splitAnchor(stripped); + const normalizedBase = baseTarget.trim(); + if (normalizedBase.length === 0) { + return ""; + } + + const shouldFlatten = looksLikeVaultPath(normalizedBase); + const candidate = shouldFlatten + ? extractPathBasename(normalizedBase) + : normalizedBase; + + const candidateWithoutExt = + shouldFlatten && candidate.toLowerCase().endsWith(".md") + ? candidate.slice(0, -3).trim() + : candidate.trim(); + if (candidateWithoutExt.length === 0) { + return ""; + } + + return `${candidateWithoutExt}${anchor}`; +} + +/** + * Rewrite wikilinks in text so each target is normalized. + */ +function normalizeWikilinkTargetsInText(text: string): string { + return text.replace( + WIKILINK_REGEX, + (_fullMatch: string, target: string, alias?: string) => { + const normalized = normalizeLinkTarget(target); + if (normalized.length === 0) { + return alias ? `[[${target}|${alias}]]` : `[[${target}]]`; + } + return alias ? `[[${normalized}|${alias}]]` : `[[${normalized}]]`; + }, + ); +} + /** * Parse all wikilinks from text with offsets. * @param text - Text containing wikilinks @@ -133,17 +231,17 @@ function createMatcher( return () => { const match = regex.exec(text); if (!match) return null; - return { alias: match[2], target: match[1]! }; + const target = match[1]; + if (typeof target !== "string") { + return null; + } + return { + alias: typeof match[2] === "string" ? match[2] : undefined, + target, + }; }; } -/** - * Normalize a wikilink target for case-insensitive comparison. - */ -function normalizeTarget(target: string): string { - return target.trim().toLowerCase(); -} - /** * Wikilink helper object with grouped functions. */ @@ -153,7 +251,8 @@ export const wikilinkHelper = { findByTarget, findEnclosingByOffset, matchesPattern, - normalizeTarget, + normalizeLinkTarget, + normalizeWikilinkTargetsInText, parse, parseWithRanges, }; diff --git a/src/utils/delimiter.ts b/src/utils/delimiter.ts index 16f4a06fc..6d11c8975 100644 --- a/src/utils/delimiter.ts +++ b/src/utils/delimiter.ts @@ -29,41 +29,3 @@ export function buildFlexibleDelimiterPattern( const escapedSymbol = escapeRegex(config.symbol); return new RegExp(`\\s*${escapedSymbol}\\s*`); } - -/** - * Migrate old string delimiter format to new config format. - * Parses spacing from the string to determine padding. - */ -export function migrateStringDelimiter( - oldDelimiter: string, -): SuffixDelimiterConfig { - const trimmed = oldDelimiter.trim(); - if (trimmed.length === 0) { - // Fallback to default - return { padded: false, symbol: "-" }; - } - - // If there's spacing on either side, consider it padded - const padded = oldDelimiter.startsWith(" ") || oldDelimiter.endsWith(" "); - - return { - padded, - symbol: trimmed, - }; -} - -/** - * Check if a value is a valid SuffixDelimiterConfig object. - */ -export function isSuffixDelimiterConfig( - value: unknown, -): value is SuffixDelimiterConfig { - return ( - typeof value === "object" && - value !== null && - "symbol" in value && - typeof (value as SuffixDelimiterConfig).symbol === "string" && - "padded" in value && - typeof (value as SuffixDelimiterConfig).padded === "boolean" - ); -} diff --git a/tests/cli-e2e/textfresser/edge-case-results.md b/tests/cli-e2e/textfresser/edge-case-results.md index a8bbdb820..74545c934 100644 --- a/tests/cli-e2e/textfresser/edge-case-results.md +++ b/tests/cli-e2e/textfresser/edge-case-results.md @@ -21,12 +21,12 @@ | H2-D | fliegen | ✅ | ✅ | ✅ | | | H2-E | Lauf | ✅ | ✅ | ✅ | | | H2-F | laufen | ✅ | ✅ | ✅ | | -| V1-A | macht | ✅ | ✅ | ✅ | | -| V1-B | fängst | ✅ | ✅ | ❌ | | -| V1-C | kauft | ✅ | ✅ | ❌ | | -| V1-D | Pass | ✅ | ✅ | ❌ | | -| V1-E | gibt | ✅ | ✅ | ❌ | | -| V1-F | hört | ✅ | ✅ | ❌ | | +| SV-A | macht | ✅ | ✅ | ✅ | | +| SV-B | fängst | ✅ | ✅ | ❌ | | +| SV-C | kauft | ✅ | ✅ | ❌ | | +| SV-D | Pass | ✅ | ✅ | ❌ | | +| SV-E | gibt | ✅ | ✅ | ❌ | | +| SV-F | hört | ✅ | ✅ | ❌ | | | PH1-A | Auf keinen Fall | ✅ | ✅ | ❌ | | | PH1-B | Hals über Kopf | ✅ | ✅ | ❌ | | | PH1-C | in Ordnung | ✅ | ✅ | ✅ | | @@ -320,9 +320,9 @@ Die Kinder [[laufen]] schnell im Park. ^h2f --- -### V1: Separable Verbs +### SV: Separable Verbs -#### V1-A: "macht" — Should detect "aufmachen" separable verb +#### SV-A: "macht" — Should detect "aufmachen" separable verb **Lemma**: OK @@ -343,7 +343,7 @@ Er [[macht]] die Tür auf. ^v1a --- -#### V1-B: "fängst" — "anfangen", inflected stem + detached prefix +#### SV-B: "fängst" — "anfangen", inflected stem + detached prefix **Lemma**: OK @@ -362,7 +362,7 @@ Wann [[fängst]] du damit an? ^v1b --- -#### V1-C: "kauft" — "einkaufen" +#### SV-C: "kauft" — "einkaufen" **Lemma**: OK @@ -381,7 +381,7 @@ Sie [[kauft]] im Supermarkt ein. ^v1c --- -#### V1-D: "Pass" — "aufpassen" imperative — TWO "auf" in sentence! +#### SV-D: "Pass" — "aufpassen" imperative — TWO "auf" in sentence! **Lemma**: OK @@ -400,7 +400,7 @@ Sie [[kauft]] im Supermarkt ein. ^v1c --- -#### V1-E: "gibt" — "zurückgeben" +#### SV-E: "gibt" — "zurückgeben" **Lemma**: OK @@ -419,7 +419,7 @@ Er [[gibt]] das Buch morgen zurück. ^v1e --- -#### V1-F: "hört" — "aufhören" +#### SV-F: "hört" — "aufhören" **Lemma**: OK diff --git a/tests/cli-e2e/textfresser/edge-case-runner.ts b/tests/cli-e2e/textfresser/edge-case-runner.ts index 517a01a6b..706aaffe0 100644 --- a/tests/cli-e2e/textfresser/edge-case-runner.ts +++ b/tests/cli-e2e/textfresser/edge-case-runner.ts @@ -123,53 +123,53 @@ const TEST_CASES: TestCase[] = [ id: "H2-F", surface: "laufen", }, - // ── V1: Separable Verbs ── + // ── SV: Separable Verbs ── { content: "Er macht die Tür auf. ^v1a", description: 'Should detect "aufmachen" separable verb', - filePath: `${TEST_RUNS_ROOT}/V1/V1-A-aufmachen.md`, - group: "V1", - id: "V1-A", + filePath: `${TEST_RUNS_ROOT}/SV/SV-A-aufmachen.md`, + group: "SV", + id: "SV-A", surface: "macht", }, { content: "Wann fängst du damit an? ^v1b", description: '"anfangen", inflected stem + detached prefix', - filePath: `${TEST_RUNS_ROOT}/V1/V1-B-anfangen.md`, - group: "V1", - id: "V1-B", + filePath: `${TEST_RUNS_ROOT}/SV/SV-B-anfangen.md`, + group: "SV", + id: "SV-B", surface: "fängst", }, { content: "Sie kauft im Supermarkt ein. ^v1c", description: '"einkaufen"', - filePath: `${TEST_RUNS_ROOT}/V1/V1-C-einkaufen.md`, - group: "V1", - id: "V1-C", + filePath: `${TEST_RUNS_ROOT}/SV/SV-C-einkaufen.md`, + group: "SV", + id: "SV-C", surface: "kauft", }, { content: "Pass bitte auf die Kinder auf! ^v1d", description: '"aufpassen" imperative — TWO "auf" in sentence!', - filePath: `${TEST_RUNS_ROOT}/V1/V1-D-aufpassen.md`, - group: "V1", - id: "V1-D", + filePath: `${TEST_RUNS_ROOT}/SV/SV-D-aufpassen.md`, + group: "SV", + id: "SV-D", surface: "Pass", }, { content: "Er gibt das Buch morgen zurück. ^v1e", description: '"zurückgeben"', - filePath: `${TEST_RUNS_ROOT}/V1/V1-E-zurueckgeben.md`, - group: "V1", - id: "V1-E", + filePath: `${TEST_RUNS_ROOT}/SV/SV-E-zurueckgeben.md`, + group: "SV", + id: "SV-E", surface: "gibt", }, { content: "Sie hört mit dem Rauchen auf. ^v1f", description: '"aufhören"', - filePath: `${TEST_RUNS_ROOT}/V1/V1-F-aufhoeren.md`, - group: "V1", - id: "V1-F", + filePath: `${TEST_RUNS_ROOT}/SV/SV-F-aufhoeren.md`, + group: "SV", + id: "SV-F", surface: "hört", }, // ── PH1: Phrasems ── @@ -327,7 +327,7 @@ async function main(): Promise { await deleteAllUnder("Library/de"); await deleteAllUnder(`${TEST_RUNS_ROOT}/H1`); await deleteAllUnder(`${TEST_RUNS_ROOT}/H2`); - await deleteAllUnder(`${TEST_RUNS_ROOT}/V1`); + await deleteAllUnder(`${TEST_RUNS_ROOT}/SV`); await deleteAllUnder(`${TEST_RUNS_ROOT}/PH1`); await deleteAllUnder(`${TEST_RUNS_ROOT}/ADJ1`); await deletePath(`${TEST_RUNS_ROOT}/A1/A1-A.md`); @@ -350,7 +350,7 @@ async function main(): Promise { const results: TestResult[] = []; // Group test cases by group to run in order - const groups = ["H1", "H2", "V1", "PH1", "ADJ1"]; + const groups = ["H1", "H2", "SV", "PH1", "ADJ1"]; for (const group of groups) { const groupCases = TEST_CASES.filter((tc) => tc.group === group); @@ -452,14 +452,14 @@ async function main(): Promise { function generateReport(results: TestResult[]): string { const now = new Date().toISOString().split("T")[0]; - const groups = ["H1", "H2", "V1", "PH1", "ADJ1"]; + const groups = ["H1", "H2", "SV", "PH1", "ADJ1"]; const groupNames: Record = { ADJ1: "Adjective Forms & Propagation", H1: "Homonym Nouns", H2: "Cross-POS", PH1: "Phrasems / Multi-Word Expressions", - V1: "Separable Verbs", + SV: "Separable Verbs", }; let md = `# Textfresser Edge Case Testing — Book of Work\n\n`; diff --git a/tests/e2e/rumbook/textferesser/lemma/000-one-noun-only.md b/tests/e2e/rumbook/textferesser/lemma/000-one-noun-only.md deleted file mode 100644 index ddc65e77d..000000000 --- a/tests/e2e/rumbook/textferesser/lemma/000-one-noun-only.md +++ /dev/null @@ -1,79 +0,0 @@ -# 000 One Noun Only (CLI Runbook) - -Goal: copy one source page into the CLI test vault, select one noun (`Manne`), run `Lemma`, and verify output. - -## Hard Stop Rules - -- Run `Lemma` only on plain text, never on already linked text. -- If the token is already inside `[[...]]`, reset the note first. -- Run `Lemma` once per token in a fresh source file state. -- If you see nested links like `[[[[...]]]]`, stop and reset source before continuing. -- If you see numbered sense targets (`1_...`, `2_...`, `3_...`) in a run that should be clean, stop and clean prior matching entries in `Worter/` before retry. - -Quick pre-check before each run: - -```bash -"$OBSIDIAN" vault="$VAULT" read path="$HEALED_PATH" | rg -n "\\[\\[\\[\\[|\\[\\[Manne\\]\\]|Manne" | head -20 -``` - -Expected before first run: plain `Manne` (not `[[Manne]]`, not `[[[[Manne]]]]`). - -## 0) Variables - -```bash -OBSIDIAN="/Applications/Obsidian.app/Contents/MacOS/Obsidian" -VAULT="cli-e2e-test-vault" -SOURCE="/Users/annagorelova/work/Textfresser_vault/Library/Text/Märchen/Aschenputtel/Aschenputtel_Page_000 ;; Aschenputtel ;; Märchen ;; Text.md" -RAW_DEST="/Users/annagorelova/work/obsidian/cli-e2e-test-vault/Library/Text/Märchen/Aschenputtel/Aschenputtel_Page_000 ;; Aschenputtel ;; Märchen ;; Text.md" -HEALED_PATH="Library/Text/Märchen/Aschenputtel/Aschenputtel_Page_000 ;; Aschenputtel ;; Märchen ;; Text-Aschenputtel-Märchen-Text.md" -``` - -## 1) Copy content into test vault - -```bash -mkdir -p "/Users/annagorelova/work/obsidian/cli-e2e-test-vault/Library/Text/Märchen/Aschenputtel" -cat "$SOURCE" > "$RAW_DEST" -``` - -## 2) Ensure vault is reachable via Obsidian CLI - -```bash -"$OBSIDIAN" vault="$VAULT" files | head -20 -``` - -## 3) Confirm final path after healing - -The plugin may heal the copied filename immediately. - -```bash -"$OBSIDIAN" vault="$VAULT" files folder="Library" ext=md | rg "Aschenputtel|Märchen|Text" -``` - -Use `HEALED_PATH` for command execution. - -## 4) Open file, select `Manne`, invoke `Lemma` (CLI only) - -```bash -CODE="(async()=>{const file=app.vault.getAbstractFileByPath('$HEALED_PATH');if(!file)throw new Error('file not found');const leaf=app.workspace.getMostRecentLeaf()??app.workspace.getLeaf(true);await leaf.openFile(file,{active:true});const view=leaf.view;if(view&&typeof view.getMode==='function'&&typeof view.setMode==='function'&&view.getMode()!=='source'){await view.setMode('source')}const editor=(view&&'editor' in view&&view.editor)?view.editor:app.workspace.activeEditor?.editor;if(!editor)throw new Error('no editor');const content=editor.getValue();const idx=content.indexOf('Manne');if(idx===-1)throw new Error('Manne not found');const toPos=(offset)=>{const s=content.slice(0,offset).split('\\n');return {line:s.length-1,ch:s[s.length-1].length}};editor.setSelection(toPos(idx),toPos(idx+5));if(typeof editor.focus==='function'){editor.focus()}const ok=app.commands.executeCommandById('cbcr-text-eater-de:lemma');if(!ok)throw new Error('command failed');return 'ok';})()" -"$OBSIDIAN" vault="$VAULT" eval "code=$CODE" -``` - -## 5) Wait for plugin idle - -```bash -"$OBSIDIAN" vault="$VAULT" eval "code=(async()=>{await app.plugins.plugins['cbcr-text-eater-de'].whenIdle();return 'idle'})()" -``` - -## 6) Verify source rewrite + dictionary output - -Check rewritten source line: - -```bash -"$OBSIDIAN" vault="$VAULT" read path="$HEALED_PATH" | rg -n "\\[\\[.*Manne|Manne" | head -5 -``` - -Check created entry files: - -```bash -"$OBSIDIAN" vault="$VAULT" files folder="Worter" ext=md | rg -n "Manne|Mann" -``` diff --git a/tests/unit/behaviors/wikilink-completion-behavior.test.ts b/tests/unit/behaviors/wikilink-completion-behavior.test.ts new file mode 100644 index 000000000..923933fc6 --- /dev/null +++ b/tests/unit/behaviors/wikilink-completion-behavior.test.ts @@ -0,0 +1,71 @@ +import { describe, expect, it } from "bun:test"; +import type { EditorView } from "@codemirror/view"; +import { createWikilinkCompletionHandler } from "../../../src/managers/obsidian/behavior-manager/wikilink-complition-behavior"; +import type { WikilinkPayload } from "../../../src/managers/obsidian/user-event-interceptor/events/wikilink/payload"; + +type WikilinkLibrarian = Parameters[0]; + +function makePayload(linkContent: string): WikilinkPayload { + return { + closePos: linkContent.length, + kind: "WikilinkCompleted", + linkContent, + view: {} as EditorView, + }; +} + +function makeContext() { + const app = { + metadataCache: { + getFirstLinkpathDest: () => null, + }, + workspace: { + getActiveFile: () => ({ path: "Books/lesson.md" }), + }, + }; + + return { + app, + vam: {}, + }; +} + +describe("wikilink completion behavior", () => { + it("passes through when corename lookup is ambiguous", () => { + const librarian = { + findMatchingLeavesByCoreName: () => [ + { basename: "die-pronoun-de", pathParts: ["Library", "de", "pronoun"] }, + { basename: "die-article-de", pathParts: ["Library", "de", "article"] }, + ], + resolveWikilinkAlias: () => null, + } as unknown as WikilinkLibrarian; + + const handler = createWikilinkCompletionHandler(librarian); + const result = handler.handle(makePayload("die"), makeContext()); + expect(result).toEqual({ outcome: "Passthrough" }); + }); + + it("keeps single-match auto-resolution behavior", () => { + const librarian = { + findMatchingLeavesByCoreName: () => [ + { basename: "die-pronoun-de", pathParts: ["Library", "de", "pronoun"] }, + ], + resolveWikilinkAlias: () => null, + } as unknown as WikilinkLibrarian; + + const handler = createWikilinkCompletionHandler(librarian); + const result = handler.handle(makePayload("die"), makeContext()); + + expect(result).toEqual({ + data: { + aliasToInsert: "die", + closePos: 3, + kind: "WikilinkCompleted", + linkContent: "die", + resolvedTarget: "die-pronoun-de", + view: expect.any(Object), + }, + outcome: "Modified", + }); + }); +}); diff --git a/tests/unit/linguistics/de-generate-contracts.test.ts b/tests/unit/linguistics/de-generate-contracts.test.ts index b17ca9a94..9e6150e4a 100644 --- a/tests/unit/linguistics/de-generate-contracts.test.ts +++ b/tests/unit/linguistics/de-generate-contracts.test.ts @@ -181,7 +181,7 @@ describe("De generate contracts", () => { it("accepts structured verb features output", () => { const output = DeFeaturesOutputSchema.safeParse({ - conjugation: "Rregular", + conjugation: "Regular", valency: { reflexivity: "NonReflexive", separability: "Separable", diff --git a/tests/unit/linguistics/de-lemma-result-schema.test.ts b/tests/unit/linguistics/de-lemma-result-schema.test.ts index 74e4b98ea..f2fac9138 100644 --- a/tests/unit/linguistics/de-lemma-result-schema.test.ts +++ b/tests/unit/linguistics/de-lemma-result-schema.test.ts @@ -24,7 +24,7 @@ describe("DeLemmaResultSchema", () => { expect(result.success).toBe(true); }); - it("accepts legacy Lexem `pos` and normalizes to posLikeKind", () => { + it("rejects legacy Lexem `pos` alias", () => { const result = DeLemmaResultSchema.safeParse({ lemma: "Haus", linguisticUnit: "Lexem", @@ -32,12 +32,10 @@ describe("DeLemmaResultSchema", () => { surfaceKind: "Lemma", }); - expect(result.success).toBe(true); - if (!result.success) return; - expect(result.data.posLikeKind).toBe("Noun"); + expect(result.success).toBe(false); }); - it("accepts legacy Phrasem `phrasemeKind` and normalizes to posLikeKind", () => { + it("rejects legacy Phrasem `phrasemeKind` alias", () => { const result = DeLemmaResultSchema.safeParse({ lemma: "auf jeden Fall", linguisticUnit: "Phrasem", @@ -45,9 +43,7 @@ describe("DeLemmaResultSchema", () => { surfaceKind: "Lemma", }); - expect(result.success).toBe(true); - if (!result.success) return; - expect(result.data.posLikeKind).toBe("DiscourseFormula"); + expect(result.success).toBe(false); }); it("rejects Morphem", () => { @@ -61,30 +57,6 @@ describe("DeLemmaResultSchema", () => { expect(result.success).toBe(false); }); - it("rejects Lexem with conflicting posLikeKind and pos", () => { - const result = DeLemmaResultSchema.safeParse({ - lemma: "run", - linguisticUnit: "Lexem", - pos: "Verb", - posLikeKind: "Noun", - surfaceKind: "Lemma", - }); - - expect(result.success).toBe(false); - }); - - it("rejects Phrasem with conflicting posLikeKind and phrasemeKind", () => { - const result = DeLemmaResultSchema.safeParse({ - lemma: "auf jeden Fall", - linguisticUnit: "Phrasem", - phrasemeKind: "DiscourseFormula", - posLikeKind: "Idiom", - surfaceKind: "Lemma", - }); - - expect(result.success).toBe(false); - }); - it("rejects Lexem + PhrasemeKind", () => { const result = DeLemmaResultSchema.safeParse({ lemma: "run", diff --git a/tests/unit/linguistics/de-verb-features.test.ts b/tests/unit/linguistics/de-verb-features.test.ts index 8dabd37a9..de0df6464 100644 --- a/tests/unit/linguistics/de-verb-features.test.ts +++ b/tests/unit/linguistics/de-verb-features.test.ts @@ -7,7 +7,7 @@ import { describe("German verb features", () => { it("parses full verb features", () => { const result = GermanVerbFullFeaturesSchema.safeParse({ - conjugation: "Rregular", + conjugation: "Regular", pos: "Verb", valency: { governedPreposition: "um", diff --git a/tests/unit/linguistics/german-linguistic-unit.test.ts b/tests/unit/linguistics/german-linguistic-unit.test.ts index 9305e464e..912179288 100644 --- a/tests/unit/linguistics/german-linguistic-unit.test.ts +++ b/tests/unit/linguistics/german-linguistic-unit.test.ts @@ -82,7 +82,7 @@ describe("GermanLinguisticUnitSchema", () => { kind: "Lexem", surface: { features: { - conjugation: "Rregular", + conjugation: "Regular", pos: "Verb", valency: { reflexivity: "NonReflexive", @@ -115,7 +115,7 @@ describe("GermanLinguisticUnitSchema", () => { kind: "Lexem", surface: { features: { - conjugation: "Rregular", + conjugation: "Regular", pos: "Verb", }, lemma: "aufmachen", diff --git a/tests/unit/note-metadata-manager/frontmatter.test.ts b/tests/unit/note-metadata-manager/frontmatter.test.ts index fd1075b11..9098e8ea5 100644 --- a/tests/unit/note-metadata-manager/frontmatter.test.ts +++ b/tests/unit/note-metadata-manager/frontmatter.test.ts @@ -5,7 +5,6 @@ import { parseFrontmatter, stripOnlyFrontmatter, } from "../../../src/stateless-helpers/note-metadata/internal/frontmatter"; -import { migrateFrontmatter } from "../../../src/stateless-helpers/note-metadata/internal/migration"; describe("frontmatter", () => { describe("parseFrontmatter", () => { @@ -193,82 +192,9 @@ title: Test expect(result.tags).toEqual(["a", "b"]); }); - it("uses completion field as fallback", () => { + it("does not map non-canonical status keys", () => { const result = frontmatterToInternal({ completion: "done" }); - expect(result.status).toBe("Done"); - }); - }); - - describe("migrateFrontmatter", () => { - it("converts frontmatter to internal format", () => { - const content = `--- -title: Test -status: done ---- -Content here`; - const transform = migrateFrontmatter(); - const result = transform(content); - - // Should not contain YAML frontmatter - expect(result).not.toContain("---"); - // Should contain internal metadata section - expect(result).toContain('
'); - // Should contain Done status - expect(result).toContain('"status":"Done"'); - // Should contain title directly (not in "imported") - expect(result).toContain('"title":"Test"'); - expect(result).not.toContain('"imported"'); - // Should preserve content - expect(result).toContain("Content here"); - }); - - it("returns original if no frontmatter", () => { - const content = "Just content"; - const transform = migrateFrontmatter(); - expect(transform(content)).toBe("Just content"); - }); - - it("handles content with only frontmatter", () => { - const content = `--- -title: Only Meta ---- -`; - const transform = migrateFrontmatter(); - const result = transform(content); - - expect(result).not.toContain("---"); - expect(result).toContain('
'); - }); - - it("keeps YAML when stripYaml is false", () => { - const content = `--- -title: Test -status: done ---- -Content here`; - const transform = migrateFrontmatter({ stripYaml: false }); - const result = transform(content); - - // Should keep YAML frontmatter - expect(result).toContain("---"); - expect(result).toContain("title: Test"); - // Should also have internal metadata section - expect(result).toContain('
'); - expect(result).toContain('"status":"Done"'); - // Should preserve content - expect(result).toContain("Content here"); - }); - - it("strips YAML when stripYaml is true (explicit)", () => { - const content = `--- -title: Test ---- -Content`; - const transform = migrateFrontmatter({ stripYaml: true }); - const result = transform(content); - - expect(result).not.toContain("---"); - expect(result).toContain('
'); + expect(result.status).toBe("NotStarted"); }); }); diff --git a/tests/unit/prompt-smith/features-schema.test.ts b/tests/unit/prompt-smith/features-schema.test.ts index d5211e609..69f117524 100644 --- a/tests/unit/prompt-smith/features-schema.test.ts +++ b/tests/unit/prompt-smith/features-schema.test.ts @@ -31,7 +31,7 @@ describe("Features schema", () => { it("accepts structured verb output", () => { const result = featuresVerbSchemas.agentOutputSchema.safeParse({ - conjugation: "Rregular", + conjugation: "Regular", valency: { reflexivity: "NonReflexive", separability: "Separable", @@ -42,7 +42,7 @@ describe("Features schema", () => { it("accepts structured verb output with null governedPreposition", () => { const result = featuresVerbSchemas.agentOutputSchema.safeParse({ - conjugation: "Rregular", + conjugation: "Regular", valency: { governedPreposition: null, reflexivity: "NonReflexive", @@ -88,7 +88,7 @@ describe("Features schema", () => { it("rejects unknown separability", () => { const result = featuresVerbSchemas.agentOutputSchema.safeParse({ - conjugation: "Rregular", + conjugation: "Regular", valency: { reflexivity: "NonReflexive", separability: "Both", @@ -99,7 +99,7 @@ describe("Features schema", () => { it("rejects unknown conjugation", () => { const result = featuresVerbSchemas.agentOutputSchema.safeParse({ - conjugation: "Regular", + conjugation: "SemiRegular", valency: { reflexivity: "NonReflexive", separability: "Separable", diff --git a/tests/unit/prompt-smith/lemma-schema.test.ts b/tests/unit/prompt-smith/lemma-schema.test.ts index f1667eeec..8c4e339f2 100644 --- a/tests/unit/prompt-smith/lemma-schema.test.ts +++ b/tests/unit/prompt-smith/lemma-schema.test.ts @@ -15,7 +15,7 @@ describe("Lemma schema", () => { expect(result.success).toBe(true); }); - it("accepts legacy Lexem output with pos and normalizes to posLikeKind", () => { + it("rejects Lexem output with legacy pos alias only", () => { const result = agentOutputSchema.safeParse({ lemma: "Haus", linguisticUnit: "Lexem", @@ -23,12 +23,10 @@ describe("Lemma schema", () => { surfaceKind: "Lemma", }); - expect(result.success).toBe(true); - if (!result.success) return; - expect(result.data.posLikeKind).toBe("Noun"); + expect(result.success).toBe(false); }); - it("accepts legacy Phrasem output with phrasemeKind and normalizes to posLikeKind", () => { + it("rejects Phrasem output with legacy phrasemeKind alias only", () => { const result = agentOutputSchema.safeParse({ lemma: "auf jeden Fall", linguisticUnit: "Phrasem", @@ -36,9 +34,7 @@ describe("Lemma schema", () => { surfaceKind: "Lemma", }); - expect(result.success).toBe(true); - if (!result.success) return; - expect(result.data.posLikeKind).toBe("DiscourseFormula"); + expect(result.success).toBe(false); }); it("rejects Phrasem output without posLikeKind", () => { diff --git a/tests/unit/pure-formatting-utils/wikilink.test.ts b/tests/unit/pure-formatting-utils/wikilink.test.ts index b3c496fbe..00da21b8e 100644 --- a/tests/unit/pure-formatting-utils/wikilink.test.ts +++ b/tests/unit/pure-formatting-utils/wikilink.test.ts @@ -126,3 +126,55 @@ describe("wikilinkHelper.createMatcher", () => { expect(nextMatch()).toBeNull(); }); }); + +describe("wikilinkHelper.normalizeLinkTarget", () => { + it("keeps plain targets unchanged", () => { + expect(wikilinkHelper.normalizeLinkTarget("fahren")).toBe("fahren"); + }); + + it("extracts basename from vault path targets", () => { + expect( + wikilinkHelper.normalizeLinkTarget( + "Worter/de/lexem/lemma/f/fah/fahre/Fahren", + ), + ).toBe("Fahren"); + }); + + it("normalizes wrapped wikilink targets", () => { + expect( + wikilinkHelper.normalizeLinkTarget( + "[[Library/de/prefix/auf-prefix-de|>auf]]", + ), + ).toBe("auf-prefix-de"); + }); + + it("preserves anchors while flattening vault paths", () => { + expect( + wikilinkHelper.normalizeLinkTarget( + "Worter/de/lexem/lemma/f/fah/fahre/Fahren#^abc", + ), + ).toBe("Fahren#^abc"); + }); + + it("does not flatten generic slash-based targets", () => { + expect( + wikilinkHelper.normalizeLinkTarget("domain/schema/field"), + ).toBe("domain/schema/field"); + }); +}); + +describe("wikilinkHelper.normalizeWikilinkTargetsInText", () => { + it("rewrites path-like targets but keeps aliases", () => { + const result = wikilinkHelper.normalizeWikilinkTargetsInText( + "[[Worter/de/lexem/lemma/f/fah/fahre/Fahren|Fahren]] + [[fahren]]", + ); + expect(result).toBe("[[Fahren|Fahren]] + [[fahren]]"); + }); + + it("keeps anchors when normalizing path-like targets", () => { + const result = wikilinkHelper.normalizeWikilinkTargetsInText( + "[[Worter/de/x/Fahren#^abc|fährt]]", + ); + expect(result).toBe("[[Fahren#^abc|fährt]]"); + }); +}); diff --git a/tests/unit/stateless-helpers/dict-note/dict-note.test.ts b/tests/unit/stateless-helpers/dict-note/dict-note.test.ts index 01cdef99e..72fc414e2 100644 --- a/tests/unit/stateless-helpers/dict-note/dict-note.test.ts +++ b/tests/unit/stateless-helpers/dict-note/dict-note.test.ts @@ -45,7 +45,7 @@ const WINDRAD_ENTRY = [ "wind turbine", ].join("\n"); -const MULTI_ENTRY_BODY = `${KOHLEKRAFTWERK_ENTRY}\n---\n---\n---\n${WINDRAD_ENTRY}`; +const MULTI_ENTRY_BODY = `${KOHLEKRAFTWERK_ENTRY}\n\n\n---\n---\n\n\n${WINDRAD_ENTRY}`; function makeNoteWithMeta(body: string, meta: Record): string { const padding = "\n".repeat(20); @@ -83,7 +83,7 @@ describe("dictNoteHelper.parse", () => { expect(e.meta).toEqual({ status: "Done" }); }); - test("parses multiple entries separated by ---\\n---\\n---", () => { + test("parses multiple entries separated by canonical separator", () => { const note = makeNoteWithMeta(MULTI_ENTRY_BODY, { entries: { "l-nom-n-m1": { status: "Done" }, @@ -126,6 +126,30 @@ describe("dictNoteHelper.parse", () => { expect(entries[0]?.meta).toEqual({}); }); + test("preserves top-level metadata fields on parse", () => { + const note = makeNoteWithMeta(KOHLEKRAFTWERK_ENTRY, { + entries: { + "l-nom-n-m1": { + emojiDescription: ["🏭", "⚡"], + ipa: "ˈkoːləˌkraftvɛɐ̯k", + semantics: "legacy semantics", + senseGloss: "legacy gloss", + status: "Done", + }, + }, + }); + + const entries = dictNoteHelper.parse(note); + expect(entries).toHaveLength(1); + expect(entries[0]?.meta).toEqual({ + emojiDescription: ["🏭", "⚡"], + ipa: "ˈkoːləˌkraftvɛɐ̯k", + semantics: "legacy semantics", + senseGloss: "legacy gloss", + status: "Done", + }); + }); + test("two sections with same kind but different titles preserved", () => { const entries = dictNoteHelper.parse(KOHLEKRAFTWERK_ENTRY); const sections = entries[0]?.sections; diff --git a/tests/unit/textfresser/common/morpheme-link-target.test.ts b/tests/unit/textfresser/common/morpheme-link-target.test.ts new file mode 100644 index 000000000..ad7e6242a --- /dev/null +++ b/tests/unit/textfresser/common/morpheme-link-target.test.ts @@ -0,0 +1,47 @@ +import { describe, expect, it } from "bun:test"; +import { resolveMorphemeItems } from "../../../../src/commanders/textfresser/common/morpheme-link-target"; +import type { LlmMorpheme } from "../../../../src/prompt-smith/schemas/morphem"; + +describe("resolveMorphemeItems", () => { + it("does not duplicate prefix suffix when surf already contains -prefix-de", () => { + const morphemes: LlmMorpheme[] = [ + { + kind: "Prefix", + separability: "Separable", + surf: "Worter/de/prefix/auf-prefix-de", + }, + ]; + + const [prefix] = resolveMorphemeItems(morphemes, "German"); + expect(prefix?.surf).toBe("auf"); + expect(prefix?.linkTarget).toBe("auf-prefix-de"); + }); + + it("builds canonical prefix target from plain prefix surface", () => { + const morphemes: LlmMorpheme[] = [ + { + kind: "Prefix", + separability: "Inseparable", + surf: "ver", + }, + ]; + + const [prefix] = resolveMorphemeItems(morphemes, "German"); + expect(prefix?.surf).toBe("ver"); + expect(prefix?.linkTarget).toBe("ver-prefix-de"); + }); + + it("strips anchors from morpheme surf/lemma tokens", () => { + const morphemes: LlmMorpheme[] = [ + { + kind: "Root", + lemma: "fahren#^lemma", + surf: "fährt#^surface", + }, + ]; + + const [root] = resolveMorphemeItems(morphemes, "German"); + expect(root?.surf).toBe("fährt"); + expect(root?.lemma).toBe("fahren"); + }); +}); diff --git a/tests/unit/textfresser/domain/propagation/morphology-roundtrip-corpus.test.ts b/tests/unit/textfresser/domain/propagation/morphology-roundtrip-corpus.test.ts index a833afc59..82d6589ef 100644 --- a/tests/unit/textfresser/domain/propagation/morphology-roundtrip-corpus.test.ts +++ b/tests/unit/textfresser/domain/propagation/morphology-roundtrip-corpus.test.ts @@ -145,7 +145,7 @@ const MORPHOLOGY_CORPUS: MorphologyCorpusFixture[] = [ }, ]; -describe("propagation-v2 morphology corpus", () => { +describe("propagation morphology corpus", () => { it("matches expected parse semantics for mixed and malformed morphology fixtures", () => { for (const fixture of MORPHOLOGY_CORPUS) { const parsed = getMorphologyPayload(fixture.note); diff --git a/tests/unit/textfresser/domain/propagation/note-adapter.test.ts b/tests/unit/textfresser/domain/propagation/note-adapter.test.ts index e47ee77db..41b2273f6 100644 --- a/tests/unit/textfresser/domain/propagation/note-adapter.test.ts +++ b/tests/unit/textfresser/domain/propagation/note-adapter.test.ts @@ -356,36 +356,36 @@ function getTypedPayload( } } -describe("propagation-v2 note adapter", () => { - it("parses typed sections with DTO semantics equivalent to v1-parsed section content", () => { - const v1Entries = dictNoteHelper.parse(TYPED_NOTE_FIXTURE); - const v1Entry = v1Entries[0]; - if (!v1Entry) { - throw new Error("Expected v1 fixture to parse one entry"); +describe("propagation note adapter", () => { + it("parses typed sections with DTO semantics equivalent to legacy parsed section content", () => { + const legacyEntries = dictNoteHelper.parse(TYPED_NOTE_FIXTURE); + const legacyEntry = legacyEntries[0]; + if (!legacyEntry) { + throw new Error("Expected legacy fixture to parse one entry"); } const expectedRelation = parseLegacyRelationSection( - getSectionContentByCssKind(v1Entry, "synonyme"), + getSectionContentByCssKind(legacyEntry, "synonyme"), ); const expectedMorphology = parseLegacyMorphologySection( - getSectionContentByCssKind(v1Entry, "morphologie"), + getSectionContentByCssKind(legacyEntry, "morphologie"), ); const expectedInflection = parseLegacyInflectionSection( - getSectionContentByCssKind(v1Entry, "flexion"), + getSectionContentByCssKind(legacyEntry, "flexion"), ); const expectedTags = parseLegacyTagsSection( - getSectionContentByCssKind(v1Entry, "tags"), + getSectionContentByCssKind(legacyEntry, "tags"), ); - const v2Entries = parsePropagationNote(TYPED_NOTE_FIXTURE); - const v2Entry = v2Entries[0]; - if (!v2Entry) { - throw new Error("Expected v2 fixture to parse one entry"); + const currentEntries = parsePropagationNote(TYPED_NOTE_FIXTURE); + const currentEntry = currentEntries[0]; + if (!currentEntry) { + throw new Error("Expected current fixture to parse one entry"); } - const actualRelation = getTypedPayload(v2Entry, "Relation"); - const actualMorphology = getTypedPayload(v2Entry, "Morphology"); - const actualInflection = getTypedPayload(v2Entry, "Inflection"); - const actualTags = getTypedPayload(v2Entry, "Tags"); + const actualRelation = getTypedPayload(currentEntry, "Relation"); + const actualMorphology = getTypedPayload(currentEntry, "Morphology"); + const actualInflection = getTypedPayload(currentEntry, "Inflection"); + const actualTags = getTypedPayload(currentEntry, "Tags"); expect(actualRelation).toEqual(expectedRelation); expect(actualMorphology).toEqual(expectedMorphology); @@ -471,4 +471,37 @@ describe("propagation-v2 note adapter", () => { const serialized = serializePropagationNote(parsed); expect(serialized.body.includes(RAW_TRANSLATION_BLOCK)).toBe(true); }); + + it("preserves top-level metadata mirrors on parse", () => { + const noteWithLegacyMeta = [ + "wort ^raw-1", + "", + 'Übersetzung', + "word", + "", + '
', + JSON.stringify({ + entries: { + "raw-1": { + emojiDescription: ["🧪"], + ipa: "ipa", + semantics: "legacy semantics", + senseGloss: "legacy gloss", + verbEntryIdentity: "conjugation:Irregular", + }, + }, + }), + "
", + ].join("\n"); + + const parsed = parsePropagationNote(noteWithLegacyMeta); + expect(parsed).toHaveLength(1); + expect(parsed[0]?.meta).toEqual({ + emojiDescription: ["🧪"], + ipa: "ipa", + semantics: "legacy semantics", + senseGloss: "legacy gloss", + verbEntryIdentity: "conjugation:Irregular", + }); + }); }); diff --git a/tests/unit/textfresser/formatters/common/header-formatter.test.ts b/tests/unit/textfresser/formatters/common/header-formatter.test.ts index 7808f1003..00a038ed2 100644 --- a/tests/unit/textfresser/formatters/common/header-formatter.test.ts +++ b/tests/unit/textfresser/formatters/common/header-formatter.test.ts @@ -52,4 +52,15 @@ describe("formatHeaderLine", () => { ); expect(result).toContain("/english)"); }); + + it("normalizes vault-path lemma targets to basename", () => { + const result = formatHeaderLine( + { emojiDescription: ["🚗"], ipa: "ˈfaːʁən" }, + "Worter/de/lexem/lemma/f/fah/fahre/Fahren", + "German", + ); + expect(result).toBe( + "🚗 [[Fahren]], [ˈfaːʁən](https://youglish.com/pronounce/Fahren/german)", + ); + }); }); diff --git a/tests/unit/textfresser/formatters/common/inflection-formatter.test.ts b/tests/unit/textfresser/formatters/common/inflection-formatter.test.ts index 936294f1b..215f24cee 100644 --- a/tests/unit/textfresser/formatters/common/inflection-formatter.test.ts +++ b/tests/unit/textfresser/formatters/common/inflection-formatter.test.ts @@ -25,4 +25,16 @@ describe("formatInflectionSection", () => { const result = formatInflectionSection({ rows: [] }); expect(result).toBe(""); }); + + it("normalizes wikilink targets that contain vault paths", () => { + const result = formatInflectionSection({ + rows: [ + { + forms: "ich [[Worter/de/lexem/lemma/f/fah/fahre/Fahren|fahre]]", + label: "Präsens", + }, + ], + }); + expect(result).toBe("Präsens: ich [[Fahren|fahre]]"); + }); }); diff --git a/tests/unit/textfresser/formatters/common/inflection-propagation-helper.test.ts b/tests/unit/textfresser/formatters/common/inflection-propagation-helper.test.ts index 4c3b97b4e..7d681c043 100644 --- a/tests/unit/textfresser/formatters/common/inflection-propagation-helper.test.ts +++ b/tests/unit/textfresser/formatters/common/inflection-propagation-helper.test.ts @@ -4,7 +4,6 @@ import { buildNounInflectionPropagationHeader, isNounInflectionPropagationHeaderForLemma, mergeLocalizedInflectionTags, - parseLegacyInflectionHeaderTag, } from "../../../../../src/commanders/textfresser/commands/generate/section-formatters/common/inflection-propagation-helper"; import type { NounInflectionCell } from "../../../../../src/linguistics/de/lexem/noun"; @@ -64,32 +63,6 @@ describe("inflectionPropagationHelper", () => { ); }); - it("parses legacy per-cell headers and localizes tags", () => { - expect( - parseLegacyInflectionHeaderTag( - "#Nominativ/Plural for: [[Kraftwerk]]", - "Kraftwerk", - "German", - ), - ).toBe("#Nominativ/Plural"); - - expect( - parseLegacyInflectionHeaderTag( - "#Nominative/Plural for: [[Kraftwerk]]", - "Kraftwerk", - "German", - ), - ).toBe("#Nominativ/Plural"); - - expect( - parseLegacyInflectionHeaderTag( - "#Nominativ/Plural for: [[Fabrik]]", - "Kraftwerk", - "German", - ), - ).toBeNull(); - }); - it("builds fallback and genus-specific noun inflection headers", () => { expect( buildNounInflectionPropagationHeader( diff --git a/tests/unit/textfresser/formatters/common/relation-formatter.test.ts b/tests/unit/textfresser/formatters/common/relation-formatter.test.ts index 99410381c..ce9af6c6a 100644 --- a/tests/unit/textfresser/formatters/common/relation-formatter.test.ts +++ b/tests/unit/textfresser/formatters/common/relation-formatter.test.ts @@ -53,4 +53,16 @@ describe("formatRelationSection", () => { const result = formatRelationSection({ relations: [] }); expect(result).toBe(""); }); + + it("normalizes relation words when prompt output contains vault paths", () => { + const result = formatRelationSection({ + relations: [ + { + kind: "Synonym", + words: ["Worter/de/lexem/lemma/f/fah/fahre/Fahren"], + }, + ], + }); + expect(result).toBe("= [[Fahren]]"); + }); }); diff --git a/tests/unit/textfresser/formatters/de/lexem/noun/header-formatter.test.ts b/tests/unit/textfresser/formatters/de/lexem/noun/header-formatter.test.ts index 2567a08cc..c129c302e 100644 --- a/tests/unit/textfresser/formatters/de/lexem/noun/header-formatter.test.ts +++ b/tests/unit/textfresser/formatters/de/lexem/noun/header-formatter.test.ts @@ -45,4 +45,16 @@ describe("noun formatHeaderLine", () => { "💨 ✨ der [[Staub]], [ˈʃtaʊ̯p](https://youglish.com/pronounce/Staub/german)", ); }); + + it("normalizes vault-path lemma targets to basename", () => { + const result = formatHeaderLine( + { emojiDescription: ["🚗"], ipa: "ˈfaːʁən" }, + "Worter/de/lexem/lemma/f/fah/fahre/Fahren", + "German", + "Maskulinum", + ); + expect(result).toBe( + "🚗 der [[Fahren]], [ˈfaːʁən](https://youglish.com/pronounce/Fahren/german)", + ); + }); }); diff --git a/tests/unit/textfresser/formatters/de/lexem/noun/inflection-formatter.test.ts b/tests/unit/textfresser/formatters/de/lexem/noun/inflection-formatter.test.ts index 7b99f52eb..a7127142c 100644 --- a/tests/unit/textfresser/formatters/de/lexem/noun/inflection-formatter.test.ts +++ b/tests/unit/textfresser/formatters/de/lexem/noun/inflection-formatter.test.ts @@ -88,4 +88,21 @@ describe("formatInflection", () => { expect(formattedSection).toBe(""); expect(cells).toHaveLength(0); }); + + it("normalizes path-like form targets to basename", () => { + const { formattedSection, cells } = formatInflection({ + cells: [ + { + article: "die", + case: "Nominative", + form: "Worter/de/lexem/lemma/f/fah/fahre/Fahren", + number: "Plural", + }, + ], + genus: "Neutrum", + }); + + expect(formattedSection).toBe("N: die [[Fahren]]"); + expect(cells[0]?.form).toBe("Fahren"); + }); }); diff --git a/tests/unit/textfresser/orchestration/lemma-cache.test.ts b/tests/unit/textfresser/orchestration/lemma-cache.test.ts index 88ded47a0..33610586c 100644 --- a/tests/unit/textfresser/orchestration/lemma-cache.test.ts +++ b/tests/unit/textfresser/orchestration/lemma-cache.test.ts @@ -56,6 +56,7 @@ function makeState(cacheAtMs: number): TextfresserState { return { attestationForLatestNavigated: null, inFlightGenerate: null, + isLibraryLookupAvailable: false, languages: { known: "English", target: "German" }, latestFailedSections: [], latestLemmaInvocationCache: { diff --git a/tests/unit/textfresser/steps/build-linguistic-unit-meta.test.ts b/tests/unit/textfresser/steps/build-linguistic-unit-meta.test.ts index cd0f8ad55..a2ad3a08f 100644 --- a/tests/unit/textfresser/steps/build-linguistic-unit-meta.test.ts +++ b/tests/unit/textfresser/steps/build-linguistic-unit-meta.test.ts @@ -127,7 +127,7 @@ function makeVerbFeatures( overrides: Partial> = {}, ): AgentOutput<"FeaturesVerb"> { return { - conjugation: "Rregular", + conjugation: "Regular", valency: { reflexivity: "NonReflexive", separability: "Separable", @@ -228,7 +228,7 @@ describe("buildLinguisticUnitMeta", () => { kind: "Lexem", surface: { features: { - conjugation: "Rregular", + conjugation: "Regular", pos: "Verb", valency: { reflexivity: "NonReflexive", diff --git a/tests/unit/textfresser/steps/closed-set-surface-hub.test.ts b/tests/unit/textfresser/steps/closed-set-surface-hub.test.ts new file mode 100644 index 000000000..3e4e7208b --- /dev/null +++ b/tests/unit/textfresser/steps/closed-set-surface-hub.test.ts @@ -0,0 +1,223 @@ +import { describe, expect, it } from "bun:test"; +import { ok } from "neverthrow"; +import { + buildClosedSetSurfaceHubActions, + buildClosedSetSurfaceHubBackfillActions, + buildClosedSetSurfaceHubContent, + buildClosedSetSurfaceHubSplitPath, +} from "../../../../src/commanders/textfresser/common/closed-set-surface-hub"; +import { + SplitPathKind, + type SplitPathToMdFile, + type SplitPathToMdFileWithReader, +} from "../../../../src/managers/obsidian/vault-action-manager/types/split-path"; +import { VaultActionKind } from "../../../../src/managers/obsidian/vault-action-manager/types/vault-action"; + +function makePath( + basename: string, + pathParts: string[], +): SplitPathToMdFile { + return { + basename, + extension: "md", + kind: "MdFile", + pathParts, + }; +} + +function makePathWithReader( + basename: string, + pathParts: string[], + content: string, +): SplitPathToMdFileWithReader { + return { + basename, + extension: "md", + kind: SplitPathKind.MdFile, + pathParts, + read: () => Promise.resolve(ok(content)), + }; +} + +describe("closed-set surface hubs", () => { + it("creates hub content when a second closed-set target appears", () => { + const pronoun = makePath("die-pronoun-de", ["Library", "de", "pronoun"]); + const article = makePath("die-article-de", ["Library", "de", "article"]); + + const actions = buildClosedSetSurfaceHubActions({ + currentClosedSetTarget: article, + lookupInLibrary: () => [pronoun], + surface: "Die", + targetLanguage: "German", + vam: { + exists: () => false, + }, + }); + + expect(actions).toHaveLength(1); + const action = actions[0]; + expect(action?.kind).toBe(VaultActionKind.UpsertMdFile); + if (!action || action.kind !== VaultActionKind.UpsertMdFile) { + throw new Error("Expected UpsertMdFile action"); + } + expect(action.payload.splitPath).toEqual( + buildClosedSetSurfaceHubSplitPath("die", "German"), + ); + expect(action.payload.content).toContain("closed-set-surface-hub"); + expect(action.payload.content).toContain("die-pronoun-de"); + expect(action.payload.content).toContain("die-article-de"); + }); + + it("does not create a hub for a single target when hub does not exist", () => { + const pronoun = makePath("die-pronoun-de", ["Library", "de", "pronoun"]); + + const actions = buildClosedSetSurfaceHubActions({ + lookupInLibrary: () => [pronoun], + surface: "die", + targetLanguage: "German", + vam: { + exists: () => false, + }, + }); + + expect(actions).toHaveLength(0); + }); + + it("updates existing hub even if only one target remains", () => { + const pronoun = makePath("die-pronoun-de", ["Library", "de", "pronoun"]); + const hubPath = buildClosedSetSurfaceHubSplitPath("die", "German"); + + const actions = buildClosedSetSurfaceHubActions({ + lookupInLibrary: () => [pronoun], + surface: "die", + targetLanguage: "German", + vam: { + exists: (path) => + path.pathParts.join("/") === hubPath.pathParts.join("/") && + path.basename === hubPath.basename, + }, + }); + + expect(actions).toHaveLength(1); + expect(actions[0]?.kind).toBe(VaultActionKind.UpsertMdFile); + }); + + it("trashes hub when no closed-set targets remain", () => { + const hubPath = buildClosedSetSurfaceHubSplitPath("die", "German"); + + const actions = buildClosedSetSurfaceHubActions({ + lookupInLibrary: () => [], + surface: "die", + targetLanguage: "German", + vam: { + exists: (path) => + path.pathParts.join("/") === hubPath.pathParts.join("/") && + path.basename === hubPath.basename, + }, + }); + + expect(actions).toHaveLength(1); + const action = actions[0]; + expect(action?.kind).toBe(VaultActionKind.TrashMdFile); + }); + + it("backfill is idempotent when existing hub content is already up to date", async () => { + const pronoun = makePath("die-pronoun-de", ["Library", "de", "pronoun"]); + const article = makePath("die-article-de", ["Library", "de", "article"]); + const hubPath = buildClosedSetSurfaceHubSplitPath("die", "German"); + const existingHubContent = buildClosedSetSurfaceHubContent({ + surface: "die", + targetLanguage: "German", + targets: [pronoun, article], + }); + + const result = await buildClosedSetSurfaceHubBackfillActions({ + lookupInLibrary: () => [pronoun, article], + targetLanguage: "German", + vam: { + exists: (path) => + path.pathParts[0] === "Library" || + (path.pathParts[0] === "Worter" && + path.basename === "closed-set-hub"), + list: () => ok([hubPath]), + listAllFilesWithMdReaders: (folder) => { + if (folder.pathParts[0] === "Library") { + return ok([ + makePathWithReader( + pronoun.basename, + pronoun.pathParts, + "", + ), + makePathWithReader( + article.basename, + article.pathParts, + "", + ), + ]); + } + return ok([ + makePathWithReader( + hubPath.basename, + hubPath.pathParts, + existingHubContent, + ), + ]); + }, + }, + }); + + expect(result.isOk()).toBe(true); + if (result.isErr()) { + throw new Error(result.error); + } + expect(result.value).toHaveLength(0); + }); + + it("backfill upserts when existing hub content is stale", async () => { + const pronoun = makePath("die-pronoun-de", ["Library", "de", "pronoun"]); + const article = makePath("die-article-de", ["Library", "de", "article"]); + const hubPath = buildClosedSetSurfaceHubSplitPath("die", "German"); + + const result = await buildClosedSetSurfaceHubBackfillActions({ + lookupInLibrary: () => [pronoun, article], + targetLanguage: "German", + vam: { + exists: (path) => + path.pathParts[0] === "Library" || + (path.pathParts[0] === "Worter" && + path.basename === "closed-set-hub"), + list: () => ok([hubPath]), + listAllFilesWithMdReaders: (folder) => { + if (folder.pathParts[0] === "Library") { + return ok([ + makePathWithReader( + pronoun.basename, + pronoun.pathParts, + "", + ), + makePathWithReader( + article.basename, + article.pathParts, + "", + ), + ]); + } + return ok([ + makePathWithReader( + hubPath.basename, + hubPath.pathParts, + "old-content", + ), + ]); + }, + }, + }); + + expect(result.isOk()).toBe(true); + if (result.isErr()) { + throw new Error(result.error); + } + expect(result.value).toHaveLength(1); + expect(result.value[0]?.kind).toBe(VaultActionKind.UpsertMdFile); + }); +}); diff --git a/tests/unit/textfresser/steps/decorate-attestation-separability.test.ts b/tests/unit/textfresser/steps/decorate-attestation-separability.test.ts index 8eb05d34c..60d05fe9d 100644 --- a/tests/unit/textfresser/steps/decorate-attestation-separability.test.ts +++ b/tests/unit/textfresser/steps/decorate-attestation-separability.test.ts @@ -73,7 +73,7 @@ function extractTransform( } describe("decorateAttestationSeparability", () => { - it("decorates separable verb with two wikilinks: prefix gets < and stem gets >", () => { + it("keeps multi-span aliases unchanged for separable verbs", () => { const ctx = makeCtx([ { kind: "Prefix", @@ -91,9 +91,7 @@ describe("decorateAttestationSeparability", () => { const content = "[[aufpassen|Pass]] auf dich [[aufpassen|auf]]"; const decorated = transform!(content); - expect(decorated).toBe( - "[[aufpassen|>Pass]] auf dich [[aufpassen|auf<]]", - ); + expect(decorated).toBe(content); }); it("skips inseparable verbs (no decoration)", () => { @@ -218,7 +216,7 @@ describe("decorateAttestationSeparability", () => { expect(decorated).toBe(content); }); - it("handles case-insensitive prefix matching", () => { + it("keeps case-variant multi-span aliases unchanged", () => { const ctx = makeCtx([ { kind: "Prefix", @@ -234,8 +232,6 @@ describe("decorateAttestationSeparability", () => { // Capitalized prefix alias (e.g., at sentence start) const content = "[[aufpassen|Auf]] dich [[aufpassen|Pass]]!"; const decorated = transform!(content); - expect(decorated).toBe( - "[[aufpassen|Auf<]] dich [[aufpassen|>Pass]]!", - ); + expect(decorated).toBe(content); }); }); diff --git a/tests/unit/textfresser/steps/disambiguate-sense.test.ts b/tests/unit/textfresser/steps/disambiguate-sense.test.ts index 4cc6773f0..23b4b1cd4 100644 --- a/tests/unit/textfresser/steps/disambiguate-sense.test.ts +++ b/tests/unit/textfresser/steps/disambiguate-sense.test.ts @@ -2,7 +2,7 @@ import { describe, expect, it } from "bun:test"; import { errAsync, ok, okAsync } from "neverthrow"; import { disambiguateSense } from "../../../../src/commanders/textfresser/commands/lemma/steps/disambiguate-sense"; import type { PromptRunner } from "../../../../src/commanders/textfresser/llm/prompt-runner"; -import type { DeEntity, GermanLinguisticUnit } from "../../../../src/linguistics/de"; +import type { DeEntity } from "../../../../src/linguistics/de"; import type { VaultActionManager } from "../../../../src/managers/obsidian/vault-action-manager"; import type { SplitPathToMdFile } from "../../../../src/managers/obsidian/vault-action-manager/types/split-path"; @@ -69,17 +69,86 @@ const API_RESULT_PHRASEM = { surfaceKind: "Lemma", } as const; +function makeNounEntity( + params: { + emojiDescription: string[]; + genus?: "Femininum" | "Maskulinum" | "Neutrum"; + lemma?: string; + senseGloss?: string; + } = { emojiDescription: ["🏦"] }, +): DeEntity<"Lexem", "Lemma"> { + return { + emojiDescription: params.emojiDescription, + features: { + inflectional: {}, + lexical: { + genus: params.genus ?? "Femininum", + nounClass: "Common", + pos: "Noun", + }, + }, + ipa: "ˈbaŋk", + language: "German", + lemma: params.lemma ?? "Bank", + linguisticUnit: "Lexem", + posLikeKind: "Noun", + ...(params.senseGloss ? { senseGloss: params.senseGloss } : {}), + surfaceKind: "Lemma", + }; +} + +function makeVerbEntity( + emojiDescription: string[], +): DeEntity<"Lexem", "Lemma"> { + return { + emojiDescription, + features: { + inflectional: {}, + lexical: { + conjugation: "Irregular", + pos: "Verb", + valency: { + governedPreposition: null, + reflexivity: "NonReflexive", + separability: "None", + }, + }, + }, + ipa: "ˈfaːʁən", + language: "German", + lemma: "fahren", + linguisticUnit: "Lexem", + posLikeKind: "Verb", + surfaceKind: "Lemma", + }; +} + +function makePhrasemEntity( + emojiDescription: string[], +): DeEntity<"Phrasem", "Lemma"> { + return { + emojiDescription, + features: { + inflectional: {}, + lexical: { phrasemeKind: "DiscourseFormula" }, + }, + ipa: "aʊ̯f ˈjeːdn̩ fal", + language: "German", + lemma: "auf jeden Fall", + linguisticUnit: "Phrasem", + posLikeKind: "DiscourseFormula", + surfaceKind: "Lemma", + }; +} + /** * Build a minimal note with entries for testing. - * Each entry has: header line with ^blockId, metadata with optional emojiDescription. + * Each entry has: header line with ^blockId, metadata with canonical entity payload. */ function buildNoteContent( entries: Array<{ entity?: DeEntity; id: string; - emojiDescription?: string[]; - linguisticUnit?: GermanLinguisticUnit; - senseGloss?: string; translationFirstLine?: string; }>, ): string { @@ -90,28 +159,19 @@ function buildNoteContent( : ""; return `${header}${translationSection}`; }); - const body = entryBlocks.join("\n\n---\n---\n\n"); + const body = entryBlocks.join("\n\n\n---\n---\n\n\n"); const meta: Record< string, { entity?: DeEntity; - emojiDescription?: string[]; - linguisticUnit?: GermanLinguisticUnit; - senseGloss?: string; } > = {}; for (const e of entries) { const entryMeta: { entity?: DeEntity; - emojiDescription?: string[]; - linguisticUnit?: GermanLinguisticUnit; - senseGloss?: string; } = {}; if (e.entity) entryMeta.entity = e.entity; - if (e.emojiDescription) entryMeta.emojiDescription = e.emojiDescription; - if (e.linguisticUnit) entryMeta.linguisticUnit = e.linguisticUnit; - if (e.senseGloss) entryMeta.senseGloss = e.senseGloss; meta[e.id.toUpperCase()] = entryMeta; } @@ -129,7 +189,7 @@ describe("disambiguateSense", () => { it("returns null when note exists but has no matching entries", async () => { const content = buildNoteContent([ - { emojiDescription: ["🏦"], id: "LX-LM-VERB-1" }, + { entity: makeVerbEntity(["🏦"]), id: "LX-LM-VERB-1" }, ]); const vam = makeVam({ content, files: [MOCK_SPLIT_PATH] }); const runner = makePromptRunner(null); @@ -140,7 +200,7 @@ describe("disambiguateSense", () => { it("returns matchedIndex when prompt matches existing sense", async () => { const content = buildNoteContent([ - { emojiDescription: ["🏦"], id: "LX-LM-NOUN-1" }, + { entity: makeNounEntity({ emojiDescription: ["🏦"] }), id: "LX-LM-NOUN-1" }, ]); const vam = makeVam({ content, files: [MOCK_SPLIT_PATH] }); const runner = makePromptRunner(1); @@ -151,7 +211,7 @@ describe("disambiguateSense", () => { it("returns null with precomputedEmojiDescription when prompt says new sense", async () => { const content = buildNoteContent([ - { emojiDescription: ["🏦"], id: "LX-LM-NOUN-1" }, + { entity: makeNounEntity({ emojiDescription: ["🏦"] }), id: "LX-LM-NOUN-1" }, ]); const vam = makeVam({ content, files: [MOCK_SPLIT_PATH] }); const runner = makePromptRunner(null, ["🪑", "🌳"]); @@ -166,7 +226,7 @@ describe("disambiguateSense", () => { it("bounds-checks matchedIndex — out-of-range treated as new sense", async () => { const content = buildNoteContent([ - { emojiDescription: ["🏦"], id: "LX-LM-NOUN-1" }, + { entity: makeNounEntity({ emojiDescription: ["🏦"] }), id: "LX-LM-NOUN-1" }, ]); const vam = makeVam({ content, files: [MOCK_SPLIT_PATH] }); // LLM returns matchedIndex 99 — not a valid index @@ -179,7 +239,7 @@ describe("disambiguateSense", () => { expect(value!.matchedIndex).toBeNull(); }); - it("returns first entry index for V2 legacy (all entries lack emojiDescription)", async () => { + it("treats all-senses-missing-emojiDescription as new sense", async () => { const content = buildNoteContent([ { id: "LX-LM-NOUN-1" }, ]); @@ -187,14 +247,13 @@ describe("disambiguateSense", () => { const runner = makePromptRunner(null); const result = await disambiguateSense(vam, runner, API_RESULT_NOUN, "context"); expect(result.isOk()).toBe(true); - // V2 legacy: treat as re-encounter of first match const value = result._unsafeUnwrap(); - expect(value).toEqual({ matchedIndex: 1 }); + expect(value).toEqual({ matchedIndex: null }); }); it("returns error when prompt runner fails", async () => { const content = buildNoteContent([ - { emojiDescription: ["🏦"], id: "LX-LM-NOUN-1" }, + { entity: makeNounEntity({ emojiDescription: ["🏦"] }), id: "LX-LM-NOUN-1" }, ]); const vam = makeVam({ content, files: [MOCK_SPLIT_PATH] }); const runner = makeFailingPromptRunner(); @@ -202,19 +261,11 @@ describe("disambiguateSense", () => { expect(result.isErr()).toBe(true); }); - it("forwards phrasemeKind hint from metadata to disambiguate prompt senses", async () => { + it("forwards phrasemeKind hint from entity metadata to disambiguate prompt senses", async () => { const content = buildNoteContent([ { - emojiDescription: ["✅"], + entity: makePhrasemEntity(["✅"]), id: "PH-LM-1", - linguisticUnit: { - kind: "Phrasem", - surface: { - features: { phrasemeKind: "DiscourseFormula" }, - lemma: "auf jeden Fall", - surfaceKind: "Lemma", - }, - }, }, ]); const vam = makeVam({ @@ -331,7 +382,7 @@ describe("disambiguateSense", () => { it("derives senseGloss from translation section when metadata gloss is missing", async () => { const content = buildNoteContent([ { - emojiDescription: ["🔒"], + entity: makeNounEntity({ emojiDescription: ["🔒"] }), id: "LX-LM-NOUN-1", translationFirstLine: "door lock", }, @@ -377,10 +428,10 @@ describe("disambiguateSense", () => { pathParts: ["Library", "de", "noun"], }; const fallbackContent = buildNoteContent([ - { emojiDescription: ["🏦"], id: "LX-LM-NOUN-1" }, + { entity: makeNounEntity({ emojiDescription: ["🏦"] }), id: "LX-LM-NOUN-1" }, ]); const preferredContent = buildNoteContent([ - { emojiDescription: ["💺"], id: "LX-LM-NOUN-2" }, + { entity: makeNounEntity({ emojiDescription: ["💺"] }), id: "LX-LM-NOUN-2" }, ]); const vam = makeVam({ contentByPath: { diff --git a/tests/unit/textfresser/steps/lemma-link-routing.test.ts b/tests/unit/textfresser/steps/lemma-link-routing.test.ts index 5c271a1b1..cdc4c6d8a 100644 --- a/tests/unit/textfresser/steps/lemma-link-routing.test.ts +++ b/tests/unit/textfresser/steps/lemma-link-routing.test.ts @@ -64,7 +64,7 @@ describe("lemma-link-routing", () => { }); expect(fromLibrary.splitPath).toEqual(libraryPath); - expect(fromLibrary.linkTarget).toBe("Library/de/noun/Staub"); + expect(fromLibrary.linkTarget).toBe("Staub"); expect(fromLibrary.shouldCreatePlaceholder).toBe(false); const placeholder = computePrePromptTarget({ @@ -78,6 +78,24 @@ describe("lemma-link-routing", () => { expect(placeholder.splitPath.pathParts[0]).toBe("Worter"); }); + it("pre-prompt target: uses full-path for ambiguous Library basename", () => { + const sourcePath = makePath("Source", ["Books", "A"]); + const libraryPronoun = makePath("ich", ["Library", "de", "pronoun"]); + const libraryArticle = makePath("ich", ["Library", "de", "article"]); + + const target = computePrePromptTarget({ + lookupInLibrary: () => [libraryPronoun, libraryArticle], + resolveLinkpathDest: () => null, + sourcePath, + surface: "ich", + targetLanguage: "German", + }); + + expect(target.splitPath).toEqual(libraryPronoun); + expect(target.linkTarget).toBe("Library/de/pronoun/ich"); + expect(target.shouldCreatePlaceholder).toBe(false); + }); + it("final target resolution honors closed/open policy and precedence", () => { const libraryPath = makePath("ich", ["Library", "de", "pronoun"]); const worterPath = makePath("ich", ["Worter", "de", "lexem", "lemma", "i", "ich", "ich"]); @@ -108,11 +126,32 @@ describe("lemma-link-routing", () => { expect(open.linkTarget).toBe("laufen"); }); - it("formats link target as full path for Library and basename for Worter", () => { + it("formats link target as basename by default", () => { const library = makePath("ich", ["Library", "de", "pronoun"]); const worter = makePath("laufen", ["Worter", "de", "lexem", "lemma", "l", "lau", "laufe"]); - expect(formatLinkTarget(library)).toBe("Library/de/pronoun/ich"); + expect(formatLinkTarget(library)).toBe("ich"); expect(formatLinkTarget(worter)).toBe("laufen"); + expect(formatLinkTarget(library, { libraryTargetStyle: "full-path" })).toBe( + "Library/de/pronoun/ich", + ); + }); + + it("falls back to full-path for Library target when basename is ambiguous", () => { + const libraryPronoun = makePath("ich", ["Library", "de", "pronoun"]); + const libraryArticle = makePath("ich", ["Library", "de", "article"]); + + const target = computeFinalTarget({ + findByBasename: () => [libraryPronoun, libraryArticle], + lemma: "ich", + linguisticUnit: "Lexem", + lookupInLibrary: () => [libraryPronoun], + posLikeKind: "Pronoun", + surfaceKind: "Lemma", + targetLanguage: "German", + }); + + expect(target.splitPath).toEqual(libraryPronoun); + expect(target.linkTarget).toBe("Library/de/pronoun/ich"); }); }); diff --git a/tests/unit/textfresser/steps/lemma-output-guardrails.test.ts b/tests/unit/textfresser/steps/lemma-output-guardrails.test.ts index 02a07c02d..1b2ed8da8 100644 --- a/tests/unit/textfresser/steps/lemma-output-guardrails.test.ts +++ b/tests/unit/textfresser/steps/lemma-output-guardrails.test.ts @@ -81,4 +81,40 @@ describe("lemma output guardrails", () => { const chosen = chooseBestEffortLemmaOutput({ first, second }); expect(chosen.output.lemma).toBe("anfangen"); }); + + it("normalizes path-like lemma outputs to basename", () => { + const evaluation = evaluateLemmaOutputGuardrails({ + context: "Er [fährt] schnell.", + output: { + contextWithLinkedParts: "Er [[Worter/de/lexem/lemma/f/fah/fahre/Fahren|fährt]] schnell.", + lemma: "Worter/de/lexem/lemma/f/fah/fahre/Fahren", + linguisticUnit: "Lexem", + posLikeKind: "Verb", + surfaceKind: "Inflected", + }, + surface: "fährt", + }); + + expect(evaluation.output.lemma).toBe("Fahren"); + }); + + it("keeps normalized contextWithLinkedParts when stripped text matches", () => { + const evaluation = evaluateLemmaOutputGuardrails({ + context: "Er [[Fahren|fährt]] schnell.", + output: { + contextWithLinkedParts: + "Er [[Worter/de/lexem/lemma/f/fah/fahre/Fahren|fährt]] schnell.", + lemma: "fahren", + linguisticUnit: "Lexem", + posLikeKind: "Verb", + surfaceKind: "Inflected", + }, + surface: "fährt", + }); + + expect(evaluation.droppedContextWithLinkedParts).toBe(false); + expect(evaluation.output.contextWithLinkedParts).toBe( + "Er [[Fahren|fährt]] schnell.", + ); + }); }); diff --git a/tests/unit/textfresser/steps/maintain-closed-set-surface-hub.test.ts b/tests/unit/textfresser/steps/maintain-closed-set-surface-hub.test.ts new file mode 100644 index 000000000..6d266885b --- /dev/null +++ b/tests/unit/textfresser/steps/maintain-closed-set-surface-hub.test.ts @@ -0,0 +1,117 @@ +import { describe, expect, it } from "bun:test"; +import { maintainClosedSetSurfaceHub } from "../../../../src/commanders/textfresser/commands/generate/steps/maintain-closed-set-surface-hub"; +import type { CommandStateWithLemma } from "../../../../src/commanders/textfresser/commands/types"; +import type { SplitPathToMdFile } from "../../../../src/managers/obsidian/vault-action-manager/types/split-path"; +import { VaultActionKind } from "../../../../src/managers/obsidian/vault-action-manager/types/vault-action"; + +function makePath( + basename: string, + pathParts: string[], +): SplitPathToMdFile { + return { + basename, + extension: "md", + kind: "MdFile", + pathParts, + }; +} + +function makeCtx(params: { + activePath: SplitPathToMdFile; + isLibraryLookupAvailable: boolean; + lookupInLibrary: (name: string) => SplitPathToMdFile[]; + vamExists: (splitPath: SplitPathToMdFile) => boolean; +}): CommandStateWithLemma { + return { + actions: [], + commandContext: { + activeFile: { + content: "", + splitPath: params.activePath, + }, + selection: null, + }, + resultingActions: [], + textfresserState: { + isLibraryLookupAvailable: params.isLibraryLookupAvailable, + languages: { known: "English", target: "German" }, + latestLemmaResult: { + attestation: { + source: { + path: makePath("Source", ["Books"]), + ref: "![[Source#^1|^]]", + textRaw: "Die Frau ...", + textWithOnlyTargetMarked: "[Die] Frau ...", + }, + target: { + offsetInBlock: 0, + surface: "Die", + }, + }, + disambiguationResult: null, + lemma: "die-pronoun-de", + linguisticUnit: "Lexem", + posLikeKind: "Pronoun", + surfaceKind: "Lemma", + }, + lookupInLibrary: params.lookupInLibrary, + vam: { + exists: params.vamExists, + }, + }, + } as unknown as CommandStateWithLemma; +} + +describe("maintainClosedSetSurfaceHub", () => { + it("no-ops when library lookup is unavailable", () => { + const currentTarget = makePath("die-pronoun-de", [ + "Library", + "de", + "pronoun", + ]); + + const result = maintainClosedSetSurfaceHub( + makeCtx({ + activePath: currentTarget, + isLibraryLookupAvailable: false, + lookupInLibrary: () => [], + vamExists: () => true, + }), + ); + + expect(result.isOk()).toBe(true); + if (result.isErr()) { + throw new Error("Expected ok result"); + } + expect(result.value.actions).toHaveLength(0); + }); + + it("plans hub actions when lookup is available", () => { + const currentTarget = makePath("die-article-de", [ + "Library", + "de", + "article", + ]); + const otherTarget = makePath("die-pronoun-de", [ + "Library", + "de", + "pronoun", + ]); + + const result = maintainClosedSetSurfaceHub( + makeCtx({ + activePath: currentTarget, + isLibraryLookupAvailable: true, + lookupInLibrary: () => [otherTarget], + vamExists: () => false, + }), + ); + + expect(result.isOk()).toBe(true); + if (result.isErr()) { + throw new Error("Expected ok result"); + } + expect(result.value.actions).toHaveLength(1); + expect(result.value.actions[0]?.kind).toBe(VaultActionKind.UpsertMdFile); + }); +}); diff --git a/tests/unit/textfresser/steps/propagate-generated-sections.test.ts b/tests/unit/textfresser/steps/propagate-generated-sections.test.ts index b70ef5366..e9c5858e1 100644 --- a/tests/unit/textfresser/steps/propagate-generated-sections.test.ts +++ b/tests/unit/textfresser/steps/propagate-generated-sections.test.ts @@ -1,237 +1,116 @@ -import { afterAll, beforeEach, describe, expect, it, mock } from "bun:test"; -import { err, ok } from "neverthrow"; -import type { GenerateSectionsResult } from "../../../../src/commanders/textfresser/commands/generate/steps/generate-sections"; -import type { CommandInput } from "../../../../src/commanders/textfresser/commands/types"; -import { dispatchActions } from "../../../../src/commanders/textfresser/orchestration/shared/dispatch-actions"; -import type { VaultActionManager } from "../../../../src/managers/obsidian/vault-action-manager"; - -const calls = { - decorate: 0, - v2: 0, +import { describe, expect, it } from "bun:test"; +import type { + GenerateSectionsResult, + ParsedRelation, +} from "../../../../src/commanders/textfresser/commands/generate/steps/generate-sections"; +import { propagateGeneratedSections } from "../../../../src/commanders/textfresser/commands/generate/steps/propagate-generated-sections"; +import type { MorphemeItem } from "../../../../src/commanders/textfresser/domain/morpheme/morpheme-formatter"; +import type { TextfresserState } from "../../../../src/commanders/textfresser/state/textfresser-state"; +import { VaultActionKind } from "../../../../src/managers/obsidian/vault-action-manager/types/vault-action"; + +const SOURCE_PATH = { + basename: "chapter-1", + extension: "md" as const, + kind: "MdFile" as const, + pathParts: ["Reading"], }; -let shouldFailV2 = false; -let serializeCalls = 0; -let moveCalls = 0; - -function okCtx(ctx: unknown) { - return ok(ctx); -} - -mock.module( - "../../../../src/commanders/textfresser/commands/generate/steps/decorate-attestation-separability", - () => ({ - decorateAttestationSeparability: (ctx: unknown) => { - calls.decorate += 1; - return okCtx(ctx); - }, - }), -); - -mock.module( - "../../../../src/commanders/textfresser/commands/generate/steps/propagate-v2", - () => ({ - propagateV2: (ctx: unknown) => { - calls.v2 += 1; - if (shouldFailV2) { - return err({ - kind: "ApiError", - reason: "propagation v2 failed", - }); - } - return okCtx(ctx); - }, - }), -); - -mock.module( - "../../../../src/commanders/textfresser/commands/generate/steps/check-attestation", - () => ({ - checkAttestation: (state: T) => okCtx(state), - }), -); - -mock.module( - "../../../../src/commanders/textfresser/commands/generate/steps/check-eligibility", - () => ({ - checkEligibility: (state: T) => okCtx(state), - }), -); - -mock.module( - "../../../../src/commanders/textfresser/commands/generate/steps/check-lemma-result", - () => ({ - checkLemmaResult: (state: T) => okCtx(state), - }), -); - -mock.module( - "../../../../src/commanders/textfresser/commands/generate/steps/resolve-existing-entry", - () => ({ - resolveExistingEntry: (state: T) => okCtx(state), - }), -); - -mock.module( - "../../../../src/commanders/textfresser/commands/generate/steps/generate-sections", - () => ({ - generateSections: (state: T) => okCtx(state), - }), -); - -mock.module( - "../../../../src/commanders/textfresser/commands/generate/steps/serialize-entry", - () => ({ - serializeEntry: (state: T) => { - serializeCalls += 1; - return okCtx(state); - }, - }), -); - -mock.module( - "../../../../src/commanders/textfresser/commands/generate/steps/move-to-worter", - () => ({ - moveToWorter: (state: T) => { - moveCalls += 1; - return okCtx(state); - }, - }), -); - -function resetCalls() { - calls.decorate = 0; - calls.v2 = 0; -} - -function makeCtx(params: { - linguisticUnit?: "Lexem" | "Phrasem"; - posLikeKind?: string; - targetLanguage?: "German" | "English"; +function makeCtx(params?: { + morphemes?: MorphemeItem[]; + relations?: ParsedRelation[]; }): GenerateSectionsResult { - const posLikeKind = params.posLikeKind ?? "Noun"; - const linguisticUnit = params.linguisticUnit ?? "Lexem"; - const targetLanguage = params.targetLanguage ?? "German"; - return { - actions: [], - textfresserState: { - languages: { known: "English", target: targetLanguage }, - latestLemmaResult: { - attestation: { source: { ref: "![[src#^1|^]]" } }, - disambiguationResult: null, - lemma: "Haus", - linguisticUnit, - posLikeKind, - surfaceKind: "Lemma", - }, - }, - } as unknown as GenerateSectionsResult; -} - -function makeCommandInput(params: { - linguisticUnit?: "Lexem" | "Phrasem"; - posLikeKind?: string; - targetLanguage?: "German" | "English"; -}): CommandInput { - const posLikeKind = params.posLikeKind ?? "Noun"; - const linguisticUnit = params.linguisticUnit ?? "Lexem"; - const targetLanguage = params.targetLanguage ?? "German"; + const morphemes = params?.morphemes ?? []; + const relations = params?.relations ?? []; return { + actions: [], + allEntries: [], commandContext: { activeFile: { content: "", splitPath: { - basename: "gehen", + basename: "aufpassen", extension: "md", kind: "MdFile", - pathParts: ["Worter", "de"], + pathParts: ["Worter"], }, }, - selection: null, }, + existingEntries: [], + failedSections: [], + inflectionCells: [], + matchedEntry: null, + morphemes, + nextIndex: 1, + relations, resultingActions: [], textfresserState: { - languages: { known: "English", target: targetLanguage }, + languages: { known: "English", target: "German" }, latestLemmaResult: { - attestation: { source: { ref: "![[src#^1|^]]" } }, + attestation: { + source: { + path: SOURCE_PATH, + ref: "![[chapter-1#^1|^]]", + textRaw: "", + textWithOnlyTargetMarked: "", + }, + target: { surface: "aufpassen" }, + }, disambiguationResult: null, - lemma: "Haus", - linguisticUnit, - posLikeKind, + lemma: "aufpassen", + linguisticUnit: "Lexem", + posLikeKind: "Verb", surfaceKind: "Lemma", }, - }, - } as unknown as CommandInput; + lookupInLibrary: () => [], + vam: { findByBasename: () => [] }, + } as unknown as TextfresserState, + } as unknown as GenerateSectionsResult; } -const SAMPLE_SLICES: ReadonlyArray<{ - linguisticUnit: "Lexem" | "Phrasem"; - posLikeKind: string; - targetLanguage?: "German" | "English"; -}> = [ - { linguisticUnit: "Lexem", posLikeKind: "Noun", targetLanguage: "German" }, - { linguisticUnit: "Lexem", posLikeKind: "Verb", targetLanguage: "German" }, - { linguisticUnit: "Phrasem", posLikeKind: "Idiom", targetLanguage: "German" }, - { linguisticUnit: "Lexem", posLikeKind: "Noun", targetLanguage: "English" }, -]; - describe("propagateGeneratedSections", () => { - beforeEach(() => { - shouldFailV2 = false; - serializeCalls = 0; - moveCalls = 0; - resetCalls(); - }); - - afterAll(() => { - mock.restore(); - }); - - it("always routes core propagation through v2 and then decorates", async () => { - const { propagateGeneratedSections } = await import( - "../../../../src/commanders/textfresser/commands/generate/steps/propagate-generated-sections" + it("runs core propagation and the source-note post-step together", async () => { + const result = propagateGeneratedSections( + makeCtx({ + morphemes: [ + { + kind: "Prefix", + linkTarget: "auf-prefix-de", + separability: "Separable", + surf: "auf", + }, + ], + relations: [{ kind: "Synonym", words: ["Heim"] }], + }), ); - - for (const slice of SAMPLE_SLICES) { - resetCalls(); - const result = propagateGeneratedSections( - makeCtx({ - linguisticUnit: slice.linguisticUnit, - posLikeKind: slice.posLikeKind, - targetLanguage: slice.targetLanguage, - }), - ); - expect(result.isOk()).toBe(true); - expect(calls.v2).toBe(1); - expect(calls.decorate).toBe(1); - } - }); - - it("v2 failure short-circuits Generate with zero emitted/dispatched actions", async () => { - shouldFailV2 = true; - const { generateCommand } = await import( - "../../../../src/commanders/textfresser/commands/generate/generate-command" + expect(result.isOk()).toBe(true); + if (result.isErr()) return; + + const actions = result.value.actions; + const upsertCount = actions.filter( + (action) => action.kind === VaultActionKind.UpsertMdFile, + ).length; + expect(upsertCount).toBeGreaterThan(0); + + const sourceProcess = actions.find( + (action) => + action.kind === VaultActionKind.ProcessMdFile && + action.payload.splitPath.basename === SOURCE_PATH.basename && + action.payload.splitPath.pathParts.join("/") === + SOURCE_PATH.pathParts.join("/"), ); - let dispatchCalls = 0; - const vam = { - dispatch: async () => { - dispatchCalls += 1; - return ok(undefined); - }, - } as unknown as VaultActionManager; + expect(sourceProcess).toBeDefined(); + if (!sourceProcess || !("transform" in sourceProcess.payload)) return; - const result = await generateCommand( - makeCommandInput({ - posLikeKind: "Noun", - }), - ).andThen((actions) => dispatchActions(vam, actions)); + const sample = "[[aufpassen|Pass]] auf dich [[aufpassen|auf]]"; + const transformed = await sourceProcess.payload.transform(sample); + expect(transformed).toBe(sample); + }); - expect(result.isErr()).toBe(true); - expect(calls.v2).toBe(1); - expect(calls.decorate).toBe(0); - expect(serializeCalls).toBe(0); - expect(moveCalls).toBe(0); - expect(dispatchCalls).toBe(0); + it("is a no-op when neither propagation nor post-step conditions apply", () => { + const result = propagateGeneratedSections(makeCtx()); + expect(result.isOk()).toBe(true); + if (result.isErr()) return; + expect(result.value.actions).toHaveLength(0); }); }); diff --git a/tests/unit/textfresser/steps/propagate-inflections.test.ts b/tests/unit/textfresser/steps/propagate-inflections.test.ts index 29381dbf4..b6e478941 100644 --- a/tests/unit/textfresser/steps/propagate-inflections.test.ts +++ b/tests/unit/textfresser/steps/propagate-inflections.test.ts @@ -307,50 +307,6 @@ describe("propagateInflections", () => { } }); - it("auto-collapses legacy per-cell entries into new format entry", () => { - const cells: NounInflectionCell[] = [ - { - article: "die", - case: "Nominative", - form: "Kraftwerke", - number: "Plural", - }, - ]; - const ctx = makeCtx(cells); - const transform = getProcessTransform(ctx, "Kraftwerke"); - expect(transform).toBeDefined(); - if (!transform) return; - - const legacyEntries: DictEntry[] = [ - { - headerContent: "#Nominativ/Plural for: [[Kraftwerk]]", - id: "LX-IN-NOUN-1", - meta: {}, - sections: [], - }, - { - headerContent: "#Akkusativ/Plural for: [[Kraftwerk]]", - id: "LX-IN-NOUN-2", - meta: {}, - sections: [], - }, - ]; - const { body } = dictNoteHelper.serialize(legacyEntries); - const output = transform(body); - const entries = dictNoteHelper.parse(output); - - expect(entries).toHaveLength(1); - expect(entries[0]?.headerContent).toBe( - "#Inflection/Noun/Maskulin for: [[Kraftwerk]]", - ); - const firstEntry = entries[0]; - if (firstEntry) { - expect(getTagsContent(firstEntry)).toBe( - "#Nominativ/Plural #Akkusativ/Plural", - ); - } - }); - it("propagates with fallback header when noun genus is unresolved", () => { const cells: NounInflectionCell[] = [ { diff --git a/tests/unit/textfresser/steps/propagate-v2-phase4.test.ts b/tests/unit/textfresser/steps/propagate-v2-phase4.test.ts deleted file mode 100644 index d8f20211b..000000000 --- a/tests/unit/textfresser/steps/propagate-v2-phase4.test.ts +++ /dev/null @@ -1,917 +0,0 @@ -import { describe, expect, it } from "bun:test"; -import type { GenerateSectionsResult } from "../../../../src/commanders/textfresser/commands/generate/steps/generate-sections"; -import { propagateGeneratedSections } from "../../../../src/commanders/textfresser/commands/generate/steps/propagate-generated-sections"; -import { propagateInflections } from "../../../../src/commanders/textfresser/commands/generate/steps/propagate-inflections"; -import { propagateMorphemes } from "../../../../src/commanders/textfresser/commands/generate/steps/propagate-morphemes"; -import { propagateMorphologyRelations } from "../../../../src/commanders/textfresser/commands/generate/steps/propagate-morphology-relations"; -import { propagateRelations } from "../../../../src/commanders/textfresser/commands/generate/steps/propagate-relations"; -import { - foldScopedActionsToSingleWritePerTarget, - propagateV2, -} from "../../../../src/commanders/textfresser/commands/generate/steps/propagate-v2"; -import { dictNoteHelper } from "../../../../src/commanders/textfresser/domain/dict-note"; -import type { DictEntry } from "../../../../src/commanders/textfresser/domain/dict-note/types"; -import { parsePropagationNote } from "../../../../src/commanders/textfresser/domain/propagation"; -import { cssSuffixFor } from "../../../../src/commanders/textfresser/targets/de/sections/section-css-kind"; -import { - DictSectionKind, - TitleReprFor, -} from "../../../../src/commanders/textfresser/targets/de/sections/section-kind"; -import { type NounInflectionCell } from "../../../../src/linguistics/de/lexem/noun"; -import { - makeSystemPathForSplitPath, - type VaultAction, - VaultActionKind, -} from "../../../../src/managers/obsidian/vault-action-manager"; -import { - SplitPathKind, - type SplitPathToMdFile, -} from "../../../../src/managers/obsidian/vault-action-manager/types/split-path"; - -type InMemoryFile = { - content: string; - splitPath: SplitPathToMdFile; -}; - -type InMemoryVault = Map; - -function makeSplitPath(params: { - basename: string; - surfaceKind: "inflected" | "lemma"; - unitKind: "lexem" | "morphem" | "phrasem"; -}): SplitPathToMdFile { - const normalized = params.basename.toLowerCase(); - const shard1 = normalized.slice(0, 1) || "_"; - const shard2 = normalized.slice(0, 3) || normalized; - const shard3 = normalized.slice(0, 5) || normalized; - - return { - basename: params.basename, - extension: "md", - kind: SplitPathKind.MdFile, - pathParts: [ - "Worter", - "de", - params.unitKind, - params.surfaceKind, - shard1, - shard2, - shard3, - ], - }; -} - -function keyFor(splitPath: SplitPathToMdFile): string { - return makeSystemPathForSplitPath(splitPath); -} - -function setFile(vault: InMemoryVault, file: InMemoryFile): void { - vault.set(keyFor(file.splitPath), file); -} - -function createStructuredTargetNote(params: { - header: string; - id: string; - includeRelation?: boolean; - includeTags?: boolean; - includeUsedInMarker?: boolean; -}): string { - const sections: DictEntry["sections"] = []; - if (params.includeRelation ?? true) { - sections.push({ - content: "", - kind: cssSuffixFor[DictSectionKind.Relation], - title: TitleReprFor[DictSectionKind.Relation].German, - }); - } - sections.push({ - content: params.includeUsedInMarker ? "Verwendet in:\n[[alt]]" : "", - kind: cssSuffixFor[DictSectionKind.Morphology], - title: TitleReprFor[DictSectionKind.Morphology].German, - }); - if (params.includeTags ?? false) { - sections.push({ - content: "#seed", - kind: cssSuffixFor[DictSectionKind.Tags], - title: TitleReprFor[DictSectionKind.Tags].German, - }); - } - - const entry: DictEntry = { - headerContent: params.header, - id: params.id, - meta: {}, - sections, - }; - - return dictNoteHelper.serialize([entry]).body; -} - -function createSeedVault(): InMemoryVault { - const vault: InMemoryVault = new Map(); - - const kohlePath = makeSplitPath({ - basename: "Kohle", - surfaceKind: "lemma", - unitKind: "lexem", - }); - setFile(vault, { - content: createStructuredTargetNote({ - header: "Kohle", - id: "LX-LM-NOUN-1", - includeTags: true, - includeUsedInMarker: true, - }), - splitPath: kohlePath, - }); - - const kraftPath = makeSplitPath({ - basename: "Kraft", - surfaceKind: "lemma", - unitKind: "lexem", - }); - setFile(vault, { - content: createStructuredTargetNote({ - header: "Kraft", - id: "LX-LM-NOUN-1", - includeUsedInMarker: true, - }), - splitPath: kraftPath, - }); - - return vault; -} - -function makeFindByBasename(vault: InMemoryVault): (basename: string) => SplitPathToMdFile[] { - return (basename: string) => { - const matches: SplitPathToMdFile[] = []; - for (const file of vault.values()) { - if (file.splitPath.basename === basename) { - matches.push(file.splitPath); - } - } - return matches; - }; -} - -function makeNounFixtureCtx(vault: InMemoryVault): GenerateSectionsResult { - const inflectionCells: NounInflectionCell[] = [ - { - article: "die", - case: "Nominative", - form: "Kohlekraftwerke", - number: "Plural", - }, - { - article: "der", - case: "Genitive", - form: "Kohlekraftwerke", - number: "Plural", - }, - ]; - - return { - actions: [], - allEntries: [], - commandContext: { - activeFile: { - content: "", - splitPath: makeSplitPath({ - basename: "Kohlekraftwerk", - surfaceKind: "lemma", - unitKind: "lexem", - }), - }, - }, - existingEntries: [], - failedSections: [], - inflectionCells, - matchedEntry: null, - morphemes: [ - { kind: "Prefix", surf: "Ur" }, - { kind: "Root", lemma: "Kohle", surf: "kohle" }, - { kind: "Root", lemma: "Kraft", surf: "kraft" }, - ], - morphology: { - compoundedFromLemmas: ["Kohle", "Kraft"], - }, - nextIndex: 1, - nounInflectionGenus: "Neutrum", - relations: [{ kind: "Synonym", words: ["Kohle"] }], - resultingActions: [], - sourceTranslation: "coal power station", - textfresserState: { - languages: { known: "English", target: "German" }, - latestLemmaResult: { - attestation: { - source: { - path: makeSplitPath({ - basename: "Reading", - surfaceKind: "lemma", - unitKind: "lexem", - }), - ref: "![[Reading#^1|^]]", - }, - }, - disambiguationResult: null, - lemma: "Kohlekraftwerk", - linguisticUnit: "Lexem", - posLikeKind: "Noun", - surfaceKind: "Lemma", - }, - lookupInLibrary: () => [], - vam: { - findByBasename: makeFindByBasename(vault), - }, - }, - } as unknown as GenerateSectionsResult; -} - -type Phase5SliceFixture = { - linguisticUnit: "Lexem" | "Phrasem"; - posLikeKind: string; - sliceKey: string; -}; - -const PHASE5_NON_VERB_SLICES: ReadonlyArray = [ - { - linguisticUnit: "Lexem", - posLikeKind: "Adjective", - sliceKey: "de/lexem/adjective", - }, - { linguisticUnit: "Lexem", posLikeKind: "Adverb", sliceKey: "de/lexem/adverb" }, - { linguisticUnit: "Lexem", posLikeKind: "Article", sliceKey: "de/lexem/article" }, - { - linguisticUnit: "Lexem", - posLikeKind: "Conjunction", - sliceKey: "de/lexem/conjunction", - }, - { - linguisticUnit: "Lexem", - posLikeKind: "InteractionalUnit", - sliceKey: "de/lexem/interactionalunit", - }, - { linguisticUnit: "Lexem", posLikeKind: "Particle", sliceKey: "de/lexem/particle" }, - { - linguisticUnit: "Lexem", - posLikeKind: "Preposition", - sliceKey: "de/lexem/preposition", - }, - { linguisticUnit: "Lexem", posLikeKind: "Pronoun", sliceKey: "de/lexem/pronoun" }, - { - linguisticUnit: "Phrasem", - posLikeKind: "Collocation", - sliceKey: "de/phrasem/collocation", - }, - { - linguisticUnit: "Phrasem", - posLikeKind: "CulturalQuotation", - sliceKey: "de/phrasem/culturalquotation", - }, - { - linguisticUnit: "Phrasem", - posLikeKind: "DiscourseFormula", - sliceKey: "de/phrasem/discourseformula", - }, - { linguisticUnit: "Phrasem", posLikeKind: "Idiom", sliceKey: "de/phrasem/idiom" }, - { linguisticUnit: "Phrasem", posLikeKind: "Proverb", sliceKey: "de/phrasem/proverb" }, -]; - -function makePhase5NonVerbFixtureCtx( - vault: InMemoryVault, - slice: Phase5SliceFixture, -): GenerateSectionsResult { - const isPhrasem = slice.linguisticUnit === "Phrasem"; - return { - actions: [], - allEntries: [], - commandContext: { - activeFile: { - content: "", - splitPath: makeSplitPath({ - basename: isPhrasem ? "Auf jeden Fall" : "Langsam", - surfaceKind: "lemma", - unitKind: isPhrasem ? "phrasem" : "lexem", - }), - }, - }, - existingEntries: [], - failedSections: [], - inflectionCells: [], - matchedEntry: null, - morphemes: [], - morphology: { - compoundedFromLemmas: [], - }, - nextIndex: 1, - nounInflectionGenus: undefined, - relations: [{ kind: "Synonym", words: ["Kohle"] }], - resultingActions: [], - sourceTranslation: "fixture translation", - textfresserState: { - languages: { known: "English", target: "German" }, - latestLemmaResult: { - attestation: { - source: { - path: makeSplitPath({ - basename: "Reading", - surfaceKind: "lemma", - unitKind: "lexem", - }), - ref: "![[Reading#^1|^]]", - }, - }, - disambiguationResult: null, - lemma: isPhrasem ? "Auf jeden Fall" : "Langsam", - linguisticUnit: slice.linguisticUnit, - posLikeKind: slice.posLikeKind, - surfaceKind: "Lemma", - }, - lookupInLibrary: () => [], - vam: { - findByBasename: makeFindByBasename(vault), - }, - }, - } as unknown as GenerateSectionsResult; -} - -function makeVerbFixtureCtx(vault: InMemoryVault): GenerateSectionsResult { - return { - actions: [], - allEntries: [], - commandContext: { - activeFile: { - content: "", - splitPath: makeSplitPath({ - basename: "Aufpassen", - surfaceKind: "lemma", - unitKind: "lexem", - }), - }, - }, - existingEntries: [], - failedSections: [], - inflectionCells: [], - matchedEntry: null, - morphemes: [ - { kind: "Prefix", separability: "Separable", surf: "auf" }, - { kind: "Root", lemma: "passen", surf: "pass" }, - ], - morphology: { - compoundedFromLemmas: ["Kohle"], - derivedFromLemma: "Kraft", - prefixEquation: { - baseLemma: "passen", - prefixDisplay: "auf", - prefixTarget: "auf", - sourceLemma: "Aufpassen", - }, - }, - nextIndex: 1, - nounInflectionGenus: undefined, - relations: [{ kind: "Synonym", words: ["Kohle"] }], - resultingActions: [], - sourceTranslation: "to pay attention", - textfresserState: { - languages: { known: "English", target: "German" }, - latestLemmaResult: { - attestation: { - source: { - path: makeSplitPath({ - basename: "Reading", - surfaceKind: "lemma", - unitKind: "lexem", - }), - ref: "![[Reading#^1|^]]", - }, - }, - disambiguationResult: null, - lemma: "Aufpassen", - linguisticUnit: "Lexem", - posLikeKind: "Verb", - surfaceKind: "Lemma", - }, - lookupInLibrary: () => [], - vam: { - findByBasename: makeFindByBasename(vault), - }, - }, - } as unknown as GenerateSectionsResult; -} - -async function resolveProcessAfterContent( - action: Extract, - before: string, -): Promise { - if ("transform" in action.payload) { - const after = action.payload.transform(before); - return typeof after === "string" ? after : await after; - } - return action.payload.after; -} - -async function applyActionsToVault(params: { - actions: ReadonlyArray; - vault: InMemoryVault; -}): Promise<{ changedPaths: Set }> { - const changedPaths = new Set(); - - for (const action of params.actions) { - if (action.kind === VaultActionKind.RenameMdFile) { - const fromKey = keyFor(action.payload.from); - const toKey = keyFor(action.payload.to); - const existing = params.vault.get(fromKey); - params.vault.delete(fromKey); - params.vault.set(toKey, { - content: existing?.content ?? "", - splitPath: action.payload.to, - }); - continue; - } - - if (action.kind === VaultActionKind.UpsertMdFile) { - const pathKey = keyFor(action.payload.splitPath); - const existing = params.vault.get(pathKey); - if (!existing) { - params.vault.set(pathKey, { - content: - typeof action.payload.content === "string" - ? action.payload.content - : "", - splitPath: action.payload.splitPath, - }); - } else if (typeof action.payload.content === "string") { - existing.content = action.payload.content; - } - continue; - } - - if (action.kind === VaultActionKind.ProcessMdFile) { - const pathKey = keyFor(action.payload.splitPath); - const before = params.vault.get(pathKey)?.content ?? ""; - const after = await resolveProcessAfterContent(action, before); - params.vault.set(pathKey, { - content: after, - splitPath: action.payload.splitPath, - }); - if (after !== before) { - changedPaths.add(pathKey); - } - continue; - } - } - - return { changedPaths }; -} - -function buildDtoSnapshot(vault: InMemoryVault): Record { - const snapshot: Record = {}; - const sortedPaths = [...vault.keys()].sort((left, right) => - left.localeCompare(right), - ); - for (const path of sortedPaths) { - const file = vault.get(path); - if (!file) { - continue; - } - const parsed = parsePropagationNote(file.content); - if (parsed.length === 0) { - continue; - } - snapshot[path] = parsed; - } - return snapshot; -} - -function buildMutationKindSet(vault: InMemoryVault): Set { - const set = new Set(); - for (const [path, file] of vault.entries()) { - const parsed = parsePropagationNote(file.content); - for (const entry of parsed) { - if (entry.headerContent.startsWith("#Inflection/")) { - set.add(`${path}|Inflection`); - } - for (const section of entry.sections) { - if (section.kind === "Raw") { - continue; - } - set.add(`${path}|${section.kind}`); - } - } - } - return set; -} - -function buildProcessWriteCountByTarget( - actions: ReadonlyArray, -): Map { - const counts = new Map(); - for (const action of actions) { - if (action.kind !== VaultActionKind.ProcessMdFile) { - continue; - } - const key = keyFor(action.payload.splitPath); - const next = (counts.get(key) ?? 0) + 1; - counts.set(key, next); - } - return counts; -} - -function runLegacyPropagationForParity( - ctx: GenerateSectionsResult, -) { - return propagateRelations(ctx) - .andThen(propagateMorphologyRelations) - .andThen(propagateMorphemes) - .andThen(propagateInflections); -} - -describe("propagation v2 phase 4 noun slice", () => { - it("keeps semantic DTO parity with legacy v1 on curated noun fixture", async () => { - const legacyVault = createSeedVault(); - const v2Vault = createSeedVault(); - - const legacyResult = runLegacyPropagationForParity( - makeNounFixtureCtx(legacyVault), - ); - const v2Result = propagateV2(makeNounFixtureCtx(v2Vault)); - - expect(legacyResult.isOk()).toBe(true); - expect(v2Result.isOk()).toBe(true); - if (legacyResult.isErr() || v2Result.isErr()) { - return; - } - - await applyActionsToVault({ - actions: legacyResult.value.actions, - vault: legacyVault, - }); - await applyActionsToVault({ - actions: v2Result.value.actions, - vault: v2Vault, - }); - - expect(buildDtoSnapshot(v2Vault)).toEqual(buildDtoSnapshot(legacyVault)); - }); - - it("is idempotent: second v2 run produces zero changed-target writes", async () => { - const vault = createSeedVault(); - const firstRun = propagateV2(makeNounFixtureCtx(vault)); - expect(firstRun.isOk()).toBe(true); - if (firstRun.isErr()) { - return; - } - - const firstApply = await applyActionsToVault({ - actions: firstRun.value.actions, - vault, - }); - expect(firstApply.changedPaths.size).toBeGreaterThan(0); - - const secondRun = propagateV2(makeNounFixtureCtx(vault)); - expect(secondRun.isOk()).toBe(true); - if (secondRun.isErr()) { - return; - } - const secondApply = await applyActionsToVault({ - actions: secondRun.value.actions, - vault, - }); - - expect(secondApply.changedPaths.size).toBe(0); - }); - - it("enforces one write action per target note on v2 path", () => { - const result = propagateV2(makeNounFixtureCtx(createSeedVault())); - expect(result.isOk()).toBe(true); - if (result.isErr()) { - return; - } - - const processCounts = buildProcessWriteCountByTarget(result.value.actions); - for (const count of processCounts.values()) { - expect(count).toBe(1); - } - }); - - it("fails fast on unsupported scoped actions (all-or-nothing fold gate)", () => { - const result = foldScopedActionsToSingleWritePerTarget([ - { - kind: VaultActionKind.TrashMdFile, - payload: { - splitPath: makeSplitPath({ - basename: "Broken", - surfaceKind: "lemma", - unitKind: "lexem", - }), - }, - }, - ]); - - expect(result.isErr()).toBe(true); - }); - - it("supports ProcessMdFile before/after payload by normalizing it to a transform", async () => { - const splitPath = makeSplitPath({ - basename: "ShapeMismatch", - surfaceKind: "lemma", - unitKind: "lexem", - }); - const result = foldScopedActionsToSingleWritePerTarget([ - { - kind: VaultActionKind.ProcessMdFile, - payload: { - after: "after", - before: "before", - splitPath, - }, - }, - ]); - - expect(result.isOk()).toBe(true); - if (result.isErr()) { - return; - } - - const vault = new Map(); - setFile(vault, { - content: "before + before", - splitPath, - }); - await applyActionsToVault({ - actions: result.value, - vault, - }); - expect(vault.get(keyFor(splitPath))?.content).toBe("after + before"); - }); - - it("supports UpsertMdFile non-null content with deterministic transform order", async () => { - const splitPath = makeSplitPath({ - basename: "NonNullUpsert", - surfaceKind: "lemma", - unitKind: "lexem", - }); - const result = foldScopedActionsToSingleWritePerTarget([ - { - kind: VaultActionKind.UpsertMdFile, - payload: { - content: "#seed", - splitPath, - }, - }, - { - kind: VaultActionKind.ProcessMdFile, - payload: { - splitPath, - transform: (content: string) => `${content}\n#tail`, - }, - }, - ]); - - expect(result.isOk()).toBe(true); - if (result.isErr()) { - return; - } - - const vault = new Map(); - await applyActionsToVault({ - actions: result.value, - vault, - }); - expect(vault.get(keyFor(splitPath))?.content).toBe("#seed\n#tail"); - }); - - it("matches legacy order-insensitive target+mutation-kind set", async () => { - const legacyVault = createSeedVault(); - const v2Vault = createSeedVault(); - - const legacyResult = runLegacyPropagationForParity( - makeNounFixtureCtx(legacyVault), - ); - const v2Result = propagateV2(makeNounFixtureCtx(v2Vault)); - - expect(legacyResult.isOk()).toBe(true); - expect(v2Result.isOk()).toBe(true); - if (legacyResult.isErr() || v2Result.isErr()) { - return; - } - - await applyActionsToVault({ - actions: legacyResult.value.actions, - vault: legacyVault, - }); - await applyActionsToVault({ - actions: v2Result.value.actions, - vault: v2Vault, - }); - - expect(buildMutationKindSet(v2Vault)).toEqual( - buildMutationKindSet(legacyVault), - ); - }); -}); - -describe("propagation v2 phase 5 non-verb slices", () => { - it("keeps semantic DTO parity with legacy v1 across migrated non-verb slices", async () => { - for (const slice of PHASE5_NON_VERB_SLICES) { - const legacyVault = createSeedVault(); - const v2Vault = createSeedVault(); - const legacyResult = runLegacyPropagationForParity( - makePhase5NonVerbFixtureCtx(legacyVault, slice), - ); - const v2Result = propagateV2(makePhase5NonVerbFixtureCtx(v2Vault, slice)); - - expect(legacyResult.isOk()).toBe(true); - expect(v2Result.isOk()).toBe(true); - if (legacyResult.isErr() || v2Result.isErr()) { - return; - } - - await applyActionsToVault({ - actions: legacyResult.value.actions, - vault: legacyVault, - }); - await applyActionsToVault({ - actions: v2Result.value.actions, - vault: v2Vault, - }); - - expect(buildDtoSnapshot(v2Vault)).toEqual(buildDtoSnapshot(legacyVault)); - } - }); - - it("is idempotent for migrated non-verb slices", async () => { - for (const slice of PHASE5_NON_VERB_SLICES) { - const vault = createSeedVault(); - const firstRun = propagateV2(makePhase5NonVerbFixtureCtx(vault, slice)); - expect(firstRun.isOk()).toBe(true); - if (firstRun.isErr()) { - return; - } - - const firstApply = await applyActionsToVault({ - actions: firstRun.value.actions, - vault, - }); - expect(firstApply.changedPaths.size).toBeGreaterThan(0); - - const secondRun = propagateV2(makePhase5NonVerbFixtureCtx(vault, slice)); - expect(secondRun.isOk()).toBe(true); - if (secondRun.isErr()) { - return; - } - const secondApply = await applyActionsToVault({ - actions: secondRun.value.actions, - vault, - }); - - expect(secondApply.changedPaths.size).toBe(0); - } - }); - - it("enforces one write action per target note for migrated non-verb slices", () => { - for (const slice of PHASE5_NON_VERB_SLICES) { - const result = propagateV2( - makePhase5NonVerbFixtureCtx(createSeedVault(), slice), - ); - expect(result.isOk()).toBe(true); - if (result.isErr()) { - return; - } - - const processCounts = buildProcessWriteCountByTarget(result.value.actions); - for (const count of processCounts.values()) { - expect(count).toBe(1); - } - } - }); -}); - -describe("propagation v2 phase 5 verb slice", () => { - it("writes source-note separability-decoration action on wrapper v2 route", async () => { - const sourcePath = makeSplitPath({ - basename: "Reading", - surfaceKind: "lemma", - unitKind: "lexem", - }); - const sourceContent = - "Beide Spannen bleiben erhalten: [[Aufpassen|Pass]] ... [[Aufpassen|auf]]."; - const vault = createSeedVault(); - setFile(vault, { content: sourceContent, splitPath: sourcePath }); - const result = propagateGeneratedSections(makeVerbFixtureCtx(vault)); - expect(result.isOk()).toBe(true); - if (result.isErr()) { - return; - } - - const sourceKey = keyFor(sourcePath); - const hasSourceProcess = result.value.actions.some( - (action) => - action.kind === VaultActionKind.ProcessMdFile && - keyFor(action.payload.splitPath) === sourceKey, - ); - expect(hasSourceProcess).toBe(true); - - await applyActionsToVault({ - actions: result.value.actions, - vault, - }); - expect(vault.get(sourceKey)?.content).toEqual(sourceContent); - }); - - it("keeps semantic DTO parity with legacy v1 on curated verb fixture", async () => { - const legacyVault = createSeedVault(); - const v2Vault = createSeedVault(); - - const legacyResult = runLegacyPropagationForParity( - makeVerbFixtureCtx(legacyVault), - ); - const v2Result = propagateV2(makeVerbFixtureCtx(v2Vault)); - - expect(legacyResult.isOk()).toBe(true); - expect(v2Result.isOk()).toBe(true); - if (legacyResult.isErr() || v2Result.isErr()) { - return; - } - - await applyActionsToVault({ - actions: legacyResult.value.actions, - vault: legacyVault, - }); - await applyActionsToVault({ - actions: v2Result.value.actions, - vault: v2Vault, - }); - - expect(buildDtoSnapshot(v2Vault)).toEqual(buildDtoSnapshot(legacyVault)); - }); - - it("is idempotent for verb slice", async () => { - const vault = createSeedVault(); - const firstRun = propagateV2(makeVerbFixtureCtx(vault)); - expect(firstRun.isOk()).toBe(true); - if (firstRun.isErr()) { - return; - } - - const firstApply = await applyActionsToVault({ - actions: firstRun.value.actions, - vault, - }); - expect(firstApply.changedPaths.size).toBeGreaterThan(0); - - const secondRun = propagateV2(makeVerbFixtureCtx(vault)); - expect(secondRun.isOk()).toBe(true); - if (secondRun.isErr()) { - return; - } - const secondApply = await applyActionsToVault({ - actions: secondRun.value.actions, - vault, - }); - - expect(secondApply.changedPaths.size).toBe(0); - }); - - it("enforces one write action per target note for verb slice", () => { - const result = propagateV2(makeVerbFixtureCtx(createSeedVault())); - expect(result.isOk()).toBe(true); - if (result.isErr()) { - return; - } - - const processCounts = buildProcessWriteCountByTarget(result.value.actions); - for (const count of processCounts.values()) { - expect(count).toBe(1); - } - }); - - it("matches legacy order-insensitive target+mutation-kind set for verb slice", async () => { - const legacyVault = createSeedVault(); - const v2Vault = createSeedVault(); - - const legacyResult = runLegacyPropagationForParity( - makeVerbFixtureCtx(legacyVault), - ); - const v2Result = propagateV2(makeVerbFixtureCtx(v2Vault)); - - expect(legacyResult.isOk()).toBe(true); - expect(v2Result.isOk()).toBe(true); - if (legacyResult.isErr() || v2Result.isErr()) { - return; - } - - await applyActionsToVault({ - actions: legacyResult.value.actions, - vault: legacyVault, - }); - await applyActionsToVault({ - actions: v2Result.value.actions, - vault: v2Vault, - }); - - expect(buildMutationKindSet(v2Vault)).toEqual( - buildMutationKindSet(legacyVault), - ); - }); -}); diff --git a/tests/unit/textfresser/steps/propagation-v2-ports-adapter.test.ts b/tests/unit/textfresser/steps/propagation-ports-adapter.test.ts similarity index 92% rename from tests/unit/textfresser/steps/propagation-v2-ports-adapter.test.ts rename to tests/unit/textfresser/steps/propagation-ports-adapter.test.ts index 3a36715ba..49a1091ac 100644 --- a/tests/unit/textfresser/steps/propagation-v2-ports-adapter.test.ts +++ b/tests/unit/textfresser/steps/propagation-ports-adapter.test.ts @@ -1,8 +1,8 @@ import { describe, expect, it } from "bun:test"; import { err, ok } from "neverthrow"; import { - createPropagationV2PortsAdapter, -} from "../../../../src/commanders/textfresser/commands/generate/steps/propagation-v2-ports-adapter"; + createPropagationPortsAdapter, +} from "../../../../src/commanders/textfresser/commands/generate/steps/propagation-ports-adapter"; import { ReadContentErrorKind, type VaultActionManager, @@ -52,7 +52,7 @@ function permissionReadError(reason: string) { } as const; } -describe("propagation-v2-ports-adapter", () => { +describe("propagation-ports-adapter", () => { it("readManyMdFiles deduplicates input and preserves deterministic first-seen order", async () => { const alpha = makePath("alpha"); const beta = makePath("beta"); @@ -71,7 +71,7 @@ describe("propagation-v2-ports-adapter", () => { }, }; - const ports = createPropagationV2PortsAdapter({ + const ports = createPropagationPortsAdapter({ lookupInLibraryByCoreName: () => [], vam, }); @@ -105,7 +105,7 @@ describe("propagation-v2-ports-adapter", () => { }, }; - const ports = createPropagationV2PortsAdapter({ + const ports = createPropagationPortsAdapter({ lookupInLibraryByCoreName: () => [], vam, }); @@ -129,7 +129,7 @@ describe("propagation-v2-ports-adapter", () => { err(fileNotFoundError("File not found: alpha")), }; - const ports = createPropagationV2PortsAdapter({ + const ports = createPropagationPortsAdapter({ lookupInLibraryByCoreName: () => [], vam, }); @@ -151,7 +151,7 @@ describe("propagation-v2-ports-adapter", () => { readContent: async () => err(unknownReadError("random io issue")), }; - const ports = createPropagationV2PortsAdapter({ + const ports = createPropagationPortsAdapter({ lookupInLibraryByCoreName: () => [], vam, }); @@ -170,7 +170,7 @@ describe("propagation-v2-ports-adapter", () => { err(permissionReadError("permission denied")), }; - const ports = createPropagationV2PortsAdapter({ + const ports = createPropagationPortsAdapter({ lookupInLibraryByCoreName: () => [], vam, }); @@ -194,7 +194,7 @@ describe("propagation-v2-ports-adapter", () => { findByBasename: () => [], readContent: async () => err(unknownReadError("unreachable")), }; - const portsMissing = createPropagationV2PortsAdapter({ + const portsMissing = createPropagationPortsAdapter({ lookupInLibraryByCoreName: () => [], vam: vamMissing, }); @@ -209,7 +209,7 @@ describe("propagation-v2-ports-adapter", () => { readContent: async () => err(fileNotFoundError("File not found: beta")), }; - const portsRace = createPropagationV2PortsAdapter({ + const portsRace = createPropagationPortsAdapter({ lookupInLibraryByCoreName: () => [], vam: vamRace, }); @@ -229,7 +229,7 @@ describe("propagation-v2-ports-adapter", () => { err(permissionReadError("permission denied")), }; - const ports = createPropagationV2PortsAdapter({ + const ports = createPropagationPortsAdapter({ lookupInLibraryByCoreName: () => [], vam, }); @@ -262,7 +262,7 @@ describe("propagation-v2-ports-adapter", () => { readContent: async () => ok(""), }; - const ports = createPropagationV2PortsAdapter({ + const ports = createPropagationPortsAdapter({ lookupInLibraryByCoreName: () => [shared, fromLibraryOnly], vam, }); @@ -282,7 +282,7 @@ describe("propagation-v2-ports-adapter", () => { it("buildTargetWriteActions returns sync upsert/process action pair", () => { const alpha = makePath("alpha"); - const ports = createPropagationV2PortsAdapter({ + const ports = createPropagationPortsAdapter({ lookupInLibraryByCoreName: () => [], vam: { exists: () => true,