fix: prevent OOM crashes during init on large codebases

[mechtar-ru]

Summary

Two fixes to prevent out-of-memory crashes when running engram init on large projects:

1. MAX_DEPTH limit in extractDirectory (ast-miner.ts) — prevents stack overflow on deep directory trees
2. MAX_FILES_PER_COMMIT limit in mineGitHistory (git-miner.ts) — prevents O(n²) memory explosion on commits with many files

Changes

ast-miner.ts

• Added MAX_DEPTH = 100 depth limit to recursive walk() function
• Wrapped readdirSync in try-catch to skip unreadable directories
• Added .engramignore support for custom exclusions
• Expanded default exclusions: target, .venv, .next, .nuxt, .output, coverage, .turbo, .cache

git-miner.ts

• Added MAX_FILES_PER_COMMIT = 50 to prevent O(n²) co-change pair explosion
• Skip dist/, build/ directories from co-change analysis

Testing

Tested on Axolotl project (2.2GB, 34K files) — now completes successfully in ~600ms.

[NickCirv]

Thanks for the diagnosis + PR — the git-miner O(n²) co-change explosion was spot-on, and the expanded IGNORED_DIRS is gold for modern monorepos.

CI is failing on all 4 matrix configs with the same 5 TypeScript errors in src/miners/ast-miner.ts:

error TS2322: Type 'Dirent<string>[]' is not assignable to type 'Dirent<NonSharedBuffer>[]'.
error TS2339: Property 'startsWith' does not exist on type 'NonSharedBuffer'.

This is the Node 25 strict-types issue that was already fixed for skills-miner.ts in v0.5.3 — readdirSync now returns Dirent<NonSharedBuffer> when encoding is omitted, which breaks all downstream string operations on .name.

One-line fix: pin encoding on the readdirSync call (same pattern as skills-miner.ts:226-229):

const entries = readdirSync(dir, { withFileTypes: true, encoding: "utf-8" });

Push that and CI should go green.

One other ask before merge: would you add a quick regression test? Two would be enough:

1. tests/git-miner.test.ts — fixture git log with a 51-file commit, assert co-change pairs don't grow unbounded.
2. tests/ast-miner.test.ts — fixture dir nested 101 levels, assert no throw + walk stops.

The fix is correct (manual verification on Axolotl + the root-cause analysis), I just want the limits locked in against future regressions. Happy to merge as soon as CI is green and tests are in.

On scope: .engramignore support is a welcome bonus — no need to split that out.

One note on our earlier comment on #5 asking you for du -sh ~/.claude/skills/ — wrong question on our part; you found the real cause faster by profiling directly. Nice work.

[mechtar-ru]

added tests