feat: add FTS5 full-text search index

[andyhtran]

Summary

• Replace brute-force JSONL streaming search with a SQLite FTS5 index (15-72x faster, 76-90% less memory)
• Add cct index sync/rebuild/status commands for index management
• Add --sort relevance and --sync flags to search command
• Incremental sync with change detection, corruption auto-recovery, and Porter stemmer

Details

The old search scanned all JSONL session files on every query (~2s for ~1200 sessions). The new approach builds an FTS5 index in the XDG cache directory (~/.cache/cct/index.db) with automatic incremental sync. Searches now complete in 50-140ms.

Key design decisions:

• Contentless FTS5 — stores byte offsets into original JSONL files rather than duplicating text, keeping the index compact
• Streaming search preserved — the -s (single-session) flag still uses the old streaming path since indexing one file is unnecessary overhead
• Substring fallback — compound terms like pre-commit that FTS5 tokenizes on punctuation fall back to substring scan when FTS returns zero results
• 5-minute sync cache — skips filesystem scan if synced recently; --sync flag bypasses this

The "recall gap" between old substring matching and FTS5 was investigated across sessions and found to be entirely false positives (e.g. "bug" matching inside "debug").

Test plan

• Verified just ci passes (format, lint, tests)
• Smoke tested all CLI commands (list, search, info, stats, export, changelog, plans, resume, index sync/rebuild/status)
• Verified incremental sync detects new/updated/deleted sessions correctly
• Verified --sort relevance surfaces sessions where the term is central vs incidental
• Verified project filter messages distinguish nonexistent project from no query matches
• Verified search with -s flag still uses streaming path
• Verified corruption recovery recreates index from corrupt DB file

🤖 Generated with Claude Code