lore-engine

Pure Rust wiki engine. SQLite FTS5 full-text search, petgraph link graph, wiki link parsing, file watching. No GUI dependencies.

What it is

A framework-agnostic engine for building personal wiki / knowledge base applications. You provide a folder of Markdown files; lore-engine gives you:

• Page CRUD with automatic wiki link resolution
• Full-text search via SQLite FTS5 with BM25 ranking
• Link graph — backlinks, forward links, orphan detection
• Wiki link parsing — [[Page Name]] and [[Page Name|alias]] with pulldown-cmark code detection
• Incremental sync — only processes files that changed (content-hash diffing)
• File watching — optional, picks up external edits in real time
• Placeholder pages — unresolved links create placeholders automatically

Who it's for

Anyone building a wiki, note-taking app, or knowledge management tool in Rust. The engine handles storage, search, and graph logic so you can focus on the UI.

Currently used by:

• Lore GUI — Qt 6.5 desktop app (personal wiki)
• lore-cli — command-line interface
• lore-mcp — MCP server (exposes the wiki to AI assistants)

Install

[dependencies]
lore-engine = "0.1"

Quick start

use lore_engine::state::AppState;
use lore_engine::engine::vault_ops;
use std::path::Path;

let state = AppState::new();

// Open a vault (folder of .md files)
let info = vault_ops::open_vault(Path::new("./my-wiki"), &state).unwrap();
println!("{} pages, {} links", info.page_count, info.link_count);

// Create a page
vault_ops::create_page("My First Page", "", &state).unwrap();

// Save content (upserts — creates if doesn't exist)
vault_ops::save_page("my-first-page", "# My First Page\n\nSee [[Other Page]].\n", &state).unwrap();

// Load a page
let page = vault_ops::load_page("my-first-page", &state).unwrap();
println!("Title: {}, Backlinks: {:?}", page.title, page.backlinks);

// Search
let results = vault_ops::search_vault("first", 10, &state).unwrap();
for r in results {
    println!("{}: {}", r.slug, r.snippet);
}

Features

Feature	What
Page CRUD	Create, read, update, delete with automatic link resolution
Upsert	save_page creates pages that don't exist, promotes placeholders
Wiki links	[[Page]], [[Page|alias]], fuzzy slug resolution, .md extension stripping
Full-text search	FTS5 with BM25 ranking, prefix matching, highlighted snippets
Title search	Fast LIKE-based title/slug search for quick switchers
Link graph	Directed graph (petgraph), backlinks, forward links, degree
Placeholders	Unresolved links create placeholder nodes, cleaned up automatically
Incremental sync	SHA-256 content hashing, only reprocesses changed files
File watching	Optional debounced watcher for external edits (500ms)
Folder tree	Build sidebar-style folder hierarchy from flat page list
Graph layout	Force-directed layout for visualization
Settings	JSON persistence for app settings (window geometry, theme, etc.)
H1 title extraction	Page title derived from first # Heading in content
Filename sanitization	Safe filenames on all platforms including Windows
Path traversal protection	Folder parameters validated against .. and absolute paths

Architecture

AppState (Arc-friendly, Mutex-wrapped)
├── db: SQLite connection (FTS5, WAL mode, foreign keys)
├── vault_path: current vault folder
├── graph: in-memory WikiGraph (petgraph DiGraph)
└── watcher: optional file system debouncer

All operations go through vault_ops functions which coordinate DB, graph, filesystem, and FTS updates. The engine knows nothing about its consumers — any frontend (CLI, GUI, server) creates its own AppState and calls the same functions.

Known limitations

• Wiki link syntax — supports [[Page]] and [[Page|alias]] only. Heading links ([[Page#Section]]) and embeds (![[Page]]) are not yet supported.
• File watching — designed for desktop GUI use. The watcher holds locks during processing, which can cause contention under concurrent access. Lock ordering is consistent (no deadlock risk), but high-frequency concurrent writes may experience brief blocking.
• Large files — content is loaded fully into memory. Files over ~10MB may cause high memory usage.
• Schema migrations — the engine is on schema v1 with no migration ladder yet. Schema changes in future versions will require a migration mechanism.
• Slug collisions — titles that differ only in punctuation (e.g. "C++" and "C#") may produce the same slug. The engine does not currently detect or prevent this.

License

Licensed under either of

• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.