Skip to content

[WIP] Sqlcipher encryption#50

Open
kenforthewin wants to merge 11 commits intomainfrom
sqlcipher-encryption
Open

[WIP] Sqlcipher encryption#50
kenforthewin wants to merge 11 commits intomainfrom
sqlcipher-encryption

Conversation

@kenforthewin
Copy link
Copy Markdown
Owner

No description provided.

@greptile-apps
Copy link
Copy Markdown

greptile-apps Bot commented Mar 29, 2026

Greptile Summary

This PR introduces SQLCipher AES-256 at-rest encryption for atomic's SQLite backend (registry and all knowledge-base databases). The key architectural change is a deferred-initialization mode for DatabaseManager: when the server starts and detects an encrypted database but has no passphrase, it enters a locked state and serves a new /api/setup/unlock endpoint instead of crashing. Once the correct passphrase is provided through the web UI, the manager is fully initialized and normal operation resumes. The frontend gains an UnlockScreen component, passphrase entry during first-time setup, and a pre-connect check that routes returning users to the unlock flow automatically.

Key changes:

  • bundledbundled-sqlcipher in atomic-core/Cargo.toml; apply_connection_pragmas() helper ensures PRAGMA key is the very first statement on every new connection across write, read-pool, temp, and new_connection paths.
  • DatabaseManager::registry changed from Arc<Registry> to RwLock<Option<Arc<Registry>>>, with initialize() holding the write lock for the full check-and-set to prevent the previously-flagged TOCTOU race.
  • DatabaseManager::registry() now returns Result instead of &Arc, and middleware/routes guard with is_initialized() returning HTTP 503 to avoid misleading 401s.
  • create_manager in main.rs correctly distinguishes SQLITE_NOTADB (encrypted) from other errors and exits on genuine corruption.
  • isServerInitialized() in HttpTransport.connect() prevents noisy WebSocket reconnect loops during the locked/setup window.

Two issues worth addressing before removing the WIP tag:

  • Startup recovery tasks (stuck-atom reset, legacy token migration) run only once at startup under an is_initialized() guard; they are permanently skipped for encrypted restarts because the guard is false at startup and never re-evaluated after unlock_instance fires.
  • unlock_instance returns HTTP 401 for all initialize() failures, including I/O errors and lock poisoning, with the hint "Check your passphrase and try again" — which is actively misleading for infrastructure failures.

Confidence Score: 5/5

Safe to merge with caveats noted — the WIP tag should stay until startup-task and unlock-error-mapping gaps are addressed.

All remaining findings are P2: the startup-tasks gap only affects encrypted restarts with pre-existing stuck atoms (an unlikely combination for most deployments), and the misleading 401 hint is a UX rough edge rather than a data-safety issue. The core concerns from previous review rounds — TOCTOU race, panic on registry access, silent error swallowing — are all addressed. The encryption implementation itself (PRAGMA key ordering, pool propagation, passphrase storage) is correct.

crates/atomic-server/src/main.rs (startup task gap) and crates/atomic-server/src/routes/setup.rs (unlock 401 for all errors)

Important Files Changed

Filename Overview
crates/atomic-core/src/db.rs Adds apply_connection_pragmas helper that applies PRAGMA key before base PRAGMAs across all connection types. Passphrase stored in struct so overflow connections use it correctly. Clean and correct implementation.
crates/atomic-core/src/manager.rs Major refactor: registry changed from Arc to RwLock<Option<Arc>> supporting deferred mode. TOCTOU fix from previous review is applied. Gap: startup recovery tasks aren't triggered after deferred unlock.
crates/atomic-core/src/registry.rs Passphrase added to Registry struct; PRAGMA key applied before WAL/NORMAL pragmas in both open_or_create_encrypted and new_connection. Unencrypted path delegates to encrypted path with None.
crates/atomic-server/src/main.rs create_manager now detects encrypted databases by checking for SQLITE_NOTADB error messages and starts in deferred mode. Startup tasks correctly guarded with is_initialized(). Gap: tasks aren't re-run after deferred unlock.
crates/atomic-server/src/routes/setup.rs New unlock_instance endpoint; setup_status correctly distinguishes fresh install vs encrypted locked via registry.db existence. All initialize() errors return HTTP 401 with wrong passphrase hint regardless of actual error type.
src/components/onboarding/UnlockScreen.tsx New standalone unlock screen component; Enter-key handler, loading state, and error display all implemented correctly.
src/components/layout/Layout.tsx Adds needs_unlock state and pre-checks saved server config for encrypted restart before falling through to onboarding. handleUnlockComplete correctly re-initializes transport and rechecks provider config.

Sequence Diagram

sequenceDiagram
    participant S as Server startup
    participant DB as SQLCipher
    participant C as Browser
    participant UI as UnlockScreen

    S->>DB: open_or_create_encrypted(data_dir, passphrase=None)
    DB-->>S: Err - file is not a database
    S->>S: new_deferred(data_dir)
    Note over S: is_initialized = false
    Note over S: Startup tasks SKIPPED

    C->>S: GET /api/setup/status
    S-->>C: needs_unlock true

    C->>UI: Show UnlockScreen
    UI-->>C: User enters passphrase

    C->>S: POST /api/setup/unlock
    S->>DB: PRAGMA key + base PRAGMAs
    DB-->>S: OK
    S->>S: is_initialized = true
    S-->>C: status unlocked

    C->>C: switchTransport with saved config
    C->>S: WebSocket connect
    S-->>C: Connected

    Note over S: Recovery tasks still not run
Loading

Reviews (4): Last reviewed commit: "fix: keep unlock button disabled while t..." | Re-trigger Greptile

Comment thread crates/atomic-core/src/manager.rs
Comment thread crates/atomic-server/src/main.rs
Comment thread crates/atomic-core/src/manager.rs
Comment thread crates/atomic-server/src/routes/setup.rs
Comment thread crates/atomic-server/src/routes/setup.rs
kenforthewin and others added 4 commits March 30, 2026 21:19
Switch rusqlite from bundled to bundled-sqlcipher permanently. Add
optional passphrase-based encryption to all SQLite databases.

Key changes:
- Database, Registry, AtomicCore, and DatabaseManager all accept
  optional passphrase for SQLCipher PRAGMA key
- apply_connection_pragmas() helper ensures PRAGMA key is always the
  first statement on every new connection (write, read pool, temp, ad-hoc)
- DatabaseManager supports deferred mode: on fresh installs with no CLI
  passphrase, databases aren't created until the user completes setup
- POST /api/setup/claim accepts optional passphrase field to encrypt
  databases at creation time
- On subsequent startups, if databases are encrypted and no
  ATOMIC_PASSPHRASE env var is set, the server prompts on stdin
- Postgres mode ignores passphrase (encryption managed by Postgres)

4 new tests: encrypted roundtrip, wrong passphrase rejection, no
passphrase on encrypted DB, encrypted read pool + new_connection.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add optional passphrase field to the claim/setup flow in WelcomeStep.
Users can check "Encrypt database at rest" and set a passphrase during
initial setup. The passphrase is sent to POST /api/setup/claim which
creates encrypted databases via SQLCipher.

Includes passphrase confirmation, minimum length validation (8 chars),
and a warning that the passphrase is unrecoverable.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the stdin passphrase prompt with a web-based unlock flow:

- Server boots in deferred/locked mode when databases are encrypted
  and no ATOMIC_PASSPHRASE env var is set
- GET /api/setup/status returns needs_unlock: true for encrypted DBs
- POST /api/setup/unlock accepts passphrase and initializes the manager
- WelcomeStep shows a "Database Locked" UI with passphrase input
  when needs_unlock is true
- After unlock, transitions to the normal manual connection flow
- ATOMIC_PASSPHRASE env var still works for headless/Docker deployments

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix TOCTOU race in initialize() by checking under the registry write lock
- Distinguish encryption errors from corruption/permission failures at startup
- Change registry() from panicking to returning Result, update all callers
- Return 503 from auth middleware when server is not initialized instead of
  misleading 401
- Add require_registry! macro for OAuth routes
- Fix missing log_buffer field in test AppState constructors
- Replace eprintln! with tracing::warn! in encryption code paths

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@kenforthewin kenforthewin force-pushed the sqlcipher-encryption branch from 80cf8ca to 72b195e Compare March 31, 2026 01:35
@kenforthewin
Copy link
Copy Markdown
Owner Author

@greptileai

kenforthewin and others added 2 commits March 30, 2026 21:40
- WS handler returns 503 when server is not initialized instead of
  failing during token verification
- Frontend checks /api/setup/status before attempting WS connection;
  skips WS and disables reconnect loop if server needs setup or unlock
- Replace eprintln! with tracing::warn! in WS lag handler

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Layout now checks transport.isConnected() before calling
verifyProviderConfigured, avoiding a 503 from the auth middleware
when the server is not yet initialized.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@kenforthewin
Copy link
Copy Markdown
Owner Author

@greptileai

kenforthewin and others added 3 commits March 30, 2026 21:48
switchTransport and switchToLocal now transfer event listeners from
the old transport to the new one. Previously, subscriptions registered
by useEmbeddingEvents (and other hooks) were lost when the transport
was replaced during onboarding, so embedding/tagging events were never
received until the page was refreshed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previously, restarting with an encrypted database routed through the
full onboarding wizard (with step indicators), and after unlock it
showed the manual connect form instead of reconnecting.

Now Layout detects the needs_unlock case before entering onboarding,
shows a dedicated UnlockScreen, and after unlock reconnects with the
saved transport config automatically.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Await the onUnlocked callback so the button stays in its "Unlocking..."
state until Layout finishes reconnecting. Only reset on error.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@kenforthewin
Copy link
Copy Markdown
Owner Author

@greptileai

kenforthewin and others added 2 commits March 30, 2026 22:17
Startup tasks (stuck-atom reset, pending embeddings/tagging, legacy
token migration) were guarded by is_initialized() at boot and never
re-evaluated after unlock. Now both unlock_instance and claim_instance
call run_post_init_tasks() after initialization so encrypted restarts
get the same recovery as unencrypted ones.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Desktop users can now opt into database encryption during first setup.
On fresh install, the welcome step shows an encryption checkbox with
passphrase fields. The claim is handled silently — the API token is
saved to disk via a new Tauri IPC command without being shown to the
user.

Key changes:
- src-tauri: Replace eager ensure_local_token() with read_cached_token()
  that only reads the token file without opening the database. Add
  save_local_token IPC command for post-claim token persistence.
- transport: Skip WS connect when no auth token (fresh install). Add
  isDesktopFreshInstall() and saveDesktopToken() helpers.
- WelcomeStep: Desktop fresh install shows encryption option and calls
  /api/setup/claim directly, then saves token via IPC.
- Layout: Detect encrypted restart on desktop by checking local sidecar
  setup/status. Use switchToLocal() after unlock instead of localStorage.

Three desktop scenarios:
- Fresh install: sidecar starts deferred, user sees encryption option,
  claim creates DBs (encrypted or not), token saved to disk.
- Encrypted restart: token file exists, sidecar detects encryption and
  defers, unlock screen shown, after unlock reconnects via switchToLocal.
- Unencrypted restart: token file read, sidecar opens DB normally,
  transport connects, app loads — no changes from previous behavior.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant