Skip to content

Fix/imap reliability and db perf#262

Open
M4lmostoso wants to merge 54 commits intoavihaymenahem:mainfrom
M4lmostoso:fix/imap-reliability-and-db-perf
Open

Fix/imap reliability and db perf#262
M4lmostoso wants to merge 54 commits intoavihaymenahem:mainfrom
M4lmostoso:fix/imap-reliability-and-db-perf

Conversation

@M4lmostoso
Copy link
Copy Markdown

Description

This PR addresses several structural issues regarding email synchronization and database stability, specifically targeting non-standard IMAP server behavior and concurrent database access patterns.

1. IMAP Reliability (DavMail/Exchange Fix)

Introduced a robust fallback mechanism in the Rust backend for cases where the IMAP server returns successful fetch responses but with unparseable or empty bodies. This is a common issue when using DavMail or certain Microsoft Exchange configurations.

  • Added a check for empty message bodies after an async-imap fetch.
  • Implemented a Raw TCP Fetch fallback that manually parses the IMAP stream when the standard library fails.

2. Database Concurrency & Performance

Optimized the SQLite engine and application-level access to prevent "database is locked" errors and transaction collisions:

  • WAL Mode: Enabled PRAGMA journal_mode = WAL to allow simultaneous reads and writes without blocking.
  • Reentrant Async Mutex: Implemented a reentrant write-queue in src/services/db/connection.ts. This ensures that all database writes are serialized at the JS level while allowing nested transaction calls without deadlocks.
  • Removed Manual Transactions: Replaced manual BEGIN/COMMIT blocks (which were prone to connection pool issues in Tauri) with JS-side serialization.
  • Resilience: Set a global busy_timeout of 5000ms.

3. Tauri v2 Security & Stability

  • Permissions: Updated capabilities/default.json with explicit SQL scopes for velo.db and added necessary logging permissions for better developer visibility.
  • Sync Optimization: Tuned IMAP sync chunk sizes (from 200 down to 100) to balance IPC overhead and system responsiveness during initial synchronization.

Testing Performed

  • Verified synchronization with DavMail (IMAP on port 1143).
  • Verified concurrent syncing of multiple accounts (Gmail API + IMAP).
  • Checked for memory leaks and database integrity after multiple interrupted sync cycles.
  • Confirmed that "Syncing labels" no longer hangs due to database locks.

Fixed Issues

  • Resolves sync hangs on "Syncing labels..."
  • Fixes cannot start a transaction within a transaction and no transaction is active errors.
  • Fixes empty message bodies for Exchange/DavMail users.

Mirko Landenna added 2 commits April 27, 2026 18:53
- Added raw TCP fallback for IMAP servers returning empty bodies (DavMail/Exchange fix)
- Enabled SQLite WAL mode for better concurrency
- Implemented reentrant async write-queue in JS to prevent database locks
- Updated Tauri v2 capabilities with proper SQL scopes
- Optimized sync chunk size for smoother performance
Mirko Landenna and others added 27 commits April 28, 2026 11:14
- Fix CSP to allow ipc://localhost so Tauri IPC works without postMessage fallback
- Fix fetchSendAsAliases called on IMAP accounts (filter to gmail_api only)
- Add migration v14 repair guard (imap_folder_path missing despite migration marked applied)
- Refactor ImapSmtpProvider to resolve UIDs from imap_folder/imap_uid columns directly,
  avoiding stale synthetic message IDs after folder moves
- Pass actual message IDs from ActionBar/EmailList to email action handlers
- Update messages table is_read/is_starred alongside threads in applyLocalDbUpdate
- Add bulk mark-read/unread in EmailList and Shift+I/Shift+U keyboard shortcuts
- Fix smart folder count badge to count distinct thread_id instead of message id
- Exclude TRASH/SPAM threads from smart folder results unless query targets those labels

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…senders

- Revert img-src CSP to original (https: scheme alone caused Tauri to not
  load the main window, leaving app stuck on splash screen)
- Refactor EmailRenderer to use srcdoc + sandbox="allow-scripts allow-popups"
  instead of doc.write() + allow-same-origin: iframe gets a null origin so
  Tauri's CSP does not apply, letting remote images load freely for trusted
  senders without any global CSP relaxation
- Height updates and link clicks communicated via postMessage from injected
  inline script (email HTML is already DOMPurify-sanitised; null-origin sandbox
  prevents iframe from accessing parent DOM or Tauri APIs)
- Fix senderAllowlisted check to normalise address before Set.has() lookup so
  mixed-case from_address correctly matches lowercase DB entries

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add People API OAuth scope (contacts.readonly)
- Create googleContacts.ts service with:
  - GoogleContactsClient with token refresh
  - syncGoogleContacts() for bidirectional sync
  - Support for multiple email addresses per contact
  - Rate limiting with retry on 429
- Add importContact() that doesn't inflate frequency
- Add migration to reset contacts frequency
- Add CSP exception for people.googleapis.com
- Add Sync Contacts button in Settings > Accounts
- Increase contacts limit to 10,000
- Add aiSidebarOpen state to composerStore with toggle function
- Redesign AiAssistPanel as full chat sidebar (not header)
- Add keyword detection: write commands → insert to body, chat keywords → sidebar
- Handle explicit language requests (skip default setting)
- Clean markdown fences from AI output before inserting
- Update prompts: no markdown fences, no translations

Closes: new feature
- Add custom tiptapExtensions with FontSize and FontFamily extensions
- Add TextStyle, Color, Underline extensions to composer
- Add font size dropdown (10-24px)
- Add font family dropdown (System, Arial, Calibri, Times, etc.)
- Add color picker with 32 colors and click-outside-to-close
- Update composer modal to max-w-6xl always
- Add ProseMirror CSS styling for editor content
…ccount label

- Add Color and Underline extensions to SignatureEditor so the shared
  EditorToolbar works correctly (color picker and underline were silently failing)
- Refactor tiptapExtensions with proper types and unset commands
- Wrap EmailRenderer iframe load handler in try/catch to avoid SecurityError
  on sandboxed iframes in WebKit; fix stray 3-space indentation
- Include account email in sync error log and footer status bar so it's
  clear which account failed

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The signature was incorrectly rendered as a sibling element in a flex row,
causing it to appear on the right side of the editor instead of below the
composed message body.
Allow users to switch sending account directly from the composer
without closing and reopening. Adds ComposerAccountSwitcher component
that shows in the header when multiple accounts are configured.

- composerStore: add composerAccountId field to track selected account
- ComposerAccountSwitcher: new compact dropdown for account selection
- Composer: use effectiveAccountId (composerAccountId ?? activeAccountId)
- All composer operations now use effectiveAccountId
- ComposerWindow: handle accountId URL param for pop-out windows

Closes: #account-switcher-feature
DavMail and some Exchange gateways silently return empty results for
`UID SEARCH n:*` range queries even when new messages have arrived,
causing the delta sync to miss emails indefinitely.

Add a fallback that runs `UID SEARCH SINCE <yesterday>` whenever the
UID range search returns zero results and a `last_sync_at` timestamp
is available. Results are filtered to UIDs > last_uid to avoid
reprocessing already-synced messages.

The fallback is triggered only on empty range results, so standard
IMAP servers (Gmail, Fastmail, etc.) are completely unaffected.
Applied to both the batch delta-check path (Rust) and the per-folder
fallback path (TypeScript).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…g style section

Adds a new "Style" section in the Composing settings tab with controls
for default font family (8 options) and font size (7 options). Settings
are persisted to SQLite, restored on startup, and applied dynamically
to the TipTap editor in the composer.
In reply and forward, the signature was being appended after all
quoted content. Now the signature appears between the user's message
and the quoted previous emails, which is the expected behavior.

- Add quotedHtml field to composerStore to separate quotes from body
- Modify ThreadView buildQuote/buildForwardQuote to return only quotes
- Update Composer getFullHtml() to order: body + signature + quotes
- Show quoted content preview (read-only) below signature in composer
- All Mail view now excludes DRAFT and TRASH threads via NOT EXISTS subquery
- DRAFT and TRASH threads are treated as always-read in the UI
- Fix startup ErrorBoundary crash: TipTap v3 editor.view is a Proxy that
  throws for unknown keys (incl. 'dom') before EditorContent mounts; guard
  the font/size effect with isOpen so it only runs when EditorContent is
  actually rendered (unmountOnExit)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Root cause: draft deletion in the Drafts folder view was Gmail-only —
getGmailClient() silently failed for IMAP accounts, leaving drafts
untouched on server and DB. Next sync recreated them.

- Add provider-agnostic deleteDraftThread() in emailActions.ts: uses
  Gmail Drafts API for gmail_api accounts, permanentDeleteThread() for
  IMAP (direct delete from Drafts folder, no UID-changing MOVE to Trash)
- Replace all three Gmail-only call sites (ActionBar, useKeyboardShortcuts,
  ContextMenuPortal) with the new unified function
- Return threadId from Gmail createDraft/updateDraft so draftAutoSave
  stores it in composerStore; deleteDraft can then clean the local DB
  for new-compose drafts that never had a threadId
- Skip server update in IMAP updateDraft when draftId is a pseudo-ID
  (imap-draft-...) to prevent one zombie draft per auto-save cycle
- Exclude DRAFT label from smart folder queries (was already done for
  TRASH and SPAM)
- Force is_read=true for DRAFT and TRASH threads during sync so they
  never appear as unread
- Remove DRAFT label alongside INBOX when trashing a draft thread so it
  disappears from the Drafts view immediately
- Fix deleteThread() cascade to also remove messages, thread_labels,
  thread_categories and attachments in one transaction
- Fix early return in executeAction() that blocked app-level shortcuts
  (ask-inbox, command-palette, help) when no thread was selected

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…tivation

- Add migration v25: adds group_id to signatures table for cross-account
  signature grouping; backfills group_id = id for existing records
- Add repair check for v25 in runMigrations (mirrors v14/v18 pattern):
  re-runs migration if column is missing despite being marked applied
- Add getAvailableSignaturesForAccount: returns one representative per
  group not yet active for the target account; computes group IDs in JS
  to avoid SQL alias issues with the Tauri SQLite plugin
- Add importSignature: clones a signature to a new account using the
  same group_id, skips if already imported
- updateSignature/deleteSignature now operate on the full group:
  updates propagate to all accounts sharing the signature; delete removes
  only the current account's record when others exist (deactivate), or
  deletes the entire group when it's the last one (permanent delete)
- SignatureEditor: account selector UI with avatar initial and ChevronDown;
  available (not-yet-activated) signatures shown grayed-out with Download
  icon; save/import errors surfaced in the UI via AlertCircle banner

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ents

Replace native <select> elements with custom dropdown components in SignatureSelector, FromSelector, and SignatureEditor to maintain consistent styling across the application.

- SignatureSelector: dropdown for signature selection in composer header
- FromSelector: dropdown for send-as alias selection in composer header
- SignatureEditor: dropdown for account selection in signature settings

All dropdowns now follow the same pattern as AccountSwitcher with:
- glass-panel styling with backdrop blur
- accent color for active item indication
- Check icon for selection visual feedback
- proper click-outside handling
- keyboard navigation support via button behavior

Updated SignatureEditor test mock to include getAvailableSignaturesForAccount.

No functional changes, purely visual consistency improvement.
- Implemented RFC Message-ID deduplication in IMAP sync
- Added SINCE fallback for delta sync when UID range search fails
- Fixed duplicate threads in EmailList infinite scroll
- Improved composer pop-out logic with localStorage state persistence
- Fixed thread labels being wiped during delta sync
- Fixed smart folder UNREAD exclusion logic for TRASH/SPAM
- Implemented DB queries for unread counts by label and category
- Added unreadCount state and refresh logic to LabelStore
- Updated Sidebar to display unread badges for all nav items, categories, and user labels
- Added event dispatching in email actions to trigger immediate unread count updates
…gressions

- Implemented bulk unread count retrieval for labels and categories.
- Updated LabelStore to manage unread counts and sync with sidebar.
- Fixed multiple TypeScript errors in UI components.
- Resolved all unit test failures by updating mocks and SQL assertions.
- Fixed composerStore to support in-app opening during tests.
…aftId, and fix thread message count synchronization
Mirko Landenna and others added 25 commits April 30, 2026 18:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant