WhatsApp support: live sync via whatsmeow + backup import

[eddowding]

I'm interested in contributing WhatsApp support. I've read through the codebase and noticed the unified schema already accounts for WhatsApp (source_type, message_type, raw_format, phone numbers on participants, reactions table, etc.) — so the storage layer looks ready.

What I'd build:

Phase 1 — Live sync via whatsmeow:

• New internal/whatsapp/ package using whatsmeow for the WhatsApp Web multi-device API
• QR code authentication flow (similar to how add-account opens a browser for Gmail OAuth)
• Session persistence so re-auth isn't needed on every sync
• Live sync: pull message history, map to existing schema (conversations, participants by phone number, attachments, reactions)
• CLI commands: add-account --whatsapp, sync dispatching by source type

Phase 2 — Backup import:

• Import from WhatsApp's "Export Chat" .txt files (quick win, but lossy — no metadata/reactions)
• Import from encrypted WhatsApp backups (Google Drive / local msgstore.db) if feasible — this is harder due to encryption key extraction

Questions before I start:

• Does this align with your current priorities, or is there other work you'd want landed first?
• Any preferences on whatsmeow vs other approaches?
• For backup import — any preference on which format to prioritise (export .txt vs encrypted backup)?
• Are there any architectural decisions or constraints I should know about for the sync pipeline?

Happy to start with a design doc or go straight to a draft PR — whatever works best for the project.

[wesm]

Hi Ed, sounds like a good initial plan. The main concern that I have is making sure that the same storage schema can be used to other chat/text message types (iMessage, SMS/Google Voice, Telegram, Android Messages, etc.). I think a flag like --whatsapp makes less sense than --type whatsapp (I guess we need to come up with the account identifier for multiple whatsapp accounts in the same database, something like the phone number string +1 617-555-1234). Can you do a few iterations with Codex and/or Claude and post the plan here or in a gist? I'd be happy to review and give feedback before you start implementing.

Thanks!

[eddowding]

Thanks Wes! Great feedback on --type whatsapp over --whatsapp — totally agree.

I've done a deep dive on the codebase, whatsmeow, and cross-platform message formats (iMessage chat.db, Telegram TDLib/export, Signal, Android SMS/MMS). Here's the design doc:

https://gist.github.com/eddowding/ac7a12ab1750dc8a7d1107547f7727d2

Key takeaways:

• The schema is already ~90% ready — source_type, message_type, participant_identifiers, reactions, media_type fields all exist and map cleanly to WhatsApp/iMessage/Telegram/SMS
• Account identifiers: E.164 phone numbers for phone-based sources (+447700900000), email for email-based. UNIQUE(source_type, identifier) means same phone can be both WhatsApp and iMessage
• CLI: msgvault add-account --type whatsapp "+447700900000", sync dispatches by type
• WhatsApp history is a one-time dump at pairing (not on-demand like Gmail API) — design accounts for this
• Media URLs expire in ~30 days — download at sync time, not lazily
• Ban risk is real — documented prominently, recommend dedicated number

Minimal changes to existing code — WhatsApp is a new internal/whatsapp/ package, plus small changes to CLI dispatch and a EnsureParticipantByPhone() store method. No schema migrations, no changes to query engine/TUI/MCP/API.

No premature abstractions — build WhatsApp first, extract common patterns once we have two working sources.

Happy to iterate on this. Let me know what you think!

[wesm]

Thanks for putting this together. I'll plan to take a closer look at this tomorrow — wanted to today but had my hands full

[rubenv]

Signal: Hardest. Desktop DB is SQLCipher-encrypted. Schema changes frequently (215+ migration versions). No official export API. Feasible but fragile.

But easy to follow in real-time: https://github.com/AsamK/signal-cli

[eddowding]

Quick update — I've successfully extracted my full WhatsApp history via the Google Drive E2E backup route. For the initial historical import, this is much simpler than whatsmeow.

The process:

1. Enable E2E encrypted backup in WhatsApp (Settings → Chats → Chat backup), choosing the 64-digit encryption key option (not password, not passkey)
2. Let it backup to Google Drive
3. Use wabdd to download the backup from Google Drive via a Google OAuth token
4. wabdd decrypt produces a plain SQLite msgstore.db

What you get:

• Full message history (mine: 1.24M messages, 13.6k conversations, April 2016 – present)
• Built-in FTS index (message_ftsv2) — search is instant
• Contacts via wa.db (also decrypted)
• Media files (photos, docs — audio/video partially supported)
• Clean schema: message table with chat_row_id, from_me, text_data, timestamp, message_type

How this fits with whatsmeow:

The two approaches are complementary:

• Backup decrypt → best for initial historical import. Full history back to day 1, zero ban risk, no live session needed. Simple offline SQLite-to-SQLite transform.
• whatsmeow → best for ongoing live sync after the initial import. Real-time message capture, but only gets messages from pairing point forward and carries ban risk.

A natural workflow: import the full archive via backup decrypt, then optionally layer whatsmeow on top for live sync going forward.

What this means for msgvault:

The import path for history becomes: msgstore.db (SQLite) → map to msgvault schema → done. No live connection, no daemon, no WebSocket.

The --type whatsapp CLI and account model from my earlier design still applies — the only change is the source is a local SQLite file rather than a whatsmeow session. Could support incremental updates too: re-backup periodically, diff against last import by timestamp.

[wesm]

That sounds promising. is it possible to sync your whole whatsapp history using https://github.com/tulir/whatsmeow instead of the google drive backup (does this only work on Android?)?

Several people have mentioned beeper as a possible "scrape everything" solution. I admit that I have not had the time to investigate this thoroughly myself.

[wesm]

In general, I don't have a problem if msgvault ends up supporting multiple strategies for import and sync, there need not be "just one way" to import whatsapp messages.

[eddowding]

WhatsMeow / Baileys / Beeper only do ~30 days history.

[wesm]

Got it. So we probably need to support multiple solutions for full history imports (the different export / backup methods that WhatsApp provide). I'd be happy to look at a pull request if you want to put something up!