diff --git a/packages/cloudflare/docs/blob-storage.md b/packages/cloudflare/docs/blob-storage.md index 2817913..cd7f2ce 100644 --- a/packages/cloudflare/docs/blob-storage.md +++ b/packages/cloudflare/docs/blob-storage.md @@ -14,13 +14,34 @@ Email bodies and attachments are large (kilobytes to megabytes). Storing them in ```typescript interface BlobStorage { - put(key: string, content: string | ArrayBuffer): Promise; - get(key: string): Promise; + put(key: string, content: string | ArrayBuffer, options?: BlobPutOptions): Promise; + get(key: string, options?: BlobGetOptions): Promise; delete(key: string): Promise; + generateKey(contentHash: string, extension: string): string; +} + +interface BlobObject { + text(): Promise; + arrayBuffer(): Promise; + httpMetadata?: Record; + customMetadata?: Record; +} + +interface BlobPutOptions { + httpMetadata?: Record; + customMetadata?: Record; +} + +interface BlobGetOptions { + range?: { offset: number; length: number }; } ``` -Any storage backend that supports key-value blob operations works: object storage (S3-compatible), file system, KV stores. +`get` returns a lazy `BlobObject` (not a string) so consumers choose whether to decode as text or binary via `.text()` / `.arrayBuffer()`. Partial reads are supported through `BlobGetOptions.range`, which IMAP `FETCH BODY[]` maps directly onto. + +`generateKey(contentHash, extension)` centralizes the key schema so applications do not hand-roll paths. The Cloudflare implementation builds keys of the form `emails/{year}/{month}/{contentHash}.{extension}`. + +Any storage backend that supports key-value blob operations works: object storage (S3-compatible), file system, KV stores. Ship-ready adapters in the repo: `createR2Storage` from `@rafters/mail-cloudflare/storage`. --- diff --git a/packages/cloudflare/docs/inbound.md b/packages/cloudflare/docs/inbound.md index 8c61fe4..2e7e002 100644 --- a/packages/cloudflare/docs/inbound.md +++ b/packages/cloudflare/docs/inbound.md @@ -10,39 +10,51 @@ The `InboundAdapter` interface handles incoming email. It receives a raw email, ```typescript interface InboundAdapter { - handleIncoming(email: InboundEmail): Promise; + handleIncoming(email: InboundEmail): Promise<{ + messageId: string; + threadId: string; + }>; +} + +interface InboundEmail { + raw: ArrayBuffer; // the complete RFC 5322 message as bytes + from: string; // parsed From header, for routing + to: string; // parsed To header, for mailbox resolution + headers: Record; // all headers as key-value pairs } ``` The adapter is responsible for: -1. Parsing the raw RFC 5322 email into structured fields -2. Finding or creating a thread (via In-Reply-To/References headers) +1. Parsing the raw RFC 5322 message from `raw` into structured fields (subject, date, CC, BCC, In-Reply-To, References, body parts, attachments) +2. Finding or creating a thread using the `In-Reply-To` and `References` headers 3. Storing the raw email and parsed content in blob storage 4. Creating the message record in the database -5. Updating thread metadata (messageCount, lastMessageAt, snippet) +5. Updating thread metadata (`unreadCount`, `lastMessageAt`, snippet) +6. Returning the resulting `messageId` and `threadId` for caller correlation --- -## Inbound email structure - -When an email arrives, the adapter receives: - -| Field | Source | -| ----------- | ---------------------------------- | -| from | From header (email + display name) | -| to | To header | -| cc, bcc | CC/BCC headers | -| subject | Subject header | -| messageId | Message-ID header | -| inReplyTo | In-Reply-To header | -| references | References header | -| date | Date header | -| textBody | Plain text content | -| htmlBody | HTML content | -| rawEmail | The complete RFC 5322 message | -| attachments | MIME attachments with metadata | -| headers | All headers as key-value pairs | +## Input vs parsed form + +The `InboundEmail` type received by `handleIncoming` is minimal on purpose: `raw`, `from`, `to`, and `headers`. The adapter does the parsing, not the caller. + +After parsing, the message row in the database carries the richer shape: + +| Field | Source | +| ---------------------------- | -------------------------- | +| `fromEmail`, `fromName` | From header | +| `toEmail`, `toName` | To header | +| `subject` | Subject header | +| `messageIdHeader` | Message-ID header | +| `inReplyTo` | In-Reply-To header | +| `references` | References header | +| `sentAt` | Date header | +| `receivedAt` | When ingestion happened | +| `blobKeyRaw` | Key for the raw .eml blob | +| `blobKeyHtml`, `blobKeyText` | Keys for parsed body blobs | +| `sizeBytes` | Raw blob size | +| `attachmentCount` | Count of MIME attachments | --- @@ -98,15 +110,32 @@ _dmarc.yourdomain.com TXT "v=DMARC1; p=quarantine; rua=mailto:dmarc@yourdomain ## Classification -After storage, the inbound adapter can optionally classify the message using an AI classifier. The classifier assigns: +After storage, the inbound adapter can optionally classify the message using an AI classifier. The classifier interface is `EmailClassifier` from `@rafters/mail/interfaces`: -- **category**: support, feedback, abuse, partnership, spam, billing, legal, other -- **confidence**: 0-100 score -- **summary**: one-line description of the message content +```typescript +interface EmailClassifier { + classify(from: string, subject: string, body: string): Promise; +} + +interface EmailClassification { + category: + | "support" + | "feedback" + | "abuse" + | "partnership" + | "spam" + | "billing" + | "legal" + | "other"; + confidence: number; // 0..100 + tags: string[]; // configurable pattern-matched tags + priority: "urgent" | "high" | "normal" | "low"; +} +``` -Classification results are stored on the message record. They drive folder assignment, priority, and dashboard filtering. +Classification results are stored on the message record (`aiCategory`, `aiConfidence`, priority, tags via label join). They drive folder assignment, thread priority, and dashboard filtering. -The classifier is a separate adapter (`ClassificationAdapter`). The default implementation uses zero-shot classification with a language model, but any classification strategy can be plugged in. +The default implementation in `@rafters/mail-workers-ai` uses DeBERTa-v3 zero-shot classification through Cloudflare Workers AI, but any classification strategy can be plugged in by implementing `EmailClassifier`. --- diff --git a/packages/cloudflare/docs/quickstart.md b/packages/cloudflare/docs/quickstart.md index ad99679..3a2e9f2 100644 --- a/packages/cloudflare/docs/quickstart.md +++ b/packages/cloudflare/docs/quickstart.md @@ -70,23 +70,30 @@ wrangler secret put RESEND_API_KEY ```typescript // src/index.ts import { createResendProvider } from "@rafters/mail-resend"; -import { createR2BlobStorage, parseInboundEmail } from "@rafters/mail-cloudflare"; +import { createR2Storage } from "@rafters/mail-cloudflare/storage"; +import { parseEmailHeaders, hashContent } from "@rafters/mail-cloudflare/parsing"; export default { // Handle inbound email from Cloudflare Email Routing - async email(message, env) { - const parsed = await parseInboundEmail(message); - const blobStorage = createR2BlobStorage(env.BLOB_STORAGE); + async email(message: ForwardableEmailMessage, env: Env) { + // Read the raw message bytes from the ReadableStream + const raw = await new Response(message.raw).arrayBuffer(); - // Store raw email in blob storage - const blobKey = await blobStorage.put(parsed.rawEmail); + // Parse RFC 5322 headers and hash the content for dedupe + const headers = parseEmailHeaders(Object.fromEntries(message.headers.entries())); + const contentHash = await hashContent(raw); - // Store message record in D1 + // Store the raw email in R2 via the BlobStorage adapter + const storage = createR2Storage({ bucket: env.BLOB_STORAGE }); + const blobKey = storage.generateKey(contentHash, "eml"); + await storage.put(blobKey, raw); + + // Insert message row in D1, update thread, dispatch to classifier queue // (wire up your service layer here) }, // Handle HTTP requests (webhooks, API) - async fetch(request, env) { + async fetch(request: Request, env: Env) { return new Response("Mail worker running"); }, }; @@ -151,7 +158,7 @@ Send an outbound email: const provider = createResendProvider({ apiKey: env.RESEND_API_KEY }); await provider.sendEmail({ from: "you@yourdomain.com", - to: ["recipient@example.com"], + to: "recipient@example.com", // single recipient per sendEmail call subject: "Hello from the edge", text: "Sent via @rafters/mail on Cloudflare Workers.", }); @@ -161,7 +168,10 @@ await provider.sendEmail({ ## 10. Add IMAP (optional) -To access your mailbox from Apple Mail, Thunderbird, or Outlook, add the IMAP server. See the [IMAP Quickstart](./imap-quickstart.md) for Cloudflare DO (WebSocket) or Node TCP (Fly.io) deployment options. +To access your mailbox from Apple Mail, Thunderbird, or Outlook, add the IMAP server. See the docs shipped with the IMAP runtime packages: + +- [`@rafters/mail-imap-server`](https://www.npmjs.com/package/@rafters/mail-imap-server) -- Node TCP for Fly.io / Railway / Fargate / VPS +- [`@rafters/mail-imap-cloudflare`](https://www.npmjs.com/package/@rafters/mail-imap-cloudflare) -- Durable Object + WebSocket for serverless --- @@ -178,8 +188,10 @@ To access your mailbox from Apple Mail, Thunderbird, or Outlook, add the IMAP se ## Next steps -- [Classification](./classification.md) -- auto-categorize incoming email -- [IMAP](./imap-quickstart.md) -- connect email clients -- [Newsletters](./newsletters.md) -- send to subscriber lists -- [App Passwords](./app-passwords.md) -- set up IMAP authentication -- [Deployment Guide](./imap-deployment.md) -- deploy IMAP on Fly, Railway, or Docker +Per-package docs ship with each npm package: + +- **Classification** -- see [`@rafters/mail-workers-ai`](https://www.npmjs.com/package/@rafters/mail-workers-ai) docs for auto-categorization +- **IMAP connect** -- see [`@rafters/mail-imap-server`](https://www.npmjs.com/package/@rafters/mail-imap-server) quickstart + deployment +- **IMAP auth** -- see [`@rafters/mail-imap`](https://www.npmjs.com/package/@rafters/mail-imap) `authentication.md` for the `AuthAdapter` contract +- **Newsletters** -- see [`@rafters/mail`](https://www.npmjs.com/package/@rafters/mail) `newsletters.md` for mailing lists, subscribers, campaigns +- **Inbound detail** -- see [`inbound.md`](./inbound.md) in this package diff --git a/packages/core/docs/migrations.md b/packages/core/docs/migrations.md index 18e36c9..e96d67a 100644 --- a/packages/core/docs/migrations.md +++ b/packages/core/docs/migrations.md @@ -14,47 +14,49 @@ This separation means the framework works with any SQLite-compatible database wi ## Schema -The core schema has 13 tables across two domains: - -### Inbox tables (10) - -| Table | Purpose | -| ------------------- | --------------------------------------------------------------- | -| mailbox | Email addresses that send and receive | -| inbox_folder | Folders per mailbox (inbox, sent, drafts, spam, trash, archive) | -| inbox_label | Labels for categorization (system and custom) | -| inbox_thread | Conversation threads | -| inbox_message | Individual email messages | -| inbox_message_label | Message-to-label associations | -| inbox_thread_label | Thread-to-label associations | -| inbox_attachment | File attachments | -| thread_assignment | Team member assignments to threads | -| thread_note | Internal notes on threads | - -### Newsletter tables (3) - -| Table | Purpose | -| ------------ | -------------------------------- | -| mailing_list | Named subscriber lists | -| subscriber | People on mailing lists | -| campaign | Broadcast messages sent to lists | +The core schema currently ships **10 inbox tables** in the exported migration SQL. An additional 3 newsletter tables are defined in the Drizzle schema but are not yet part of the migration SQL and are not written to by any shipped service -- they are reserved for future platform-side broadcast tracking and can be ignored at install time. + +### Inbox tables (10) -- in `migrationSQL` + +| Table | Purpose | +| --------------------- | --------------------------------------------------------------- | +| `mailbox` | Email addresses that send and receive | +| `inbox_folder` | Folders per mailbox (inbox, sent, drafts, spam, trash, archive) | +| `inbox_label` | Labels for categorization (system and custom) | +| `inbox_thread` | Conversation threads | +| `inbox_message` | Individual email messages | +| `inbox_message_label` | Message-to-label associations | +| `inbox_thread_label` | Thread-to-label associations | +| `inbox_attachment` | File attachments | +| `thread_assignment` | Team member assignments to threads | +| `thread_note` | Internal notes on threads | + +### Newsletter tables (3) -- schema-only, NOT in `migrationSQL` + +| Table | Purpose | +| --------------------- | --------------------------------------------------- | +| `platform_audience` | Platform-side mirror of the provider's mailing list | +| `platform_subscriber` | Platform-side mirror of a subscriber on an audience | +| `broadcast_audit` | Audit trail for campaign sends | + +These are available as Drizzle schema exports (`platformAudience`, `platformSubscriber`, `broadcastAudit` from `@rafters/mail/schema`) if you want to include them in your own migrations, but the shipped `migrationSQL` string does not create them. The `EmailProvider` mailing list / subscriber / campaign methods talk to the provider's API (Resend, etc.), not to these tables. --- ## System folders -When a mailbox is created, these folders are initialized automatically: +System folders are not created automatically at migration time. Apply the migration, create a mailbox, then call `FolderService.initSystemFolders(mailboxId)` to populate the following folders for that mailbox: -| Folder | Slug | Purpose | -| ------- | ------- | ---------------------------------- | -| Inbox | inbox | Default landing for incoming email | -| Sent | sent | Outbound messages | -| Drafts | drafts | Unsent compositions | -| Spam | spam | Spam-classified messages | -| Trash | trash | Soft-deleted messages | -| Archive | archive | Archived threads | +| Folder | Slug | Purpose | +| ------- | --------- | ---------------------------------- | +| Inbox | `inbox` | Default landing for incoming email | +| Sent | `sent` | Outbound messages | +| Drafts | `drafts` | Unsent compositions | +| Spam | `spam` | Spam-classified messages | +| Trash | `trash` | Soft-deleted messages | +| Archive | `archive` | Archived threads | -System folders cannot be deleted. Custom folders can be created by the user. +System folders have `isSystem = true` and cannot be deleted via `FolderService.delete`. Custom folders can be created with `FolderService.create`. --- @@ -62,11 +64,11 @@ System folders cannot be deleted. Custom folders can be created by the user. All tables follow these conventions: -- **IDs**: UUIDv7 via default function. Text primary keys. Timestamp-ordered. -- **Timestamps**: integer milliseconds. -- **Soft delete**: every table has deletedAt. Null means active. -- **JSON columns**: SQLite text with JSON mode. Parsed at read time. -- **User references**: plain text columns (ownerId, assigneeId, etc.). No foreign keys to external auth tables. The auth adapter resolves identity at runtime. +- **IDs**: UUIDv7 via default function. Text primary keys. Timestamp-ordered, so sort-by-id is chronological. +- **Timestamps**: `integer` with `mode: 'timestamp_ms'`, default `unixepoch('subsecond') * 1000`. Millisecond precision. +- **Soft delete**: all tables have `deletedAt` except the two label join tables (`inbox_message_label`, `inbox_thread_label`), where deletion is a direct row removal because there is nothing semantic to "soft delete" -- a label is either applied or not. +- **JSON columns**: SQLite `text` with `mode: 'json'`. Stored as serialized JSON strings, parsed at read time. +- **User references**: plain `text` columns (`ownerId`, `assigneeId`, `authorId`, `appliedBy`, etc.). No foreign keys to external auth tables. The `AuthAdapter` resolves identity at runtime. --- diff --git a/packages/core/docs/newsletters.md b/packages/core/docs/newsletters.md index c201b9c..3e95759 100644 --- a/packages/core/docs/newsletters.md +++ b/packages/core/docs/newsletters.md @@ -1,118 +1,113 @@ # Newsletters and Broadcasts -Sending email to audiences. Mailing lists, subscribers, campaigns. +Sending email to many recipients via mailing lists, subscribers, and campaigns. --- ## Vocabulary -@rafters/mail uses specific terms: +`@rafters/mail` uses generic platform terms in its interface surface: -| Term | Not this | Meaning | -| ------------ | --------- | -------------------------------- | -| Mailing list | Audience | A named list of subscribers | -| Subscriber | Contact | Someone on a mailing list | -| Campaign | Broadcast | A message sent to a mailing list | +| Interface name | Vendor synonyms | Meaning | +| -------------- | ----------------------------------- | --------------------------------- | +| `MailingList` | Audience (Resend), List (Mailchimp) | A named collection of subscribers | +| `Subscriber` | Contact, Audience Member | Someone on a mailing list | +| `Campaign` | Broadcast | A message sent to a mailing list | -Vendor terms (audience, contact, broadcast) appear only inside adapter implementations. +Vendor terms (`audience`, `broadcast`) appear inside adapter implementations and in the platform-side mirror tables, but they do not leak into the interface surface consumers call. --- -## Mailing lists +## The EmailProvider surface -A mailing list is a collection of subscribers who opted in to receive email. +All mailing list, subscriber, and campaign operations go through the `EmailProvider` interface. The provider is the authoritative source of truth for list membership and delivery. `@rafters/mail` does not duplicate that data locally unless the consumer opts into platform-side tracking. -| Field | Purpose | -| ----------- | ---------------------------------- | -| name | Display name ("Weekly Newsletter") | -| description | What subscribers signed up for | -| mailboxId | Which mailbox sends campaigns | - -Lists are per-mailbox. The support mailbox and the marketing mailbox have separate lists. - ---- - -## Subscribers - -| Field | Purpose | -| -------------- | --------------------------------------------- | -| email | Subscriber email address | -| name | Display name (optional) | -| status | subscribed, unsubscribed, bounced, complained | -| subscribedAt | When they opted in | -| unsubscribedAt | When they opted out | -| metadata | Custom fields (JSON) | +```typescript +interface EmailProvider { + // Mailing lists + createMailingList(name: string): Promise; + getMailingList(id: string): Promise; + deleteMailingList(id: string): Promise; + + // Subscribers + addSubscriber(listId: string, email: string, data?: SubscriberData): Promise; + removeSubscriber(listId: string, subscriberId: string): Promise; + updateSubscriber(subscriberId: string, updates: SubscriberUpdates): Promise; + listSubscribers(listId: string): Promise; + + // Campaigns + sendCampaign(params: CampaignParams): Promise<{ id: string }>; + getCampaign(id: string): Promise<{ id: string; subject: string; sentAt: Date }>; + createCampaignDraft(params: CampaignParams): Promise<{ id: string }>; + sendCampaignDraft(campaignId: string): Promise<{ id: string }>; + getCampaignStatus(campaignId: string): Promise; + + // Audiences (read-only list of available mailing lists) + listAudiences(): Promise; +} +``` -### Status lifecycle +### MailingList, Subscriber, Campaign shapes -``` -subscribed -> unsubscribed (user opts out) -subscribed -> bounced (delivery failure) -subscribed -> complained (marked as spam) +```typescript +interface MailingList { + id: string; + name: string; + createdAt: Date; +} + +interface Subscriber { + id: string; + email: string; + firstName?: string; + lastName?: string; + unsubscribed: boolean; +} + +interface CampaignParams { + listId: string; + subject: string; // 1..200 chars + html: string; // min 1 char + text?: string; + from: string; // sender email + replyTo?: string; + name?: string; +} + +interface CampaignStatus { + id: string; + status: "draft" | "queued" | "sending" | "sent" | "cancelled"; + subject: string; + sentAt: Date | null; +} ``` -Once a subscriber is `unsubscribed`, `bounced`, or `complained`, they do not receive future campaigns. Re-subscription requires explicit opt-in. +`sendCampaign` publishes a campaign immediately. `createCampaignDraft` + `sendCampaignDraft` is the two-step flow for drafts that might be edited before send. Polling `getCampaignStatus` is how you observe the send through `queued -> sending -> sent`. --- -## Campaigns - -A campaign is a message sent to all active subscribers on a mailing list. +## Platform-side mirror tables (optional) -| Field | Purpose | -| ------------- | ------------------------------------------------ | -| mailingListId | Target list | -| subject | Email subject | -| htmlBody | HTML content (rendered from template) | -| textBody | Plain text fallback | -| status | draft, scheduled, sending, sent, failed | -| scheduledAt | When to send (optional, for scheduled campaigns) | -| sentAt | When sending completed | +The `@rafters/mail` schema ships three newsletter-related tables in `packages/core/src/schema/newsletter.ts`: -### Campaign lifecycle +| Table | Purpose | +| --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `platform_audience` | Local mirror of a mailing list, keyed by `providerListId`. Stores `name`, `description`, and a URL-safe `slug`. Use when you want to show audiences in your own UI without re-fetching from the provider. | +| `platform_subscriber` | Association between one of your app users (`userId`) and one audience (`audienceId`), plus the `providerSubscriberId` for provider correlation. Use when you need to answer "which of my users is on which list" without round-tripping the provider. | +| `broadcast_audit` | Audit log of campaign sends: `providerCampaignId`, `subject`, `contentHash`, `sentBy` (user id), `audienceName`, `recipientCount`, `sentAt`. Use for compliance, "who sent what to whom" queries, and content-hash deduplication across sends. | -``` -draft -> scheduled -> sending -> sent -draft -> sending -> sent -draft -> sending -> failed -``` - -### Sending - -Campaigns are sent individually to each subscriber (not BCC). Each send is a separate API call to the email provider. This allows per-recipient tracking (delivery, opens, clicks) and personalization. - -Rate limiting is handled by the email provider adapter. The campaign runner respects the provider's concurrency limits. +**These tables are not in the exported `migrationSQL` string.** They exist as Drizzle schema definitions you can include in your own migrations if you want platform-side tracking. If you only use the `EmailProvider` interface and trust the provider's own audience/subscriber storage, you do not need these tables at all. --- ## Templates -Campaign content is typically rendered from a template: +Campaign HTML is usually rendered from a React Email template via the `TemplateRenderer` interface. See the `@rafters/mail-react-email` package for the shipped renderer and baseline templates, and its `docs/templates.md` for writing your own. -```typescript -const html = await renderer.render(NewsletterEmail, { - title: "Weekly Update", - content: markdownContent, - unsubscribeUrl: `https://yourdomain.com/unsubscribe?id=${subscriber.id}`, -}); -``` - -Every campaign email must include an unsubscribe link. The template adapter handles this requirement. +Every campaign email should include an unsubscribe link as required by CAN-SPAM and similar regulations. The provider handles the actual unsubscribe flow when a recipient clicks the link; the template is responsible for emitting a valid URL that the provider routes back to the subscriber record. --- -## Analytics - -Campaign analytics are derived from webhook events: - -| Metric | Source | -| ------------ | ------------------------------------------- | -| Sent | Total send attempts | -| Delivered | Delivery confirmations from provider | -| Bounced | Permanent delivery failures | -| Opened | Open tracking events (if tracking enabled) | -| Clicked | Click tracking events (if tracking enabled) | -| Unsubscribed | Unsubscribe actions from this campaign | -| Complained | Spam complaints from this campaign | +## Delivery and analytics -Analytics are per-campaign. Aggregate metrics (subscriber growth, engagement rates) are computed from campaign history. +Delivery events (delivered, bounced, complained, opened, clicked) come from the provider's webhook. `@rafters/mail-resend` ships a webhook handler at `@rafters/mail-resend/webhooks` that validates the signature and invokes user-provided callbacks for each event type. Correlate events back to campaigns via the `providerCampaignId` stored in `broadcast_audit`, or query the provider directly with `getCampaignStatus`. diff --git a/packages/core/docs/threading.md b/packages/core/docs/threading.md index 1ff65cc..fe2cd7b 100644 --- a/packages/core/docs/threading.md +++ b/packages/core/docs/threading.md @@ -1,41 +1,71 @@ # Threading -How @rafters/mail builds and maintains email threads. +How `@rafters/mail` builds and maintains email threads. --- -## How threads work +## RFC 5322 headers -Every email carries headers that identify its place in a conversation: +Every email carries three headers that identify its place in a conversation: -- **Message-ID**: a globally unique identifier assigned when the email is sent -- **In-Reply-To**: the Message-ID of the email being replied to -- **References**: the full chain of Message-IDs in the conversation, oldest to newest +- **`Message-ID`**: a globally unique identifier assigned when the email is sent +- **`In-Reply-To`**: the `Message-ID` of the email being replied to +- **`References`**: the full chain of `Message-ID` values in the conversation, oldest to newest -When a new message arrives, the threading engine uses these headers to find or create a thread: +`@rafters/mail` ships three building blocks for working with these headers from the `@rafters/mail/threading` subpath export: -1. Check `In-Reply-To` -- if it matches an existing message, join that thread -2. Check `References` -- walk the chain looking for any known message -3. Fall back to subject matching within the same mailbox (configurable) -4. If no match, create a new thread +```typescript +import { generateMessageId, buildReferences, generateSnippet } from "@rafters/mail/threading"; + +const id = generateMessageId("yourdomain.com"); +// -> "<019d7d...@yourdomain.com>" + +const refs = buildReferences(" ", ""); +// -> " " (already present, not re-appended) + +const snippet = generateSnippet("Hello world\n\nLong body text...", 200); +// -> "Hello world Long body text..." (first 200 chars) +``` + +`buildReferences` caps the chain at 50 entries to prevent unbounded growth; the most recent 50 are kept and the oldest are dropped. This matches the RFC 5322 recommendation for trimming long reference chains. --- -## Thread model +## Thread matching is the inbound adapter's job + +The `@rafters/mail/threading` module does not match messages to threads. Matching is the responsibility of the code that ingests inbound email -- typically an implementation of `InboundAdapter` -- because it requires database queries that the threading module cannot do on its own. + +The expected matching strategy, in priority order: -A thread is a container for related messages: +1. **`In-Reply-To`** -- if the incoming header matches a `messageIdHeader` on an existing `inbox_message` row, join that message's thread. +2. **`References`** -- walk the chain; any hit on an existing `messageIdHeader` joins that thread. +3. **Fresh thread** -- if no header match, create a new thread. -| Field | Purpose | -| ------------- | ------------------------------------ | -| subject | The conversation subject | -| snippet | Preview of the most recent message | -| participants | All email addresses involved | -| messageCount | Total messages in the thread | -| unreadCount | Messages not yet marked as read | -| status | open, pending, resolved, closed | -| priority | low, normal, high, urgent | -| folderId | Current folder (nullable) | -| lastMessageAt | Timestamp of the most recent message | +Subject-based matching is intentionally NOT part of this flow. Subject lines collide across unrelated conversations; header-based matching is the only reliable path. + +The shipped `InboxEmailService.composeEmail` always creates a fresh thread. Replies go through `replyToThread`, which takes an explicit `threadId` -- the caller is expected to know the target thread. Inbound matching logic is not yet shipped in a runtime adapter. + +--- + +## Thread model + +A thread is a container for related messages in `inbox_thread`: + +| Field | Type | Purpose | +| --------------- | -------------------------- | --------------------------------------- | +| `subject` | text (NOT NULL) | The conversation subject | +| `snippet` | text (nullable) | Preview of the most recent message | +| `participants` | json string[] (nullable) | All email addresses involved | +| `messageCount` | integer (default 1) | Total messages in the thread | +| `unreadCount` | integer (default 1) | Messages not yet marked as read | +| `status` | text (default "open") | `open`, `pending`, `resolved`, `closed` | +| `priority` | text (default "normal") | `low`, `normal`, `high`, `urgent` | +| `folderId` | text (nullable) | Current folder. `ON DELETE SET NULL`. | +| `startedAt` | timestamp (NOT NULL) | When the thread was created | +| `lastMessageAt` | timestamp (NOT NULL) | Timestamp of the most recent message | +| `updatedAt` | timestamp (NOT NULL, auto) | Last time any thread field changed | +| `archivedAt` | timestamp (nullable) | When archived, if applicable | +| `deletedAt` | timestamp (nullable) | Soft-delete marker | Threads track their own read state via `unreadCount` rather than a single boolean. A thread with 10 messages where 3 are unread shows `unreadCount: 3`. @@ -70,9 +100,11 @@ UUIDv7 is timestamp-ordered, which means Message-IDs naturally sort by creation ## Thread assignment -Messages are assigned to threads at ingest time. The assignment is permanent -- a message belongs to one thread for its lifetime. If the threading engine cannot find a match, it creates a new single-message thread. +Messages are assigned to threads at ingest time. The assignment is permanent -- a message belongs to one thread for its lifetime. If the inbound adapter cannot find a match via `In-Reply-To` / `References`, it creates a new single-message thread. + +Moving a thread between folders is a logical operation that updates `inbox_thread.folderId`; messages stay in place. -Moving a thread between folders moves all its messages. Deleting a thread soft-deletes all its messages. +**On deletion:** `inbox_message.threadId` has `ON DELETE CASCADE`, so **hard-deleting** a thread row also hard-deletes all its messages. **Soft-deleting** a thread (setting `deletedAt`) does NOT cascade to messages -- soft deletes are manual per-row. If you want to hide a thread and all its messages together, either hard-delete (irreversible) or soft-delete the thread AND each message explicitly. --- diff --git a/packages/resend/docs/outbound.md b/packages/resend/docs/outbound.md index d5c6c25..0cfc411 100644 --- a/packages/resend/docs/outbound.md +++ b/packages/resend/docs/outbound.md @@ -6,76 +6,110 @@ How the system sends email. ## The email provider -Outbound email is sent through an `EmailProvider` adapter. The provider handles the HTTP API call to your email sending service. +Outbound email is sent through an `EmailProvider` adapter. The provider handles the HTTP API call to your email sending service. The interface lives in `@rafters/mail/interfaces` and covers transactional sends plus mailing list, subscriber, and campaign management. ```typescript interface EmailProvider { - sendEmail(params: SendEmailParams): Promise; + // Transactional + sendEmail(params: EmailParams): Promise<{ id: string }>; + + // Mailing lists + createMailingList(name: string): Promise; + getMailingList(id: string): Promise; + deleteMailingList(id: string): Promise; + + // Subscribers + addSubscriber(listId: string, email: string, data?: SubscriberData): Promise; + removeSubscriber(listId: string, subscriberId: string): Promise; + updateSubscriber(subscriberId: string, updates: SubscriberUpdates): Promise; + listSubscribers(listId: string): Promise; + + // Campaigns + sendCampaign(params: CampaignParams): Promise<{ id: string }>; + getCampaign(id: string): Promise<{ id: string; subject: string; sentAt: Date }>; + createCampaignDraft(params: CampaignParams): Promise<{ id: string }>; + sendCampaignDraft(campaignId: string): Promise<{ id: string }>; + getCampaignStatus(campaignId: string): Promise; + + // Audiences + listAudiences(): Promise; } ``` -Parameters: +### EmailParams (transactional sends) -| Field | Required | Purpose | -| ------- | -------- | --------------------------- | -| from | yes | Sender email address | -| to | yes | Recipient email address(es) | -| subject | yes | Email subject | -| html | no | HTML body | -| text | no | Plain text body | -| cc | no | CC recipients | -| bcc | no | BCC recipients | -| replyTo | no | Reply-To address | -| headers | no | Custom headers | +| Field | Type | Required | Purpose | +| --------- | -------------- | -------- | ----------------------------------------------------------------------- | +| `to` | string (email) | yes | Recipient email address (single recipient per send) | +| `subject` | string | yes | Email subject, minimum 1 character | +| `html` | string | no | HTML body | +| `text` | string | no | Plain text body | +| `from` | string (email) | no | Sender address. Falls back to the provider's default `from` if omitted. | +| `replyTo` | string (email) | no | Reply-To address | -At least one of `html` or `text` must be provided. +At least one of `html` or `text` should be provided. `sendEmail` returns `{ id }` where `id` is the sending provider's message identifier, used later for webhook correlation. + +Multi-recipient sends (CC, BCC, or multiple `to`) are handled outside `sendEmail`. Use the mailing list + campaign APIs for batched sends, or issue multiple `sendEmail` calls for individual delivery to several recipients. --- ## Composing and replying +Compose and reply go through the higher-level `InboxEmailService` (from `@rafters/mail/services`), which wraps the `EmailProvider` with thread management, blob storage, and RFC 5322 header generation. + +```typescript +interface InboxEmailService { + composeEmail(params: ComposeEmailParams): Promise<{ threadId: string; messageId: string }>; + replyToThread(params: ReplyToThreadParams): Promise<{ messageId: string }>; +} +``` + ### New message ```typescript -await mailService.compose({ +await inboxEmailService.composeEmail({ mailboxId: "mbx_01J...", senderId: "user_01J...", - to: ["recipient@example.com"], - subject: "Hello", - body: "Plain text content", - bodyHtml: "

HTML content

", + to: ["recipient@example.com"], // array, at least one + subject: "Hello", // max 500 chars + body: "Plain text content", // required, min 1 char + bodyHtml: "

HTML content

", // optional + cc: ["cc@example.com"], // optional + bcc: ["bcc@example.com"], // optional }); ``` The compose flow: -1. Generates a Message-ID using UUIDv7 -2. Creates a new thread (or finds existing by subject) -3. Sends via the EmailProvider -4. Stores the sent message in the database with `isOutbound: true` -5. Stores content in blob storage +1. Generates a `Message-ID` using UUIDv7 +2. Creates a new thread for the subject +3. Sends via the `EmailProvider` +4. Stores the sent message row in the database with `isOutbound: true` +5. Stores the raw email and parsed body content in blob storage ### Reply ```typescript -await mailService.replyToThread({ +await inboxEmailService.replyToThread({ threadId: "thread_01J...", mailboxId: "mbx_01J...", senderId: "user_01J...", body: "Reply text", bodyHtml: "

Reply HTML

", + cc: ["cc@example.com"], + bcc: ["bcc@example.com"], }); ``` The reply flow: 1. Loads the thread and its most recent message -2. Builds the References header from the thread's message chain -3. Sets In-Reply-To to the most recent message's Message-ID -4. Generates a new Message-ID -5. Sends via the EmailProvider +2. Builds the `References` header from the thread's message chain +3. Sets `In-Reply-To` to the most recent message's `Message-ID` +4. Generates a new `Message-ID` +5. Sends via the `EmailProvider` 6. Stores the reply in the same thread with `isOutbound: true` -7. Updates thread metadata (messageCount, lastMessageAt, snippet) +7. Updates thread metadata (`unreadCount`, `lastMessageAt`, snippet) --- diff --git a/packages/workers-ai/docs/classification.md b/packages/workers-ai/docs/classification.md index 46e1a5a..c195de0 100644 --- a/packages/workers-ai/docs/classification.md +++ b/packages/workers-ai/docs/classification.md @@ -4,28 +4,23 @@ Automatic categorization of incoming email using AI or rule-based classifiers. --- -## The classification adapter +## The classifier interface ```typescript -interface ClassificationAdapter { - classify(message: ClassificationInput): Promise; +interface EmailClassifier { + classify(from: string, subject: string, body: string): Promise; } -interface ClassificationInput { - subject: string; - from: string; - textBody: string; - htmlBody?: string; -} - -interface ClassificationResult { - category: AiCategory; - confidence: number; // 0-100 - summary: string; // one-line description +interface EmailClassification { + category: AiCategory; // one of: support, feedback, abuse, partnership, + // spam, billing, legal, other + confidence: number; // 0..100 + tags: string[]; // pattern-matched tags (configurable) + priority: ThreadPriority; // urgent, high, normal, low } ``` -The adapter is called after a message is stored. Classification results are written to the message record. +The classifier is called after a message is stored. The result is written to the message record's `aiCategory`, `aiConfidence`, and related fields, and to the thread's `priority`. Tags are applied as labels via `inboxMessageLabel`. --- @@ -92,20 +87,21 @@ Spam detection can combine multiple signals: the AI classifier's spam category, The simplest classifier uses zero-shot classification with a language model: ```typescript -const classifier: ClassificationAdapter = { - async classify(input) { +const classifier: EmailClassifier = { + async classify(from, subject, body) { const result = await model.run({ - text: `${input.subject}\n\n${input.textBody}`, + text: `${subject}\n\n${body}`, labels: ["support", "feedback", "billing", "partnership", "abuse", "legal", "spam", "other"], }); return { category: result.label as AiCategory, confidence: Math.round(result.score * 100), - summary: await model.summarize(input.textBody, { maxLength: 100 }), + tags: [], // fill in via pattern matching if desired + priority: "normal", // derive from category or subject signals }; }, }; ``` -You can also implement rule-based classification (regex patterns on subject/sender), or a hybrid that uses rules first and falls back to AI for uncertain cases. +You can also implement rule-based classification (regex patterns on subject/sender), or a hybrid that uses rules first and falls back to AI for uncertain cases. The shipped `@rafters/mail-workers-ai` classifier uses this hybrid approach with configurable `tagPatterns`.