Skip to content

feat: add ESEARCH (RFC 4731) and PARTIAL (RFC 9394) support#347

Open
peterzen wants to merge 1 commit intopostalsys:masterfrom
peterzen:feat/esearch-partial
Open

feat: add ESEARCH (RFC 4731) and PARTIAL (RFC 9394) support#347
peterzen wants to merge 1 commit intopostalsys:masterfrom
peterzen:feat/esearch-partial

Conversation

@peterzen
Copy link
Copy Markdown

@peterzen peterzen commented Mar 28, 2026

Summary

Adds ESEARCH support to ImapFlow's search() method, enabling efficient search result handling for large mailboxes.

  • Handles COUNT, MIN, MAX, ALL, and PARTIAL result keywords from * ESEARCH untagged responses
  • Emits UID SEARCH RETURN (COUNT PARTIAL 1:100 ...) when returnOptions is passed and the server advertises ESEARCH capability
  • Callers without returnOptions get the same number[] result as before
  • Graceful fallback — derives ESearchResult client-side from the plain number[] (COUNT from .length, MIN/MAX from sorted ends, ALL via packMessageRange) when the server lacks ESEARCH

New API

// Existing call — unchanged, returns number[]
const uids = await client.search({ seen: false }, { uid: true });

// New: ESEARCH with returnOptions — returns ESearchResult
const result = await client.search({ seen: false }, {
  uid: true,
  returnOptions: ['COUNT', 'MIN', 'MAX', 'ALL']
});
// result = { count: 1847, min: 1001, max: 87432, all: '1001,1005:1010,...' }

// New: Paged search via RFC 9394 PARTIAL
const page = await client.search({ before: new Date('2022-01-01') }, {
  uid: true,
  returnOptions: ['COUNT', { partial: '1:100' }]
});
// page = { count: 34201, partial: { range: '1:100', messages: '2001,2003:2020,...' } }

Motivation

For large mailboxes (70k+ messages), client.search() currently returns every matching UID as a number[], which is expensive to transfer and parse. ESEARCH allows:

  • COUNT only — single integer instead of full UID list (O(1) vs O(n) on the wire)
  • MIN/MAX — boundary UIDs without materializing the full set
  • ALL as compact sequence-set"1,5:10,20" instead of space-separated individual UIDs
  • PARTIAL (RFC 9394) — server-side paged results

Implementation notes

  • Capability check: ESEARCH RETURN clause is only emitted when connection.capabilities.has('ESEARCH')
  • The IMAP parser represents parenthesized groups as plain JS Arrays, not {type: 'LIST'} objects — the ESEARCH handler checks both forms for robustness
  • Unknown ESEARCH keywords (e.g. MODSEQ from CONDSTORE) are silently skipped with correct stream alignment
  • returnOptions strings are normalized to uppercase in both the command builder and fallback path

Tested against

  • Dovecot with ESEARCH capability — 150k message mailbox
  • Verified COUNT, ALL, MIN, MAX all return correct results
  • PARTIAL handled gracefully when server doesn't advertise the capability

Extend search() to accept returnOptions for efficient server-side
result summarization. When the server advertises ESEARCH capability,
emits UID SEARCH RETURN (...) and parses the ESEARCH untagged response
into an ESearchResult object (count, min, max, all, partial).

Key changes:
- ESEARCH response parser (parseEsearchResponse) in search.js
- RETURN clause builder with PARTIAL range support
- Client-side ESearchResult derivation when server lacks ESEARCH
- TypeScript ESearchResult interface and search() overloads
- 13 nodeunit tests (parser, command builder, fallback, compat)

Backward compatible: callers without returnOptions still receive
number[] exactly as before.
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Mar 28, 2026

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants