[Due for payment 2026-04-14] [$250] [Sentry] fix The database connection is closing issue

[mountiny]

Sentry issue https://expensify.sentry.io/issues/7092239780

Thread https://expensify.slack.com/archives/C05LX9D6E07/p1770496102994319

Please re-state the problem that we are trying to solve in this issue.

When Onyx operations attempt to create an IndexedDB transaction, the browser may have already closed the database connection, causing:
InvalidStateError: Failed to execute 'transaction' on 'IDBDatabase': The database connection is closing.

This affects ~58,700 occurrences across 430 users on all browsers (Chrome: 31K, Safari: 18K, Firefox: 5K, Mobile Safari: 3K, Edge: 1K). The error is an unhandled promise rejection.

What is the root cause of that problem?

The existing db.onclose handler (line 22) sets dbp = undefined when the browser closes the connection, which protects future calls to getDB(). However, any operations that have already resolved the dbp promise still hold a stale db reference and proceed to call db.transaction() at line 59 — which throws InvalidStateError on the now-closing connection.

This can be triggered by:

• The browser closing idle IDB connections (Safari is especially aggressive, but Chrome/Firefox do this too under memory pressure or tab suspension)
• Another tab triggering a version upgrade (versionchange event — currently unhandled)
• The verifyStoreExists path closing the DB (line 39) while concurrent operations still hold the old reference

What changes do you think we should make in order to solve the problem?

All changes are in lib/storage/providers/IDBKeyValProvider/createStore.ts:

1. Retry mechanism with a single retry on InvalidStateError
      — Extract the transaction chain (getDB → verifyStoreExists → db.transaction) into openTransactionAndExecute()
      — Wrap the call in .catch(): if the error is InvalidStateError, reset dbp = undefined and retry once with a fresh connection
      — If the retry also fails, the error is thrown normally and still surfaces in Sentry exactly as it does today

2. Diagnostic logging on every retry (does NOT hide the issue)
      — Log at alert level with: dbName, storeName, txMode, and closedBy
      — closedBy tracks why the connection was closed: 'browser' (user-agent closed it), 'versionchange' (cross-tab upgrade), 'verifyStoreExists' (missing store path), or 'unknown'
      — This gives us more visibility than today (where we only get a crash count with no context)

3. onversionchange handler
      — When another tab triggers a DB version upgrade, proactively close the connection and reset dbp
      — Without this, cross-tab upgrades can block indefinitely and in-flight operations crash on the stale connection

4. onclose handler now logs
      — The existing onclose handler now logs when the browser unexpectedly closes the connection, giving us data on browser-initiated closures even when no retry is needed

Key safety guarantees:

• Data is never lost — the transaction fails before any data is written; the retry completes successfully with a valid connection
• Only one retry is attempted — no infinite loops
• If the retry fails, the error propagates normally and Sentry captures it as before
• Normal operations (no InvalidStateError) are completely unaffected — zero performance impact
• The closedBy logging gives us instrumentation to continue investigating the root cause of the spike

cc @fabioh8010

Issue Owner

Current Issue Owner: @mkhutornyi

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~022029317821194916452
• Upwork Job ID: 2029317821194916452
• Last Price Increase: 2026-03-04

[MelvinBot]

Proposal

Please re-state the problem that we are trying to solve in this issue.

When Onyx operations attempt to create an IndexedDB transaction, the browser may have already closed the database connection, causing InvalidStateError: Failed to execute 'transaction' on 'IDBDatabase': The database connection is closing. This affects ~58,700 occurrences across 430 users on all browsers (Chrome: 31K, Safari: 18K, Firefox: 5K, Mobile Safari: 3K, Edge: 1K). The error is an unhandled promise rejection that currently has zero error handling.

What is the root cause of that problem?

There are two race conditions in react-native-onyx's createStore.js, both converging on the same unprotected db.transaction() call at the return statement:

return (txMode, callback) => getDB()
    .then(verifyStoreExists)
    .then((db) => callback(db.transaction(storeName, txMode).objectStore(storeName)));

Race Condition A (primary — ~58K occurrences): getDB() returns the cached dbp promise which resolves to a db object. Between the promise resolving and db.transaction() executing, the browser closes the connection (Safari does this aggressively for idle connections; Chrome/Firefox do it under memory pressure or tab suspension). The db.onclose handler fires and sets dbp = undefined, which helps future calls to getDB() re-open the connection — but the in-flight .then() chain already has the stale db reference. db.transaction() throws InvalidStateError synchronously inside the .then() callback.

Race Condition B (secondary): The verifyStoreExists function (added in PR #684) calls db.close() when it detects the store is missing and needs to recreate it with a version bump. This immediately puts the connection into "closing" state. Any concurrent operation that already resolved getDB() with this same db object will hit InvalidStateError when it reaches db.transaction().

Why the error is unhandled: All 9 storage operations (7 via idb-keyval, 2 direct calls in multiMerge/multiSet) flow through the same single choke point. None of them have any try/catch or error handling around db.transaction(). The tryOrDegradePerformance wrapper in storage/index.js only catches synchronous throws — since db.transaction() is called inside an async .then() chain, the InvalidStateError becomes a rejected promise that bypasses the try/catch entirely. Additionally, tryOrDegradePerformance only checks for "Internal error opening backing store", not InvalidStateError.

There is also no onversionchange handler, meaning cross-tab database upgrades can forcibly close connections and trigger the same error.

What changes do you think we should make in order to solve the problem?

All changes in react-native-onyx's lib/storage/providers/IDBKeyValProvider/createStore.ts:

1. Wrap db.transaction() with retry on InvalidStateError — Extract the transaction chain into a helper. If db.transaction() throws InvalidStateError, reset dbp = undefined and retry once with a fresh connection. If the retry also fails, propagate the error normally (Sentry still captures it). This is the standard approach used by other libraries facing the same issue (localForage, Firebase, Workbox).

2. Add onversionchange handler — When another tab triggers a DB version upgrade, proactively close the connection and reset dbp before the browser force-closes it. This prevents the race window where in-flight operations use a connection that's about to be killed.

3. Add diagnostic logging on retry — Log dbName, storeName, txMode, and the reason the connection was closed (browser, versionchange, verifyStoreExists, or unknown). This gives visibility into which trigger is most common without hiding the error.

4. Enhance onclose handler with logging — The existing onclose handler silently resets dbp. Adding a log line gives production visibility into browser-initiated closures.

What alternative solutions did you explore? (Optional)

• Degrade to MemoryOnlyProvider: Too aggressive — the IDB connection can be re-established with a single retry. Degrading to memory-only loses all persistent state.
• Serialize all operations through a queue: Prevents Race Condition B but doesn't help with Race Condition A (browser-initiated closures), and adds significant overhead to all IDB operations.
• Check db.closePending before db.transaction(): IndexedDB doesn't expose a reliable connection state property — the only way to know the connection is closing is to catch the error.

Detailed Investigation

Evidence Chain

1. The single choke point — createStore.js line 83: db.transaction(storeName, txMode) is called inside a .then() with zero error handling. Every storage operation flows through this line.

2. Stale reference problem — db.onclose (line 54) sets dbp = undefined for future callers, but the promise .then() chain already has the resolved db object. The stale db reference is used to call db.transaction(), which throws.

3. No upstream protection — idb-keyval v6.2.1 has the identical unprotected pattern. Its functions (get, set, del, getMany, delMany, clear, keys) simply call customStore(txMode, callback) with no try/catch.

4. tryOrDegradePerformance doesn't help — It wraps fn() in try/catch, but fn() returns a promise. The InvalidStateError is thrown inside an async .then() chain, becoming a rejected promise that the synchronous try/catch cannot intercept. The error message also doesn't match either of the two checked patterns.

5. verifyStoreExists secondary race — Added in PR #684 (merged Oct 2025) to handle corrupted databases from third-party scripts. When the store is missing, it calls db.close() and reopens with a version bump. Any concurrent operation holding the old db reference will fail.

6. Missing onversionchange — No handler exists anywhere in the codebase. Cross-tab upgrades (including those triggered by verifyStoreExists) can force-close connections on other tabs.

Relevant Code

• createStore.js:81-83 — The unprotected db.transaction() call (single choke point)
• createStore.js:50-54 — db.onclose handler that only helps future calls
• createStore.js:62-79 — verifyStoreExists with db.close() race
• storage/index.js:62-85 — tryOrDegradePerformance that doesn't catch this error
• idb-keyval/dist/index.cjs:32-46 — Upstream functions with zero error handling

Open Questions

• What percentage of the ~58,700 errors come from Safari vs other browsers? The issue reports Chrome: 31K, Safari: 18K — confirming this isn't Safari-only (despite the original code comment suggesting Safari is the culprit).
• How often does verifyStoreExists actually find the store missing? If rarely, Race Condition A (browser-initiated closures) dominates.
• Does the app commonly run in multiple tabs? If so, the onversionchange handler is critical.

---

Next Steps for Contributor+ team: Reply with @MelvinBot implement this to create a draft PR, or @MelvinBot <your feedback> to refine this analysis.

[melvin-bot]

Current assignee @Krishna2323 is eligible for the External assigner, not assigning anyone new.

[melvin-bot]

Job added to Upwork: https://www.upwork.com/jobs/~022029317821194916452

[melvin-bot]

@Krishna2323 Huh... This is 4 days overdue. Who can take care of this?