perf(sync): reduce wallet sync lookup churn and add tracing#58
Merged
perf(sync): reduce wallet sync lookup churn and add tracing#58
Conversation
Restructure SqlxWalletDb internals to isolate context, batch state, and shared caches while keeping BDK trait behavior compatible. This avoids eager cache hydration side effects, makes cache errors recoverable, and commits DB writes before cache mutation for safer sync behavior at large wallet scale.
Move keychain filtering into SQL and add a lightweight iter_txs(false) path to reduce wallet sync query cost. Replace panic-prone unimplemented/unwrap/expect paths in pg wallet database adapters with recoverable errors to avoid process crashes under inconsistent data.
Simplify control flow and batching code in the Postgres BDK adapters without changing behavior. This keeps recent sync performance optimizations while reducing duplication and making cache/DB paths easier to reason about.
Execute script pubkeys, utxos, and transactions persistence in a single SQL transaction during commit_batch to avoid partial DB state on failures. Also unify tx lookup behavior for raw vs summary reads, remove unused script_pubkeys load-all path, and document retained non-transactional persist helpers.
Avoid repeated full load_all deserialization in iter_txs(true) by memoizing the raw transaction cache after first successful load. Preserve cache correctness for staged updates and add focused unit coverage for the raw-cache-loaded flag.
Add focused unit coverage for transaction cache behavior used by iter_txs summary/raw paths, and harden summary loading with negative-height validation. Also strengthen raw-cache-loaded atomic ordering with acquire/release semantics.
Reduce duplication and clarify control flow in the BDK Postgres adapters while preserving the recent atomic commit and cache semantics. This keeps behavior unchanged but makes error handling and query paths easier to reason about.
Respect the Database::get_tx include_raw flag by stripping raw transaction data when requested. Also document the process-local raw tx cache hint and clarify commit_batch atomic scope boundaries.
Warm script-path and tx-summary caches during iteration to reduce repeated find_path/find_by_id lookups in wallet sync. Treat soft-delete cleanup for already-settled UTXOs as a non-fatal condition to prevent retry cascades.
Add tracing spans across SqlxWalletDb BatchOperations and Database trait methods so sync call ordering and cache-hit paths are visible in traces.
There was a problem hiding this comment.
Pull request overview
This PR aims to improve wallet sync performance and observability by reducing repeated DB lookups (via cache hydration), making a specific UTXO cleanup error non-fatal, and adding tracing spans around BDK database/batch operations.
Changes:
- Treat
UtxoError::UtxoAlreadySettledErrorduring soft-deleted UTXO cleanup as non-fatal by rolling back and undeleting the BDK UTXO row. - Add a
list_scripts_with_pathsquery and use it to hydrate an in-process script pubkey+path cache, tracked via a loaded bitmask. - Add
tracing::instrumentspans toSqlxWalletDbBatchOperationsandDatabasetrait method implementations.
Reviewed changes
Copilot reviewed 3 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
src/job/sync_wallet.rs |
Makes soft-deleted UTXO cleanup resilient to already-settled Bria UTXOs by restoring BDK visibility and continuing. |
src/bdk/pg/script_pubkeys.rs |
Adds a scripts+paths query/helper to support bulk cache hydration and avoid repeated find_path calls. |
src/bdk/pg/mod.rs |
Introduces script pubkey hydration flags and uses bulk loading to warm caches; adds tracing around DB/batch calls. |
.sqlx/query-a49bfbc8c8ca875c146ac678f5f5633b758af6a7070b939032394b6525d4a7c3.json |
SQLx offline metadata for the new keychain_id + keychain_kind scripts-with-paths query. |
.sqlx/query-846df15d0a9709ad69c5ff8e6c3e6679816cf5c9f86ea2b4a9d1b95458ec56a1.json |
SQLx offline metadata for the new keychain_id scripts-with-paths query. |
Files not reviewed (2)
- .sqlx/query-846df15d0a9709ad69c5ff8e6c3e6679816cf5c9f86ea2b4a9d1b95458ec56a1.json: Language not supported
- .sqlx/query-a49bfbc8c8ca875c146ac678f5f5633b758af6a7070b939032394b6525d4a7c3.json: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 5 changed files in this pull request and generated 2 comments.
Files not reviewed (2)
- .sqlx/query-846df15d0a9709ad69c5ff8e6c3e6679816cf5c9f86ea2b4a9d1b95458ec56a1.json: Language not supported
- .sqlx/query-a49bfbc8c8ca875c146ac678f5f5633b758af6a7070b939032394b6525d4a7c3.json: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 5 changed files in this pull request and generated no new comments.
Files not reviewed (2)
- .sqlx/query-846df15d0a9709ad69c5ff8e6c3e6679816cf5c9f86ea2b4a9d1b95458ec56a1.json: Language not supported
- .sqlx/query-a49bfbc8c8ca875c146ac678f5f5633b758af6a7070b939032394b6525d4a7c3.json: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary