Skip to content

perf(bdk): harden pg wallet cache and large-wallet sync paths#59

Merged
dolcalmi merged 15 commits intomainfrom
perf/bdk-cache-miss-short-circuit
Apr 17, 2026
Merged

perf(bdk): harden pg wallet cache and large-wallet sync paths#59
dolcalmi merged 15 commits intomainfrom
perf/bdk-cache-miss-short-circuit

Conversation

@dolcalmi
Copy link
Copy Markdown

@dolcalmi dolcalmi commented Apr 17, 2026

Summary

  • refactor SqlxWalletDb internals by splitting cache, lookup, and trait implementation concerns for readability while preserving behavior
  • improve large-wallet performance by paging and streaming script/transaction loaders, adding miss-batch resolution, and reducing repeated DB lookups in hot sync paths
  • harden correctness around raw-vs-summary transaction lookups, staged tx deletion semantics, and post-commit cache degradation handling (with cache invalidation + targeted tests)

@dolcalmi dolcalmi requested a review from Copilot April 17, 2026 05:47
Copilot AI reviewed Apr 17, 2026
View reviewed changes
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors the Postgres-backed BDK wallet database implementation to separate caching, lookup, and trait implementations, while improving large-wallet sync performance by adding paged/streamed loaders and batched miss resolution.

Changes:

  • Split SqlxWalletDb responsibilities into new cache, lookups, and db_traits modules (plus updated tests).
  • Add batched lookup helpers (find_by_ids, find_paths_for_scripts) and convert full-table loads to paged streaming queries.
  • Update SQLx offline metadata to reflect new/modified queries.

Reviewed changes

Copilot reviewed 10 out of 13 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
src/bdk/pg/utxos.rs Removes non-transactional persist_all helper, leaving transactional persist path.
src/bdk/pg/transactions.rs Adds paging/streaming loaders and find_by_ids for batched tx fetches.
src/bdk/pg/script_pubkeys.rs Adds batched script-path lookup and paged/streamed script listing.
src/bdk/pg/mod.rs Reorganizes module structure; updates/extends WalletCache tests.
src/bdk/pg/lookups.rs Introduces centralized lookup logic + miss batching policy.
src/bdk/pg/db_traits.rs Moves BDK trait implementations and adds post-commit cache invalidation behavior.
src/bdk/pg/cache.rs New wallet cache implementation, including miss/pending tracking and invalidation.
.sqlx/query-*.json Adds/updates SQLx offline query metadata for new/changed SQL.
Files not reviewed (3)
  • .sqlx/query-1c966653b607ef9cd8a305aac131cf64a17ab0803fdc55db8b55a14fced7178d.json: Language not supported
  • .sqlx/query-456fb4994f750068fc126ac3add78fbc539b5073bdfa1265760869e094353ade.json: Language not supported
  • .sqlx/query-fbfacf3613c2dbe990a8c02ea2755ea2c981b717f87a42e83cd934d2abf33443.json: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/bdk/pg/transactions.rs Outdated
Comment thread src/bdk/pg/lookups.rs Outdated
Comment thread src/bdk/pg/lookups.rs Outdated
Comment thread src/bdk/pg/cache.rs
Comment thread src/bdk/pg/cache.rs
Comment thread src/bdk/pg/transactions.rs Outdated
@dolcalmi dolcalmi requested a review from Copilot April 17, 2026 06:04
Copilot AI reviewed Apr 17, 2026
View reviewed changes
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 13 changed files in this pull request and generated 2 comments.

Files not reviewed (3)
  • .sqlx/query-1c966653b607ef9cd8a305aac131cf64a17ab0803fdc55db8b55a14fced7178d.json: Language not supported
  • .sqlx/query-68df480b792fa014a5d66aa05f2d7caa1305d81382f32f2b55c36d31fa8e3583.json: Language not supported
  • .sqlx/query-fbfacf3613c2dbe990a8c02ea2755ea2c981b717f87a42e83cd934d2abf33443.json: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/bdk/pg/transactions.rs
Comment thread src/bdk/pg/transactions.rs
@dolcalmi dolcalmi requested a review from Copilot April 17, 2026 06:16
Copilot AI reviewed Apr 17, 2026
View reviewed changes
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 13 changed files in this pull request and generated 1 comment.

Files not reviewed (3)
  • .sqlx/query-1c966653b607ef9cd8a305aac131cf64a17ab0803fdc55db8b55a14fced7178d.json: Language not supported
  • .sqlx/query-68df480b792fa014a5d66aa05f2d7caa1305d81382f32f2b55c36d31fa8e3583.json: Language not supported
  • .sqlx/query-fbfacf3613c2dbe990a8c02ea2755ea2c981b717f87a42e83cd934d2abf33443.json: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/bdk/pg/transactions.rs Outdated
@dolcalmi dolcalmi requested a review from Copilot April 17, 2026 06:28
Copilot AI reviewed Apr 17, 2026
View reviewed changes
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 13 changed files in this pull request and generated 1 comment.

Files not reviewed (3)
  • .sqlx/query-1c966653b607ef9cd8a305aac131cf64a17ab0803fdc55db8b55a14fced7178d.json: Language not supported
  • .sqlx/query-68df480b792fa014a5d66aa05f2d7caa1305d81382f32f2b55c36d31fa8e3583.json: Language not supported
  • .sqlx/query-fbfacf3613c2dbe990a8c02ea2755ea2c981b717f87a42e83cd934d2abf33443.json: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/bdk/pg/cache.rs
@dolcalmi dolcalmi requested a review from Copilot April 17, 2026 06:36
Copilot AI reviewed Apr 17, 2026
View reviewed changes
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 13 changed files in this pull request and generated 1 comment.

Files not reviewed (3)
  • .sqlx/query-1c966653b607ef9cd8a305aac131cf64a17ab0803fdc55db8b55a14fced7178d.json: Language not supported
  • .sqlx/query-68df480b792fa014a5d66aa05f2d7caa1305d81382f32f2b55c36d31fa8e3583.json: Language not supported
  • .sqlx/query-fbfacf3613c2dbe990a8c02ea2755ea2c981b717f87a42e83cd934d2abf33443.json: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/bdk/pg/transactions.rs Outdated
@dolcalmi dolcalmi requested a review from Copilot April 17, 2026 06:45
Copilot AI reviewed Apr 17, 2026
View reviewed changes
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 13 changed files in this pull request and generated 1 comment.

Files not reviewed (3)
  • .sqlx/query-1c966653b607ef9cd8a305aac131cf64a17ab0803fdc55db8b55a14fced7178d.json: Language not supported
  • .sqlx/query-68df480b792fa014a5d66aa05f2d7caa1305d81382f32f2b55c36d31fa8e3583.json: Language not supported
  • .sqlx/query-fbfacf3613c2dbe990a8c02ea2755ea2c981b717f87a42e83cd934d2abf33443.json: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/bdk/pg/cache.rs
@dolcalmi dolcalmi requested a review from Copilot April 17, 2026 06:54
Copilot AI reviewed Apr 17, 2026
View reviewed changes
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 13 changed files in this pull request and generated 4 comments.

Files not reviewed (3)
  • .sqlx/query-1c966653b607ef9cd8a305aac131cf64a17ab0803fdc55db8b55a14fced7178d.json: Language not supported
  • .sqlx/query-68df480b792fa014a5d66aa05f2d7caa1305d81382f32f2b55c36d31fa8e3583.json: Language not supported
  • .sqlx/query-fbfacf3613c2dbe990a8c02ea2755ea2c981b717f87a42e83cd934d2abf33443.json: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/bdk/pg/db_traits.rs
Comment thread src/bdk/pg/lookups.rs Outdated
Comment thread src/bdk/pg/lookups.rs Outdated
Comment thread src/bdk/pg/transactions.rs Outdated
@dolcalmi dolcalmi requested a review from Copilot April 17, 2026 07:10
Copilot AI reviewed Apr 17, 2026
View reviewed changes
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 13 changed files in this pull request and generated no new comments.

Files not reviewed (3)
  • .sqlx/query-1c966653b607ef9cd8a305aac131cf64a17ab0803fdc55db8b55a14fced7178d.json: Language not supported
  • .sqlx/query-68df480b792fa014a5d66aa05f2d7caa1305d81382f32f2b55c36d31fa8e3583.json: Language not supported
  • .sqlx/query-fbfacf3613c2dbe990a8c02ea2755ea2c981b717f87a42e83cd934d2abf33443.json: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@dolcalmi dolcalmi merged commit d2993fb into main Apr 17, 2026
5 checks passed
@dolcalmi dolcalmi deleted the perf/bdk-cache-miss-short-circuit branch April 17, 2026 07:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@dolcalmi