perf(chain-db): speed up chain sync batching and rollback handling#720
Merged
AndrewWestberg merged 1 commit intomasterfrom Jan 31, 2026
Merged
perf(chain-db): speed up chain sync batching and rollback handling#720AndrewWestberg merged 1 commit intomasterfrom
AndrewWestberg merged 1 commit intomasterfrom
Conversation
There was a problem hiding this comment.
Pull request overview
Improve initial chain sync throughput by batching DB writes, reducing per-block query overhead, and preventing unbounded catch-up buffering while keeping rollback semantics correct.
Changes:
- Cap sync batch growth during catch-up via configurable
maxCatchupBlockBufferSize. - Gate rollback work to RollBackward events and add per-block metadata lookup caching.
- Batch chain inserts / UTXO creation + spending updates; bump Gradle and Kogmios versions; add perf PRD/task docs.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| newm-chain/src/main/kotlin/io/newm/chain/daemon/MonitorAddressDaemon.kt | Adds config-driven cap for catch-up buffer growth. |
| newm-chain/src/main/kotlin/io/newm/chain/daemon/BlockDaemon.kt | Adds rollback gating, buffer-cap logic, and per-block metadata caching. |
| newm-chain-db/src/main/kotlin/io/newm/chain/database/repository/LedgerRepositoryImpl.kt | Reworks spent/created UTXO writes into batch SQL/batchInsert to reduce N+1 and update overhead. |
| newm-chain-db/src/main/kotlin/io/newm/chain/database/repository/ChainRepositoryImpl.kt | Switches chain writes to batchInsert and computes etaV in-memory per batch. |
| gradle/wrapper/gradle-wrapper.properties | Updates Gradle wrapper version. |
| buildSrc/src/main/kotlin/Versions.kt | Updates Kogmios version constant. |
| .opencode | Adds agent config entry. |
| .agent/task/newm-chain-db-improvement/newm-chain-db-performance-tasks.json | Adds tracked performance tasks list. |
| .agent/task/newm-chain-db-improvement/newm-chain-db-performance-prd.md | Adds PRD describing performance plan and completion status. |
| .agent/skills/git-commit-formatter/SKILL.md | Adds skill doc for conventional commit formatting. |
newm-chain-db/src/main/kotlin/io/newm/chain/database/repository/ChainRepositoryImpl.kt
Show resolved
Hide resolved
newm-chain-db/src/main/kotlin/io/newm/chain/database/repository/ChainRepositoryImpl.kt
Show resolved
Hide resolved
newm-chain-db/src/main/kotlin/io/newm/chain/database/repository/LedgerRepositoryImpl.kt
Show resolved
Hide resolved
newm-chain-db/src/main/kotlin/io/newm/chain/database/repository/LedgerRepositoryImpl.kt
Show resolved
Hide resolved
6faa55c to
45b8a97
Compare
wlara
approved these changes
Jan 30, 2026
Batch chain and ledger writes, cache per-block metadata lookups, and cap catch-up buffer growth to improve initial sync performance. Update gradle/kogmios versions and record perf PRD tasks.
45b8a97 to
427b7d9
Compare
Member
Author
|
Dropping the prompt here for future dev's reference. I looped on this until the PRD was complete clearing the context window before each loop. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Batch chain and ledger writes, cache per-block metadata lookups, and cap catch-up buffer growth to improve initial sync performance. Update gradle/kogmios versions and record perf PRD tasks.