check block subtrees optimisation

[freemans13]

No description provided.

[github-actions]

🤖 Claude Code Review

Status: Complete

---

Critical Issue Found:

• O(n²) bubble sort at lines 1204-1209 - This creates catastrophic performance degradation for large blocks. For 8M transactions, performs ~64 trillion comparisons vs 184 million with sort.Slice (347,826x slower). Must fix before production use.

History:

✅ Fixed Issues:

• Dependency detection bugs in both prefetchAndCacheParentUTXOs and pipelined processing - now correctly use two-pass approach
• All previously reported critical bugs have been resolved

Positive Notes:

The optimization shows strong fundamentals with batch UTXO prefetching, parallel processing strategies, and comprehensive testing. The two-pass dependency tracking is now correctly implemented throughout.

[github-actions]

New inline comment added for line 367 in check_block_subtrees.go:

Critical Bug: Chunking breaks transaction dependencies

The chunking logic splits transactions across chunk boundaries without considering dependencies. A child transaction in chunk 2 may try to validate before its parent from chunk 1 is committed to the UTXO store.

Example: If tx3 in chunk 1 hasn't been committed when chunk 2 starts, tx4 (which spends from tx3) will fail validation.

Fix: Remove the chunking or implement dependency-aware chunking that keeps dependent transactions together.

[github-actions]

🤖 Claude Code Review

Status: Complete

Summary

This PR implements a significant performance optimization for block subtree validation, achieving 2-5x throughput improvements through batch UTXO prefetching and adaptive processing strategies. The implementation is well-architected with comprehensive tests and benchmarks.

Highlights

Performance improvements:

• Batch UTXO prefetching eliminates per-transaction round-trips
• Validation time stays nearly constant as transaction count increases (125ms for 41 txs vs 123ms for 191 txs)
• Adaptive strategy selection: level-based for <100 txs, pipelined for ≥100 txs
• 2.4x throughput at threshold (100 txs), 4.7x for large blocks (191 txs)

Code quality:

• Critical bug fixes applied (two-pass dependency detection)
• Extensive use of object pools to reduce GC pressure (640MB+ savings)
• Comprehensive test coverage across 3 backend stores (SQLite/Postgres/Aerospike)
• Production-realistic benchmarks with real Aerospike + validator

Review Notes

All previously reported critical bugs have been fixed in the current code:

• Prefetch parent detection now uses two-pass algorithm (lines 652-684)
• Pipeline dependency graph builds complete txMap first (lines 1203-1222)

The optimization approach is sound and follows best practices for high-throughput transaction processing.

History:

• ✅ Fixed: Critical parent detection bugs (two-pass algorithm implemented)
• ✅ Verified: Memory pooling strategy reduces allocations significantly
• ✅ Confirmed: Test coverage validates correctness across all backends

[github-actions]

Critical Performance Issue: O(n²) bubble sort

This bubble sort has O(n²) complexity. For 8M transactions, this performs ~64 trillion comparisons and will cause catastrophic performance degradation.

Impact: This single sort could take hours or days for large blocks, making the entire optimization worthless.

Fix: Use standard library sort which is O(n log n):

sort.Slice(sortedTxs, func(i, j int) bool {
    return sortedTxs[i].level < sortedTxs[j].level
})

For 8M transactions:

• Bubble sort: ~64 trillion comparisons
• sort.Slice: ~184 million comparisons (347,826x faster)

This is blocking the entire optimization from being production-ready.

[sonarqubecloud]

Quality Gate failed

Failed conditions
56.6% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud