An experimental aggregation node for the Unicity network. Accepts client transaction certification requests, batches them into rounds, proposes each round's state transition to BFT Core for certification, and returns verifiable inclusion proofs backed by a Radix / Sparse Merkle Tree (SMT).
Focus areas:
- SMT structure providing efficient consistency proofs for trustless operation
- Consistency proofs for every round of operation
- Scaling beyond available system memory (fully disk-backed SMT)
- Speculative execution of the next round while waiting for BFT certification
- Configurable persistence — trade off restart speed vs. memory footprint vs. performance
clients ──POST /──> JSON-RPC server ──mpsc──> RoundManager<S: SmtStore>
│
AggregatorState <───────────────┤ commit certified state
│ │
DashMap<StateID, ├─ BFT Core (libp2p)
RecordInfo> │
│ SmtStore (generic)
get_inclusion_proof ├─ MemSmt (all in RAM)
└─ DiskSmt (RocksDB-backed)
Request flow:
POST /— JSON-RPC 2.0 dispatchcertification_request— hex-CBOR payload → predicate + signature + StateID validation → queued inRoundManagerviampscRoundManager— collects requests, fires a round on a timer (default 1 s) or batch-size limit; creates an SMT snapshot, inserts leaves, proposes root hash to BFT Core, awaits the Unicity Certificate (UC)- On UC success — commits snapshot, stores finalized records; on UC failure — discards snapshot, re-queues requests
get_inclusion_proof.v2— returns[blockNumber, [certData, merklePathCbor, ucCbor]]from the certified SMT
Speculative execution: while waiting for the UC (~1.5 s), the next block's requests are inserted speculatively into a forked snapshot. On UC success the speculative snapshot is immediately promoted, eliminating dead time between rounds. On UC failure both snapshots are discarded and all requests are re-queued. The disk-backed path additionally uses a layered overlay so the speculative snapshot can read the proposed round's uncommitted mutations without any RocksDB writes.
| Crate | Description |
|---|---|
crates/rsmt |
Standalone Sparse Merkle Tree library — path-compressed Merkle Patricia trie, consistency proofs, serialisation. No async, no I/O. |
crates/smt-store |
SMT storage backends behind a common SmtStore / SmtStoreSnapshot trait. Contains MemSmt (fully in-memory, optional DB persistence) and DiskSmt (lazy disk-backed via RocksDB). |
crates/aggregator |
The aggregator service — HTTP API, generic RoundManager<S: SmtStore>, BFT Core connectivity, application-level RocksDB persistence. |
| Tool | Minimum version | Notes |
|---|---|---|
| Rust | 1.75 | rustup update stable |
| C compiler | any | required by secp256k1-sys and librocksdb-sys |
| CMake | 3.x | required by librocksdb-sys |
| Clang / LLVM | any | required by bindgen (RocksDB) |
On macOS all dependencies arrive via Xcode Command Line Tools + Homebrew (brew install cmake llvm). On Debian/Ubuntu: apt install build-essential cmake clang.
Note: RocksDB is always compiled. The first build compiles librocksdb from source and takes several minutes. Subsequent builds are incremental.
# Build everything
cargo build --workspace
# Release build
cargo build --workspace --releaseThe fastest way to bring up a standalone aggregator for development or testing:
cargo run --release -p uni-aggregator --bin aggregator -- \
--bft-mode stub \
--listen 0.0.0.0:8080 \
--round-duration-ms 800 \
--batch-limit 50000The stub BFT committer signs its own UCs with a hardcoded test key — no BFT Core process needed. With no --db-path the aggregator runs fully in-memory; state is lost on restart.
Use --smt-backend to choose how the SMT tree is stored and recovered. All five modes share the same RoundManager<S: SmtStore> code path.
| Backend | Flag | Requires --db-path |
Restart behaviour |
|---|---|---|---|
| Pure in-memory | --smt-backend mem |
No | State lost on restart; BFT partition state must be reset too |
| In-memory + leaf persistence | --smt-backend mem-leaves |
Yes | Replays all leaves from smt_leaves CF to rebuild tree; verifies root hash |
| In-memory + leaf persistence + shutdown snapshot | --smt-backend mem-leaves-x |
Yes | Like mem-leaves during operation; on graceful shutdown (ctrl+c / SIGTERM) saves full internal nodes for faster recovery on next start |
| In-memory + full persistence | --smt-backend mem-full |
Yes | Loads complete node tree from smt_nodes CF directly; faster restart than mem-leaves |
| Fully disk-backed | --smt-backend disk |
Yes | Only root hash loaded at start; nodes materialised on demand from RocksDB |
When --smt-backend is omitted it defaults to disk if --db-path is set, mem otherwise.
# In-memory with leaf persistence (good balance for medium trees)
cargo run --release -p uni-aggregator --bin aggregator -- \
--bft-mode stub \
--db-path /var/lib/aggregator/db \
--smt-backend mem-leaves
# In-memory with leaf persistence + shutdown snapshot (best balance)
cargo run --release -p uni-aggregator --bin aggregator -- \
--bft-mode stub \
--db-path /var/lib/aggregator/db \
--smt-backend mem-leaves-x
# In-memory with full node persistence (fastest restart)
cargo run --release -p uni-aggregator --bin aggregator -- \
--bft-mode stub \
--db-path /var/lib/aggregator/db \
--smt-backend mem-full
# Fully disk-backed (unbounded tree size, working set bounded by cache)
cargo run --release -p uni-aggregator --bin aggregator -- \
--bft-mode stub \
--db-path /var/lib/aggregator/db \
--smt-backend disk \
--cache-mb 256cargo run --release -p uni-aggregator --bin aggregator -- \
--bft-mode live \
--bft-peer-id <BFT_CORE_PEER_ID> \
--bft-addr /ip4/<BFT_HOST>/tcp/26652 \
--p2p-addr /ip4/0.0.0.0/tcp/0 \
--auth-key-hex <32-byte-hex-secp256k1-auth-key> \
--sig-key-hex <32-byte-hex-secp256k1-signing-key> \
--partition-id 1 \
--uc-timeout-ms 15000 \
--db-path /var/lib/aggregator/db \
--smt-backend disk \
--consistency-proof-mode rsmtAll flags are also readable from environment variables (AGGREGATOR_*):
| Flag | Env | Default | Description |
|---|---|---|---|
--listen |
AGGREGATOR_LISTEN |
0.0.0.0:8080 |
HTTP listen address |
--round-duration-ms |
AGGREGATOR_ROUND_DURATION_MS |
1000 |
Round timer (ms) |
--batch-limit |
AGGREGATOR_BATCH_LIMIT |
1000 |
Max requests per round |
--bft-mode |
AGGREGATOR_BFT_MODE |
stub |
stub or live |
--consistency-proof-mode |
AGGREGATOR_CONSISTENCY_PROOF_MODE |
off |
off, rsmt (hash-based), or zk (SP1 ZK proof) |
--zk-proof-kind |
AGGREGATOR_ZK_PROOF_KIND |
compressed |
core, compressed, groth16, plonk — only used when --consistency-proof-mode zk |
--db-path |
AGGREGATOR_DB_PATH |
(empty) | RocksDB directory; empty = in-memory only |
--smt-backend |
AGGREGATOR_SMT_BACKEND |
(auto) | mem, mem-leaves, mem-leaves-x, mem-full, or disk |
--cache-mb |
AGGREGATOR_CACHE_MB |
0 |
RocksDB block cache size in MB (0 = RocksDB default ~8 MB) |
--partition-id |
AGGREGATOR_PARTITION_ID |
1 |
BFT Core partition ID |
--bft-peer-id |
AGGREGATOR_BFT_PEER_ID |
BFT Core root node peer ID | |
--bft-addr |
AGGREGATOR_BFT_ADDR |
/ip4/127.0.0.1/tcp/26652 |
BFT Core multiaddr |
--p2p-addr |
AGGREGATOR_P2P_ADDR |
/ip4/0.0.0.0/tcp/0 |
Our libp2p listen multiaddr |
--auth-key-hex |
AGGREGATOR_AUTH_KEY |
secp256k1 key for libp2p identity | |
--sig-key-hex |
AGGREGATOR_SIG_KEY |
secp256k1 key for signing CRs | |
--uc-timeout-ms |
AGGREGATOR_UC_TIMEOUT_MS |
15000 |
UC inactivity timeout (ms); re-sends handshake to BFT Core when no UC arrives within this period; set to ≥3× BFT Core's T2 for this partition |
--fake-state-transitions |
AGGREGATOR_FAKE_STATE_TRANSITIONS |
false |
Use last UC's hash as PreviousHash instead of the real SMT root |
--log-level |
RUST_LOG |
info |
trace, debug, info, warn, error |
Starting a fresh node against a live BFT Core network:
-
Provision keys. Generate two secp256k1 private keys — one for libp2p identity (
auth-key) and one for signing Certification Requests (sig-key). The BFT Core operator must register the signing key's public key in the network's trust base for yourpartition-id. -
Obtain the BFT Core peer ID and address from the network operator. The peer ID is a libp2p multihash (base58,
12D3...format). -
Create the data directory:
mkdir -p /var/lib/aggregator/db
-
Start the aggregator. On first start with an empty
--db-paththe aggregator begins at block 1 with an empty SMT and immediately connects to BFT Core. BFT Core delivers a sync UC that establishes the initial round number and certified state hash. -
Verify connectivity:
curl http://localhost:8080/health # {"status":"ok","blockNumber":"1"}The block number advances by 1 each time a round is certified.
-
Recovery after restart. Stop the aggregator at any time and restart with the same
--db-pathand--smt-backend. Recovery behaviour depends on the backend — see the table above.
All methods are JSON-RPC 2.0 over POST /.
Submit a state transition for certification.
{"jsonrpc":"2.0","id":1,"method":"certification_request","params":"<hex-encoded CBOR>"}Response (success): {"jsonrpc":"2.0","id":1,"result":{"status":"OK"}}
Retrieve a certified inclusion proof.
{"jsonrpc":"2.0","id":2,"method":"get_inclusion_proof.v2","params":{"stateId":"<hex>"}}Returns a hex-encoded CBOR value: [blockNumber, [certData, merklePathCbor, ucCbor]].
Returns HTTP 404 while the state is pending certification (SDK retries automatically).
{"jsonrpc":"2.0","id":3,"method":"get_block_height","params":null}# All tests
cargo test --workspace
# SMT library only
cargo test -p rsmt
# Aggregator only
cargo test -p uni-aggregator
# SMT storage backends
cargo test -p smt-store
# Run a specific test
cargo test -p rsmt consistency::tests::two_leaf_consistency
cargo test -p smt-store disk::tests::proof_equivalenceCriterion benchmarks are defined in crates/rsmt/benches/smt.rs and crates/smt-store/benches/store.rs. They measure individual operations in isolation with statistical confidence intervals and HTML reports.
# Run all benchmarks (both crates)
cargo bench --workspace
# Run only rsmt benchmarks (batch_insert, verify_consistency, inclusion_proof)
cargo bench -p rsmt --bench smt
# Run only smt-store benchmarks (mem_full_commit, mem_leaves_commit, disk_commit)
cargo bench -p smt-store --bench store
# Run a single group by name filter
cargo bench -p rsmt --bench smt -- batch_insert
cargo bench -p smt-store --bench store -- disk_commit
# Save a named baseline, then compare after a code change
cargo bench -p rsmt --bench smt -- --save-baseline before
# ... make changes ...
cargo bench -p rsmt --bench smt -- --baseline before
# Cross-branch comparison (baseline on main, compare on current branch)
git worktree add ../rugregator-main main
CARGO_TARGET_DIR=/absolute/path/to/rugregator/target \
cargo bench -p rsmt --manifest-path ../rugregator-main/Cargo.toml --bench smt -- --save-baseline main
cargo bench -p rsmt --bench smt -- --baseline main
git worktree remove ../rugregator-mainHTML reports land in target/criterion/. Open target/criterion/report/index.html for the full comparison view.
Measures raw insertion throughput and proof-generation latency for each SMT backend in isolation, with no BFT Core or HTTP overhead.
# Pure in-memory (fastest, no DB overhead)
cargo run --release -p uni-aggregator --bin perf-test -- \
--backend mem --rounds 6 --batch-sizes 1000,5000,10000
# In-memory + leaf persistence (commit = leaf CF write)
cargo run --release -p uni-aggregator --bin perf-test -- \
--backend mem-leaves --rounds 6 --batch-sizes 1000,5000,10000
# In-memory + full node persistence (commit = full node tree write)
cargo run --release -p uni-aggregator --bin perf-test -- \
--backend mem-full --rounds 6 --batch-sizes 1000,5000,10000
# Disk-backed (insert = materialise+insert+overlay; commit = RocksDB write)
cargo run --release -p uni-aggregator --bin perf-test -- \
--backend disk --cache-mb 256 --rounds 6 --batch-sizes 1000,5000,10000
# CSV output for all backends
cargo run --release -p uni-aggregator --bin perf-test -- \
--backend disk --rounds 8 --batch-sizes 1000,5000,10000,25000 --csvIf db file is specified --db-path <fn> then each run inserts batches cumulatively so tree size grows across runs. Mixing backends on different runs works on sensible cases, for example, it is possible to start with mem-leaves-x, write internal nodes at exit, and then switch to disk.
Reported columns:
| Column | Description |
|---|---|
pre_fill |
Leaves already in the tree before this batch |
inserted |
Leaves actually inserted (duplicates skipped) |
leaves/s |
Insertion throughput |
insert |
Time to insert + compute root hash |
commit |
Time to persist the round (RocksDB write for disk/mem-full/mem-leaves) |
proof p50/p95 |
Inclusion proof generation latency percentiles |
# Terminal 1 — start aggregator
cargo run --release -p uni-aggregator --bin aggregator -- \
--bft-mode stub \
--round-duration-ms 800 \
--batch-limit 50000 \
--listen 0.0.0.0:8080
# Terminal 2 — run load generator (TypeScript SDK)
cd scripts/perf-test
npm install
npm run perf-test -- --workers 1 --duration 5Watch round size and certified throughput in the aggregator logs:
INFO round finalized block=12 root=0x… certified=847 submitted=847 spec_queued=312
certified = requests included in the finalized block.
spec_queued = requests already inserted speculatively into the next block.
The SMT tree uses Arc<Branch> children everywhere. SmtSnapshot::create() and fork() are O(1): they clone two Arc reference counts at the root rather than deep-copying the tree. Modified nodes are path-copied (Arc::make_mut unwraps shared nodes in-place; clones only when shared), so speculative execution adds at most O(batch_size × tree_depth) new allocations per round — not another copy of the entire tree.
Peak memory during a speculative round is roughly 1× tree size + O(batch). Without CoW it would be 2–3× tree size.
All five SMT backends (mem, mem-leaves, mem-leaves-x, mem-full, disk) share a single RoundManager<S> implementation. The SmtStore / SmtStoreSnapshot trait pair abstracts over snapshot creation, speculative fork/commit/discard, batch insertion, and proof generation. Switching backends is a one-line config change with zero code duplication.
MemSmt supports five persistence modes, selectable at startup:
None(--smt-backend mem): no writes to RocksDB; fastest per-round commit, but no crash recovery. BFT Core partition state must also be reset on restart.LeavesOnly(--smt-backend mem-leaves): leaf values are appended to thesmt_leavescolumn family on every commit. On restart, all leaves are replayed throughbatch_insertto reconstruct the tree, and the resulting root is compared against the last certified root stored insmt_meta. Recovery time is O(n × log n) in the number of leaves.LeavesWithShutdownSnapshot(--smt-backend mem-leaves-x): identical toLeavesOnlyduring normal operation (low per-round commit overhead). On graceful shutdown (SIGINT / SIGTERM), the entire in-memory tree is persisted tosmt_nodesin a single atomic write. On next startup, ifsmt_nodescontains data, the tree is loaded directly (O(n) I/O) instead of replaying leaves; the node data is then cleaned up so subsequent per-round commits remain leaves-only. Bothmem-leavesandmem-leaves-xautomatically detect and use full-node data if present (e.g. from a previousmem-fullormem-leaves-xrun).Full(--smt-backend mem-full): leaves and all internal nodes are written tosmt_nodeson every commit. On restart, the full tree is loaded directly fromsmt_nodes— O(n) I/O, no recomputation.
Every Certification Request sent to BFT Core can optionally carry a cryptographic consistency proof — a witness that the new SMT root was derived from the previous certified root by appending only the declared leaves, with no deletions or modifications.
Three modes are available via --consistency-proof-mode:
| Mode | Value | Description |
|---|---|---|
| Off | off |
No proof attached (default) |
| RSMT | rsmt |
Hash-based consistency proof, O(batch) size |
| ZK | zk |
SP1 zkVM proof, constant size (~200 KB); requires --features uni-aggregator/zk build |
See README-ZK.md for full build and configuration instructions for ZK mode.
RSMT mode (--consistency-proof-mode rsmt):
What it proves: Let h₀ be the root certified in the last UC and h₁ be the root in the current Certification Request. The proof witnesses the exact set of (key, value) leaves appended to the tree going from h₀ to h₁. A verifier can replay the proof to independently compute both h₀ and h₁ and confirm they match the Input Record hashes in consecutive UCs.
How: The round manager calls batch_insert_with_proof (in crates/rsmt/src/consistency.rs) instead of the plain batch_insert. This performs the same tree mutation in one pass while recording a flat post-order opcode sequence (ProofOp):
| Opcode | CBOR encoding | Meaning |
|---|---|---|
S(h) |
[0, bytes|null] |
Unchanged subtree — hash, or null for empty |
L |
[1] |
New leaf — key/value consumed from sorted batch |
N(d) |
[2, depth] |
Internal node at depth d; two children precede it on the stack |
The stream is post-order (left subtree, right subtree, then node), matching the tree's LSB-first traversal order. Only three opcodes are needed because nodes carry no path data — the verifier reconstructs hashes bottom-up using only the leaf values and the depth at each node.
Verification is a stack machine (in verify_consistency):
- Sort the batch by LSB-first key order (
get_sort_key). - Scan opcodes left to right, maintaining a stack of
(pre_hash, post_hash)pairs:S(h): push(h, h)— unchanged subtree, same hash in both states.L: pop(key, value)from the sorted batch, push(None, hash_leaf(key, value)).N(depth): pop right(rh₀, rh₁), pop left(lh₀, lh₁). Resolve pre-state: if both children existed,h₀ = hash_node(lh₀, rh₀, depth); if only one, propagate that child's hash (new junction). Computeh₁ = hash_node(lh₁, rh₁, depth). Push(h₀, h₁).
- Accept iff the stream is exhausted, the batch is consumed, and the stack contains exactly one element
(old_root, new_root).
The opcode stream is CBOR-encoded and attached to the CR as the zk_proof field. BFT Core validators run verify_consistency before including the round in the certified ledger.
The SMT hashing algorithm is a zero-cost type parameter — no virtual dispatch, no runtime overhead.
Default: all unparameterised public functions use SHA-256 (Sha256Hasher). No caller changes needed for the default path.
To use Blake2s or Blake2b:
use rsmt::{Blake2sHasher, Blake2bHasher, batch_insert_with, batch_insert_with_proof_with, verify_consistency_with};
use rsmt::tree::SparseMerkleTree;
let mut tree = SparseMerkleTree::new();
// Single-leaf insertion with Blake2s
tree.add_leaf_with::<Blake2sHasher>(key, value);
// Single-leaf insertion with Blake2b
tree.add_leaf_with::<Blake2bHasher>(key, value);
// Batch insertion with Blake2s (no proof)
batch_insert_with::<Blake2sHasher>(&mut tree, &batch)?;
// Batch insertion with Blake2b (no proof)
batch_insert_with::<Blake2bHasher>(&mut tree, &batch)?;
// Batch insertion with consistency proof using Blake2s
let (proof, roots) = batch_insert_with_proof_with::<Blake2sHasher>(&mut tree, &batch)?;
// Batch insertion with consistency proof using Blake2b
let (proof, roots) = batch_insert_with_proof_with::<Blake2bHasher>(&mut tree, &batch)?;
// Proof verification with Blake2s
verify_consistency_with::<Blake2sHasher>(&proof, old_root, new_root, &batch)?;
// Proof verification with Blake2b
verify_consistency_with::<Blake2bHasher>(&proof, old_root, new_root, &batch)?;The SmtHasher trait is:
pub trait SmtHasher: Copy + 'static {
fn hash_leaf(key: &SmtKey, value: &[u8]) -> [u8; 32];
fn hash_node(left: &[u8; 32], right: &[u8; 32], depth: u8) -> [u8; 32];
}Custom implementations (e.g. SHA-3/keccak, Poseidon) can be plugged in by implementing SmtHasher.
Important: We're just experimenting. To use a different hash function for aggregation we need to create a tree partitioning scheme and assign tokens to partitions at the mint time and extend partition configs etc.
The in-memory SMT holds the entire tree in a heap-allocated Patricia trie. At ~200 bytes per node this exhausts a 16 GB machine at roughly 80 million leaves.
The disk-backed SMT uses a materialise → insert → persist cycle:
- Before each round:
materializerloads only the nodes on the batch keys' paths from RocksDB (plus sibling hashes asBranch::Stubstubs). All other nodes remain on disk. - During:
batch_insertruns on the partial in-memory tree. Stubs act as opaque leaf hashes — they are never traversed, only their cached hash is used. Arc-based copy-on-write ensures only modified path nodes are cloned. - After:
persisterwalks the modified tree, diffs it against the originally loaded set, and writes mutations to anOverlay(HashMap<NodeKey, Option<bytes>>).
On BFT success: commit_overlay flushes the overlay to a single RocksDB WriteBatch (atomic) and updates the LRU cache.
On BFT rollback: the overlay is dropped — RocksDB is never touched.
Speculative execution for the disk path uses a layered overlay:
proposed_snap ──fork()──> spec_snap
own_overlay own_overlay (new mutations)
parent_overlay → proposed_snap's overlay (Arc clone)
The spec snapshot reads own_overlay → parent_overlay → LRU cache → RocksDB, seeing the proposed round's uncommitted mutations without any DB writes.
Memory usage is bounded by batch size + cache size, not total tree size. A 10 000-leaf batch against a 500 million-leaf tree materialises only ~10 000 × tree_depth ≈ 300 000 nodes.
Startup: reads only the committed root hash from smt_meta — no leaf replay, no tree reconstruction.
rugregator/
├── Cargo.toml # Workspace root
├── scripts/... # Setup and testing scripts
└── crates/
├── rsmt/ # Standalone SMT library
│ └── src/
│ ├── tree.rs # Core insertion, hash caching
│ ├── path.rs # Path helpers
│ ├── hash.rs # SmtHasher trait, Sha256Hasher, Blake2sHasher, Blake2bHasher
│ ├── snapshot.rs # O(1) copy-on-write snapshots (fork/commit/discard)
│ ├── consistency.rs # batch_insert_with_proof, ProofOp, CBOR encoding
│ ├── proof.rs # get_path, MerkleTreePath, CBOR wire format
│ ├── node_serde.rs # Compact binary node serialisation
│ └── types.rs # Branch, LeafBranch, NodeBranch, Stub
├── smt-store/ # SMT storage backends
│ └── src/
│ ├── traits.rs # SmtStore + SmtStoreSnapshot traits
│ ├── mem.rs # MemSmt — fully in-memory (PersistMode: None/LeavesOnly/LeavesWithShutdownSnapshot/Full)
│ └── disk/
│ ├── store.rs # DiskSmt — main disk-backed entry point
│ ├── materializer.rs # Partial tree loading from RocksDB
│ ├── persister.rs # Post-mutation write-back
│ ├── overlay.rs # Speculative write buffer
│ ├── cache.rs # LRU node cache
│ ├── snapshot.rs # DiskSmtSnapshot (layered overlays)
│ ├── node_key.rs # Absolute bit-path DB keys
│ └── tests.rs # Equivalence, rollback, restart, proof tests
└── aggregator/
└── src/
├── main.rs # Entry point, CLI, SMT backend wiring
├── config.rs # Config (--smt-backend and all other flags)
├── storage.rs # AggregatorState, on-demand proof generation
├── storage_rocksdb.rs# RocksDB Store impl (records, blocks, meta CFs)
├── api/ # HTTP server, JSON-RPC handlers, CBOR types
├── round/
│ ├── manager.rs # RoundManager<S: SmtStore>, speculative execution
│ ├── live_committer.rs # libp2p BFT Core connectivity
│ └── state.rs # ProcessedRecord
├── validation/ # Predicate, StateID, signature checks
└── bin/
└── perf_test.rs # SMT benchmark across all backends