A minimal Rollup Derivation Pipeline built using Reth's ExEx.
- Motivation
- Crates
- Implementation Status
- What is an ExEx?
- What are Ethereum Blobs?
- KZG Commitments and Versioned Hashes
- The L1 and L2 Relationship
- What is a Sequencing Epoch?
- What is a Batch?
- What are Optimism Channels?
- What are Frames?
- Derivation Pipeline Flow
- Sources
Inspired by this great Paradigm article, I've decided to build this minimal Rollup Derivation Pipeline specifically for Unichain, even though it can be easily abstracted to be usable across other op-stack L2's. This is merely for fun and should not be used in prd!
The project is split into a few crates:
-
derivexex - The ExEx binary. Runs alongside Reth, listens for new L1 blocks, fetches blobs from the beacon node, and pipes everything through the pipeline. Also handles persistence (SQLite) and reorg recovery.
-
derivexex-pipeline - Where the actual derivation happens. Blob decoding, frame parsing, channel assembly, batch decoding (both single and span batches), L2 block building. All sync code, no networking - just give it data and it spits out L2 blocks.
-
derivexex-stream - Standalone async version that doesn't need Reth. Subscribes to L1 via WebSocket, fetches blobs, tracks safe/finalized heads, detects reorgs. Useful if you just want to stream L2 blocks posted to Mainnet without running a full node.
-
derivexex-types - Shared types for serialization (channel state, checkpoints). Kept separate so the other crates don't have circular deps.
This is a project built for learning Kona and Reth internals, not meant for production use. Here's what I've built so far:
Blobs are fetched from the Beacon API (consensus layer), since only Blob Versioned Hashes (hash derived from the KZG Commitment) are stored in the execution layer.
Implements the OP v0 blob encoding.
Channels are formed of frames that are grouped by their 16-byte channel ID and reassembled in order. A channel is complete when is_last flag is set and all prior frames (0 to N) have arrived.
Handles both batch types from the OP Stack spec, logic derived from Kona's repo:
- Single Batch
- Span Batch: Introduced later, more efficient since it has more data packed.
Parses TransactionDeposited events from the OptimismPortal contract. Deposits are included in the first L2 block of each epoch.
Builds the L1 attributes deposit transaction (the system tx that goes first in every L2 block). Supports both Ecotone and Isthmus hardfork formats.
Assembles complete L2 blocks: L1 info deposit first, then user deposits (if epoch start), then sequencer transactions from batches.
Decodes all transaction types (legacy, EIP-2930, EIP-1559, EIP-7702, deposits) using op-alloy-consensus. Recovers signer addresses from signatures.
Detects L1 reorgs and prunes invalidated frames, channels, and epoch data. The stream crate emits reorg events so consumers can react.
Tracks safe and finalized L1 heads (stream crate). Useful for knowing when derived L2 blocks are considered safe vs finalized.
SQLite for checkpoints and recovery. Saves progress so you can resume after restart without re-deriving everything.
Epoch validation (verifying timestamps match the spec), metrics/observability, and actual state execution (we build L2 blocks but don't run them through the EVM).
An ExEx is basically a Future that runs alongside Reth, where it's futures are polled.
Blobs (Binary Large Object) were introduced on Ethereum in Dencun fork (2024). They are a temporary (they are pruned from consensus after ~18 days, more below), cheaper way for Layer 2's to post data to the L1 and have a standard size of 128kb. Blobs contents are called frames (it's definition is just below).
Each blob (128kb) has a KZG commitment, a 48byte proof of its contents. The L1 execution layer doesn't store full blobs, only their versioned hashes. A versioned hash is derived from the KZG commitment and is what gets stored in EIP-4844 transactions. To fetch a blob from the beacon node (sidecar), you use the versioned hash as a lookup key.
Each L2 block is tied to an L1 block called its "L1 origin". A L1 block can be the origin for other multiple L2 blocks, it's also called Sequencing Epoch on Optimism spec.
A batch is the data needed to build one L2 block. It contains an epoch number, an L2 timestamp, and a list of transactions. Batches are compressed together into channels for compression efficiency.
A channel is a sequence of batches compressed together. Compressing multiple batches as a group yields better compression ratios than compressing each individually. A channel is identified by a unique 16-byte ID and info about a certain L2 block can be span across more than one L1 block.
A frame is a chunk of a channel that fits into a blob. Since blobs are limited to 128KB and channels can be larger, channels are split into ordered frames. Each frame contains a channel ID, a frame number, payload data, and a flag indicating if it's the last frame. Once all frames arrive, the channel is reassembled and decompressed.
L1 Blobs → Frames → Channel → Decompress → RLP Decode → Raw Bytes Batch ->
TODO: Write more about this flow