feat: bytecode commitment

[quangvdao]

Overview

This PR introduces Committed Bytecode Mode, a new verification mode where the verifier only needs bytecode commitments instead of full bytecode data. This enables succinct verification independent of program size by using claim reductions and folded openings.

---

Key Difficulties & Tradeoffs

1. Trace Length Must Be ≥ Bytecode Size

Constraint: In Committed mode, padded_trace_length >= bytecode_size is required.

Why: Stage 8 folds bytecode chunk openings into the joint opening proof by embedding bytecode polynomials (defined over log_K cycle variables) into the main opening domain (defined over log_T cycle variables). This embedding uses a Lagrange selector ∏(1 - r_extra) over the "missing" cycle variables, which only works when log_T >= log_K.

Alternative not pursued: Supporting arbitrary log_K vs log_T would require:

• Changing batched sumcheck semantics from "repetition padding" to "selector padding" (equivalently, front-loaded versus back-loaded)
• Modifying Stage 6b scheduling to use max(log_T, log_K) rounds
• Embedding all trace-derived polynomials with selector padding (not just bytecode)
• Adding a "zero row" padding mode in witness generation

This is a cross-cutting change affecting sumcheck, Stage 6b scheduling, Dory context sizing, Stage 8 claims, and streaming VMV paths. Given scope and risk, we enforce T >= K instead.

2. Bytecode Commitment Chunking (448 Lanes → k_chunk Chunks)

Structure: Bytecode fields are organized into 448 "lanes":

• 384 lanes for register one-hots (3 × 128 registers: rs1, rs2, rd)
• 64 lanes for dense fields (pc, imm, 13 circuit flags, 7 instruction flags, 41 lookup table selectors, 1 RAF flag)

Chunking depends on log_k_chunk:

• k_chunk=16 (small traces): 448/16 = 28 commitments
• k_chunk=256 (large traces): 448/256 = 2 commitments

Tradeoff: More chunks means more commitment overhead but matches the existing one-hot RA polynomial structure. The log_k_chunk value is derived from trace length (4 if log_T < 16, else 8), so small traces incur more bytecode commitments.

3. AddressMajor Layout Requires Coefficient Permutation

Problem: Under DoryLayout::AddressMajor, bytecode chunk coefficients are stored in "cycle-major" index order (cycle * K + address), which makes BindingOrder::LowToHigh bind lane/address bits first. But BytecodeClaimReduction Phase 1 (Stage 6b) must bind cycle bits first to match the staged r_bc semantics from Stage 6a.

Solution: In BytecodeClaimReductionProver::initialize, we permute AddressMajor chunk coefficients into CycleMajor order before running the reduction sumcheck. This is a pure index permutation (variable renaming); the resulting evaluations match the committed polynomial when the opening point is interpreted in the unified [lane || cycle] order.

This mirrors the approach used in AdviceClaimReduction where advice polynomials are also permuted for correct variable binding order.

4. Stage 6 Split Required for Mid-Stage Claim Emission

Problem: The verifier's O(K) work happens when computing Val_s(r_bc) evaluations. To eliminate this, we need to emit these claims right after binding the bytecode address variables r_bc, before the cycle-phase rounds begin.

Solution: Split Stage 6 into:

• Stage 6a: Address-phase sumchecks (bind r_bc for bytecode, r_address for booleanity)
• Stage 6b: Cycle-phase sumchecks + claim reductions

This required splitting both BytecodeReadRaf and Booleanity into separate address/cycle sumcheck instances, with intermediate claims cached in the opening accumulator to chain the phases.

---

Key Changes

1. New Bytecode Mode Enum

File: jolt-core/src/zkvm/config.rs

• Added BytecodeMode enum with two variants:
  • Full (default): Verifier materializes bytecode-dependent polynomials (O(K) work)
  • Committed: Uses staged Val claims + claim reduction + folded Stage 8 opening
• Added OneHotConfig::from_log_k_chunk() constructor for explicit chunk size specification

2. Bytecode Chunk Infrastructure

New file: jolt-core/src/zkvm/bytecode/chunks.rs

• total_lanes(): Returns 448 total lanes for bytecode fields (3×32 register one-hots + 2 fields + flags + lookup table flags + RAF flag)
• lane_value(): Maps a global lane index to the corresponding bytecode field value
• build_bytecode_chunks(): Builds per-chunk multilinear polynomials B_i(lane, k) for bytecode commitment

3. Bytecode Preprocessing Refactor

File: jolt-core/src/zkvm/bytecode/mod.rs

• Added TrustedBytecodeCommitments<PCS> struct for type-safe trusted bytecode commitments
  • derive() method computes commitments from full bytecode (offline preprocessing)
  • Stores commitments, log_k_chunk, and bytecode_len
• Added VerifierBytecode<PCS> enum:
  • Full(Arc<BytecodePreprocessing>): Verifier has full bytecode
  • Committed(TrustedBytecodeCommitments<PCS>): Verifier only has commitments
• Moved code_size from BytecodePreprocessing to JoltSharedPreprocessing.bytecode_size

4. Bytecode Claim Reduction

New file: jolt-core/src/zkvm/claim_reductions/bytecode.rs

Two-phase claim reduction that batches 5 bytecode Val-stage claims:

• Stage 6b (Cycle Phase): Binds cycle variables first
• Stage 7 (Lane Phase): Binds lane variables, caches final bytecode chunk openings

Key components:

• BytecodeReductionPhase enum: CycleVariables / LaneVariables
• BytecodeClaimReductionParams: Stores η powers, lane weights, challenges
• BytecodeClaimReductionProver/Verifier: Runs the two-phase sumcheck
• compute_chunk_lane_weights(): Computes per-chunk lane weight tables from Val stage gammas

5. Stage 6 Split: 6a (Address) + 6b (Cycle)

Files: jolt-core/src/zkvm/prover.rs and verifier.rs

Splits Stage 6 into two phases to support bytecode claim reduction:

• Stage 6a: Address-phase sumchecks (BytecodeReadRaf address, Booleanity address)
• Stage 6b: Cycle-phase sumchecks + bytecode/advice claim reductions

New provers/verifiers:

• BytecodeReadRafAddressSumcheckProver/Verifier (address phase)
• BytecodeReadRafCycleSumcheckProver/Verifier (cycle phase)
• BooleanityAddressSumcheckProver/Verifier (address phase)
• BooleanityCycleSumcheckProver/Verifier (cycle phase)

6. Prover/Verifier Preprocessing Updates

JoltProverPreprocessing changes:

• Now stores bytecode: Arc<BytecodePreprocessing> (prover always has full access)
• New optional field: bytecode_commitments: Option<TrustedBytecodeCommitments<PCS>>
• New optional field: bytecode_commitment_hints: Option<Vec<PCS::OpeningProofHint>>
• New constructors: new() (Full mode) and new_committed() (Committed mode)
• is_committed_mode() helper method

JoltSharedPreprocessing changes:

• Removed bytecode: Arc<BytecodePreprocessing>
• Added bytecode_size: usize as the single source of truth for bytecode size

JoltVerifierPreprocessing changes:

• Added bytecode: VerifierBytecode<PCS> field
• New constructors: new_full() and new_committed()

7. Proof Serialization Updates

File: jolt-core/src/zkvm/proof_serialization.rs

• Split stage6_sumcheck_proof → stage6a_sumcheck_proof + stage6b_sumcheck_proof
• Added bytecode_mode: BytecodeMode to JoltProof
• Added CommittedPolynomial::BytecodeChunk(usize) variant
• Added new VirtualPolynomial variants for claim reduction

8. RLC Polynomial / Stage 8 Integration

File: jolt-core/src/poly/rlc_polynomial.rs

• Added bytecode_polys to StreamingRLCContext
• Added vmp_bytecode_contribution() for bytecode chunk VMP computation
• Bytecode chunks are embedded in Stage 8 joint opening by fixing extra cycle variables to 0

9. Booleanity Sumcheck Split

File: jolt-core/src/subprotocols/booleanity.rs

Split the booleanity sumcheck into address and cycle phases:

• BooleanityAddressSumcheckProver: Handles first log_k_chunk rounds
• BooleanityCycleSumcheckProver: Handles remaining log_t rounds (reconstructs state from accumulator)

10. SDK/Macro Updates

File: jolt-sdk/macros/src/lib.rs

New generated functions:

• preprocess_<fn>() - Full mode preprocessing
• preprocess_committed_<fn>() - Committed mode preprocessing
• preprocess_shared_<fn>() - Returns (JoltSharedPreprocessing, BytecodePreprocessing) tuple
• build_prover_committed_<fn>() - Build prover closure for committed mode
• prove_committed_<fn>() - Prove function for committed mode

11. Test Infrastructure

New file: jolt-core/src/zkvm/tests.rs

Moved e2e tests from prover.rs to dedicated module with new test infrastructure:

• E2ETestConfig builder for test configuration
• run_e2e_test() unified test runner
• Supports all axes: program, bytecode mode, Dory layout, trace size, advice

12. Example Updates

All examples updated to use new preprocessing API:

• Replaced preprocess_shared_* + preprocess_prover_* + preprocess_verifier_* with preprocess_* + verifier_preprocessing_from_prover_*
• Fibonacci example adds --committed-bytecode CLI flag

13. Error Handling

File: jolt-core/src/utils/errors.rs

New error variants:

• InvalidBytecodeConfig(String) - Bytecode configuration mismatch
• BytecodeTypeMismatch(String) - Wrong VerifierBytecode variant access

---

Usage

// Full mode (default)
let prover_preprocessing = JoltProverPreprocessing::new(shared, bytecode);

// Committed mode  
let prover_preprocessing = JoltProverPreprocessing::new_committed(shared, bytecode);

// Prove with committed bytecode
let prover = RV64IMACProver::gen_from_elf_with_bytecode_mode(
    &preprocessing,
    elf_contents,
    inputs,
    untrusted_advice,
    trusted_advice,
    commitment,
    hint,
    BytecodeMode::Committed,
);

CLI example:

cargo run --release --example fibonacci -- --committed-bytecode

[quangvdao]

Close in lieu of #1221