Skip to content

Virtual Raindex event ingestion#2175

Open
hardyjosh wants to merge 3 commits into2025-09-26-virtual-raindexfrom
2025-10-01-vr-event-stream
Open

Virtual Raindex event ingestion#2175
hardyjosh wants to merge 3 commits into2025-09-26-virtual-raindexfrom
2025-10-01-vr-event-stream

Conversation

@hardyjosh
Copy link
Contributor

Caution

Chained to #2167, merge first and update target for this to main

Motivation

We need the virtual raindex to replay on-chain activity without live RPC calls so we can fork mainnet orderbooks into an offline, deterministic sandbox. Before this work there was no log ingestion at all—virtual state could only be mutated directly—so parity checks, backfills, and event-driven simulations were impossible.

Solution

Introduce a dedicated events module that decodes OrderBook and Store logs into RaindexMutation batches (covering add/remove, deposit/withdraw, take orders, and clear/bounty flows), add complementary unit tests, and extend the integration harness to execute real takes/clears on an Anvil instance, replay the emitted logs, and assert the reconstructed snapshot matches the live virtual state.

The README now documents the ingestion pipeline so downstream callers can bootstrap virtual forks directly from recorded logs.

Checks

By submitting this for review, I'm confirming I've done the following:

  • made this PR as small as possible
  • unit-tested any new functionality
  • linked any relevant issues or PRs
  • included screenshots (if this involves a front-end change)

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Oct 2, 2025

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch 2025-10-01-vr-event-stream

Comment @coderabbitai help to get the list of available commands and usage tips.

@hardyjosh hardyjosh self-assigned this Oct 2, 2025
fn u256_to_usize(value: &U256) -> Result<usize> {
value
.try_into()
.map_err(|_| RaindexError::Unimplemented("index too large for usize"))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unimplemented doesn't seem like the best error variant to use here

Comment on lines +237 to +246
/// Converts an iterator of [`OrderBookEvent`]s into a flat list of mutations.
pub fn orderbook_events_to_mutations<'a>(
events: impl IntoIterator<Item = OrderBookEvent<'a>>,
) -> Result<Vec<RaindexMutation>> {
let mut mutations = Vec::new();
for event in events {
mutations.extend(orderbook_event_to_mutations(event)?);
}
Ok(mutations)
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: consider (.flatten_ok requires itertools)

Suggested change
/// Converts an iterator of [`OrderBookEvent`]s into a flat list of mutations.
pub fn orderbook_events_to_mutations<'a>(
events: impl IntoIterator<Item = OrderBookEvent<'a>>,
) -> Result<Vec<RaindexMutation>> {
let mut mutations = Vec::new();
for event in events {
mutations.extend(orderbook_event_to_mutations(event)?);
}
Ok(mutations)
}
/// Converts an iterator of [`OrderBookEvent`]s into a flat list of mutations.
pub fn orderbook_events_to_mutations<'a>(
events: impl IntoIterator<Item = OrderBookEvent<'a>>,
) -> Result<Vec<RaindexMutation>> {
events
.into_iter()
.map(orderbook_event_to_mutations)
.flatten_ok()
.collect()
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants