Skip to content

feat: add ethereum support#912

Merged
scarmuega merged 3 commits intotxpipe:v3from
mduthey:feat/support-ethereum
Oct 16, 2025
Merged

feat: add ethereum support#912
scarmuega merged 3 commits intotxpipe:v3from
mduthey:feat/support-ethereum

Conversation

@mduthey
Copy link
Copy Markdown

@mduthey mduthey commented Oct 15, 2025

Summary by CodeRabbit

  • New Features

    • Added Bitcoin and Ethereum sources and watch commands (multi-chain support).
    • Enhanced terminal output to display Cardano, Bitcoin, and Ethereum blocks/txs.
    • Added Prometheus-style pipeline metrics and configurable retry, intersect, finalize, and stateful cursor options.
  • Documentation

    • Introduced comprehensive v3 docs: Installation (binary, Docker, Kubernetes, from source), Usage (daemon, dump, watch), Advanced (metrics, retry, intersect/finalize, custom network, cursor), Sources (Cardano, Bitcoin, Ethereum, S3, Hydra, Mithril, U5C), Sinks (AWS, GCP, Kafka, Redis, Webhook, File Rotate, Elastic, RabbitMQ, Stdout, Terminal), Filters (Parse CBOR, Split Block, Select, Legacy V1, Wasm), Reference, and Guides.
    • Updated README: Bitcoin listed as a Source; expanded watch examples.
  • Chores

    • Added example daemon configs for Bitcoin and Ethereum.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Oct 15, 2025

Caution

Review failed

Failed to post review comments

Walkthrough

Introduces multi-chain support (Cardano, Ethereum, Bitcoin) via new framework modules and a new Chain/Record model. Adds Ethereum and Bitcoin sources and watch subcommands behind feature flags. Refactors filters/sinks to use Cardano-namespaced records. Updates daemon config types. Adds extensive v3 documentation, examples, and metadata.

Changes

Cohort / File(s) Summary
Build & Features
Cargo.toml
Adds eth/btc features; optional deps: bitcoind-async-client, corepc-types, alloy; wires alloy with provider-ws.
Top-level Docs & README
README.md
Updates feature status, watch instructions per chain, and branch notes.
Docs: v3 Index & Usage
docs/v3/index.mdx, docs/v3/usage/*, docs/v3/installation/*, docs/v3/.../_meta.yaml
Adds v3 overview, daemon/dump/watch usage, installation guides (binary, docker, source, k8s), and navigation metadata.
Docs: Sources
docs/v3/sources/*
Adds pages for Cardano N2C/N2N/S3/U5C, plus new sources: btc (bitcoin-rpc), eth (ethereum-rpc), hydra, mithril.
Docs: Sinks
docs/v3/sinks/*
Adds sink docs: stdout, terminal, webhook, kafka, redis, rabbitmq, elasticsearch, aws_lambda, aws_s3, aws_sqs, gcp_cloudfunction, gcp_pubsub, file_rotate.
Docs: Advanced
docs/v3/advanced/*
Adds advanced features docs: custom network, intersect/finalize options, pipeline metrics, retry policy, stateful cursor.
Docs: Filters
docs/v3/filters/*
Adds filter docs: legacy_v1, parse_cbor, split_block, select, rollback_buffer, wasm.
Docs: Reference
docs/v3/reference/*
Adds data dictionary and section metadata.
Examples
examples/btc/daemon.toml, examples/eth/daemon.toml
Adds example daemon configs for Bitcoin and Ethereum with file rotation sink.
Watch CLI Refactor
src/bin/oura/watch.rs, src/watch/mod.rs, src/watch/cardano.rs, src/watch/btc.rs, src/watch/eth.rs
Replaces ad-hoc args with subcommands per chain; adds chain-specific runners and args; feature-gated btc/eth.
Dump CLI Update
src/bin/oura/dump.rs
Introduces DumpChain enum; maps to cardano::ChainConfig; updates magic arg type.
Daemon Config Type
src/daemon/mod.rs
Changes ConfigRoot.chain to Option (from Option).
Framework Core Refactor
src/framework/mod.rs, src/framework/cardano/mod.rs, src/framework/bitcoin/mod.rs, src/framework/ethereum/mod.rs, src/framework/substrate/mod.rs
Introduces new Chain enum, Context using Chain, per-chain Record enums; moves legacy_v1 under cardano; adds bitcoin/ethereum/substrate records and JSON conversions.
Sources: Cardano updates
src/sources/{n2c.rs,n2n.rs,s3.rs,u5c.rs,hydra.rs,mithril.rs}
Wraps payloads as Record::Cardano(...); extracts Cardano chain config where needed.
Sources: New ETH/BTC
src/sources/eth.rs, src/sources/btc.rs, src/sources/mod.rs
Adds Ethereum WS and Bitcoin RPC sources; extends enum Config/Bootstrapper and spawns under feature flags.
Filters: Namespace & Variants
src/filters/{parse_cbor.rs,split_block.rs,into_json.rs}
Switches to Record::Cardano(cardano::Record::...) variants; into_json maps only Cardano to GenericJson.
Filters: Legacy V1
src/filters/legacy_v1/{mod.rs,prelude.rs,map.rs,crawl.rs,cip15.rs,cip25.rs}
Moves imports under framework::cardano::legacy_v1; adjusts event construction to Cardano-wrapped records; uses Cardano chain config.
Filters: Select & Wasm
src/filters/select/eval/mod.rs, src/filters/wasm_plugin.rs
Evaluates only Cardano ParsedTx/ParsedBlock; wasm plugin introduces CardanoRecord alias and routing.
Sinks: Adjustments
src/sinks/{aws_s3.rs,elasticsearch.rs,terminal/format.rs,assert/*}
Switches to Cardano-wrapped records; elasticsearch assumes Cardano chain; terminal formatter expanded for per-chain variants; assert sink updated to Cardano legacy_v1 events.
Library Exposure
src/lib.rs
Publishes watch module.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant CLI as CLI (watch)
  participant Runner as Watch Runner
  participant Daemon as Daemon
  participant Source as Source (per chain)
  participant Filters as Filters
  participant Sink as Sink

  CLI->>Runner: Parse subcommand (cardano|bitcoin|ethereum)
  Runner->>Daemon: Build ConfigRoot (Chain, Source, Intersect, Filters, Sink)
  Daemon->>Source: spawn()
  loop stream blocks
    Source-->>Daemon: ChainEvent::Apply(Point, Record::{Cardano|Bitcoin|Ethereum})
    Daemon->>Filters: process(record)
    Filters-->>Daemon: record'
    Daemon->>Sink: deliver(record')
    Sink-->>Daemon: ack
  end
  Daemon-->>Runner: shutdown
Loading
sequenceDiagram
  autonumber
  participant ETH as Ethereum Source
  participant WS as WS Provider
  participant Node as ETH Node
  ETH->>WS: connect(url)
  WS-->>ETH: connected
  ETH->>Node: subscribe newHeads
  loop headers
    Node-->>ETH: header(hash,height)
    ETH->>Node: eth_getBlockByHash
    Node-->>ETH: block
    ETH-->>Daemon: Apply(Point(height,hash), Record::Ethereum(ParsedBlock))
  end
Loading
sequenceDiagram
  autonumber
  participant BTC as Bitcoin Source
  participant RPC as bitcoind RPC
  loop interval
    BTC->>RPC: getblockchaininfo
    alt new best_block_hash
      BTC->>RPC: getblock by hash
      RPC-->>BTC: block(json)
      BTC-->>Daemon: Apply(Point(height,hash), Record::Bitcoin(ParsedBlock))
    else unchanged
      BTC-->>BTC: Idle
    end
  end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~75 minutes

Possibly related PRs

Suggested reviewers

  • paulobressan

Poem

A whisk of wires, a hop through chains,
I twitch my nose at blocks and panes.
Cardano burrows, Ethereum gleams,
Bitcoin hums in polling streams.
With sinks and filters, logs rotate—
I thump: “All set!” and celebrate.
(_/)\n> (•_•)\n> />🍃 multi-chain estate

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 12.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The title is concise and clearly conveys the primary change—adding Ethereum support—which aligns with the major additions of eth features, documentation, and code in this PR.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@mduthey mduthey changed the base branch from main to v3 October 15, 2025 15:54
@scarmuega scarmuega merged commit 6b62e13 into txpipe:v3 Oct 16, 2025
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants