Skip to content

ci(bench): txgen e2e bench#3669

Open
Rjected wants to merge 16 commits intomainfrom
dan/replace-tempo-bench-with-txgen
Open

ci(bench): txgen e2e bench#3669
Rjected wants to merge 16 commits intomainfrom
dan/replace-tempo-bench-with-txgen

Conversation

@Rjected
Copy link
Copy Markdown
Member

@Rjected Rjected commented Apr 20, 2026

Creates a new workflow and .nu for a bench workflow that uses txgen.

In the future, we will deprecate bench-e2e.yml and make bench-txgen-dispatch.yaml the default. This will also allow us to remove upload-clickhouse.sh and tempo.nu generate-summary backward compatibility.

@Rjected Rjected force-pushed the dan/replace-tempo-bench-with-txgen branch 2 times, most recently from d0fe75a to 2e545f4 Compare April 20, 2026 19:13
@Rjected
Copy link
Copy Markdown
Member Author

Rjected commented Apr 20, 2026

derek bench backend=txgen preset=tip20 duration=30 tps=1000 bloat=1

1 similar comment
@Rjected
Copy link
Copy Markdown
Member Author

Rjected commented Apr 20, 2026

derek bench backend=txgen preset=tip20 duration=30 tps=1000 bloat=1

@Rjected Rjected force-pushed the dan/replace-tempo-bench-with-txgen branch from 2e545f4 to b3fd9f8 Compare April 20, 2026 22:00
@Rjected
Copy link
Copy Markdown
Member Author

Rjected commented Apr 20, 2026

derek bench backend=txgen preset=tip20 duration=30 tps=1000 bloat=1

1 similar comment
@Rjected
Copy link
Copy Markdown
Member Author

Rjected commented Apr 20, 2026

derek bench backend=txgen preset=tip20 duration=30 tps=1000 bloat=1

@decofe
Copy link
Copy Markdown
Member

decofe commented Apr 20, 2026

cc @Rjected

❌ Benchmark failed. View logs

@Rjected Rjected force-pushed the dan/replace-tempo-bench-with-txgen branch from e29f7f3 to 6ffb79c Compare April 21, 2026 20:14
@github-actions
Copy link
Copy Markdown
Contributor

This PR has been marked stale due to 7 days of inactivity.

@github-actions github-actions Bot added the stale label Apr 30, 2026
@shekhirin shekhirin changed the title wip: add new workflows for benching with txgen ci(bench): txgen e2e bench May 7, 2026
@shekhirin shekhirin added the A-ci Related to github workflows or other build and lint tools label May 7, 2026
@shekhirin shekhirin marked this pull request as ready for review May 7, 2026 09:35
@shekhirin shekhirin requested a review from Zygimantass as a code owner May 7, 2026 09:35
@decofe
Copy link
Copy Markdown
Member

decofe commented May 7, 2026

cc @shekhirin

Txgen benchmark complete. View job

Bench Comparison: origin/main vs local

Configuration

  • Bloat: 1 MiB
  • Preset: tip20
  • Target TPS: 1000
  • Duration: 5s
  • Snapshot: schelk
  • Baseline blocks: 13
  • Feature blocks: 12

Tempo Metrics

Metric Baseline Feature Delta
Avg TPS 917.0 1000.0 9.1%
TPS P50 999.0 1000.0 0.1%
TPS P90 1001.0 1001.0 0.0%
TPS P99 1001.0 1001.0 0.0%
Gas Throughput [Mgas/s] 44.7 48.7 8.9%
Block Time P50 [ms] 1000.0 1000.0 0.0%
Block Time P90 [ms] 1001.0 1001.0 0.0%
Block Time P99 [ms] 1001.0 1001.0 0.0%

Latency (Secondary)

Metric Baseline Feature Delta
Latency Mean [ms] 1000.1 1000.1 0.0%
Latency Std Dev [ms] 0.7 0.7 0.0%
Latency P50 [ms] 1000.0 1000.0 0.0%
Latency P90 [ms] 1001.0 1001.0 0.0%
Latency P99 [ms] 1001.0 1001.0 0.0%

Per-Run Details

Run Blocks Total Tx Success Failed Avg TPS Block P50 Mgas/s
baseline-1 7 5000 5000 0 833.0 1000.0 40.6
feature-1 6 5000 5000 0 1000.0 1000.0 48.7
feature-2 6 5000 5000 0 1000.0 1000.0 48.7
baseline-2 6 5000 5000 0 1000.0 1000.0 48.7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-ci Related to github workflows or other build and lint tools M-prevent-stale

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants