docs: update Performance section with two-host benchmark numbers#49
Merged
levleontiev merged 3 commits intomainfrom Mar 18, 2026
Merged
docs: update Performance section with two-host benchmark numbers#49levleontiev merged 3 commits intomainfrom
levleontiev merged 3 commits intomainfrom
Conversation
#48) - Hardware: 2 × c7i.xlarge, cluster placement group, eu-central-1 - New latency table: p50 304µs / p90 543µs decision service - Enforcement overhead framing: +69µs p50 / +134µs p90 over raw nginx - New throughput: 195k RPS (simple and complex) - Removed tiktoken/token estimation row (not supported) - Updated inline tagline to overhead framing
#48) - Hardware: 2 × c7i.xlarge, cluster placement group, eu-central-1 - New latency table: p50 304µs decision service - Enforcement overhead framing: +69µs p50 / +134µs p90 over raw nginx - New throughput: 195k RPS (simple and complex) - Removed tiktoken row, updated tagline to overhead framing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #48
Summary
< 70 µs enforcement overhead · 195k RPSTest plan