Problem
The fairness accuracy benchmark (bench_fairness_accuracy) crashes after ~2000 enqueues with "connection closed by server". The server appears to panic or crash under sustained high-throughput enqueue load via the binary protocol SDK.
Observed in
Symptoms
fairness enqueue: 2000/10000 (101 ops/s)
thread 'main' panicked at crates/fila-bench/src/benchmarks/fairness.rs:97:18:
enqueue: Status(Unavailable("connection lost: connection closed by server"))
Likely cause
The server crashes or panics under sustained binary protocol load. The gRPC SDK handled backpressure differently (HTTP/2 flow control). The binary protocol SDK's try_send for delivery frames may be dropping too many messages, or there's a panic in the Lua script execution path under heavy load.
Related fixes in this epic
- Delivery channel deadlock fixed: background reader now uses
try_send instead of send.await to prevent blocking when channel is full
- Double-consume fixed: server drains existing consumers before registering new ones
- These fixes resolved the latency benchmark hang but the fairness crash persists
Problem
The fairness accuracy benchmark (
bench_fairness_accuracy) crashes after ~2000 enqueues with "connection closed by server". The server appears to panic or crash under sustained high-throughput enqueue load via the binary protocol SDK.Observed in
Symptoms
Likely cause
The server crashes or panics under sustained binary protocol load. The gRPC SDK handled backpressure differently (HTTP/2 flow control). The binary protocol SDK's
try_sendfor delivery frames may be dropping too many messages, or there's a panic in the Lua script execution path under heavy load.Related fixes in this epic
try_sendinstead ofsend.awaitto prevent blocking when channel is full