No multiplexing on client connections.

[arpit006]

Wanted to highlight a problem with the current implementation involving channels.
The way connections are used:-

1. Connections are created and pushed to the channel
2. Whenever a request is to be served, a connection is obtained from the channel, request is served and then connection is pushed back to the channel.

The problem with this approach is that if there are n connections in channel, at max we can only serve n requests concurrently because of the nature of the implementation which defies the core property on which gRPC was built ==> multiplexing.

I encountered a similar problem in my org where we had to cater a throughput of 300K RPS, which seemed unattainable with current implementation along with some other necessary handlings.

We came up with a custom solution of our own which abstracts all of the problems we faced and with this solution we were able to serve 300K RPS with 0 overhead because of custom code.

I have documented my entire journey of the problems we faced and the solution we coded to cater the same. I am posting it here with a hope that it helps anyone stuck with similar problem.

Medium Article