proxy: disable buffering for gateway recording#23
proxy: disable buffering for gateway recording#23recursivetree wants to merge 1 commit intoRoachbones:mainfrom
Conversation
|
Have you noticed any performance impact from this? I remember avoiding constant flushing because I was concerned about ruining websocket performance on computers with hard drives. |
|
I haven't noticed an performance impact, but I haven't tested it on harddrives either since I only have SSDs. Things to consider:
|
576350a to
e29be31
Compare
|
I've decided to split the non-controversial parts of this PR into #26 so we can discuss this in detail. If anyone has an HDD and is able to test, I'd appreciate it if you could do so. |
|
I have a HDD, but which parts of it need testing? |
I'm not an expert for which statistics to look at for measuring storage performance. But once we know that, I'd suggest running a test with and without this change? |
I've noticed the same issue as in #16, but with gateway traffic. This PR fixes this by flushing after every message.
Since the websocket data is binary and not text, we cannot use pythons line buffering mode. Instead, we flush manually.
Since not flushing has caused some issues with missing entries in my gateway_index file, I've also added logic to detect the highest unused flow id even if some data got lost from the gateway_index.