MailHog works ineffectively with maildir and large volume of messages #298
Description
The problem with MailHog on high volumes of messages data (e.g. 16GB) configured with maildir, is that when someone tries to check the e-mail in the UI, which results into the API call on a client’s behalf:
https://example.com/api/v2/messages?limit=1
(UI does limit=50, but even with limit=1 the issue is imminent)
MailHog seems to try to read all the e-mail messages available to it. The corresponding log message is:
Listing messages in /home/mailhog/mount/maildir
After which it echoes every message it reads from maildir folder into logs. At the same time the client request hangs on waiting, and MailHog process continuously grows in memory consumption. Eventually, either client request is terminated by ELB/proxy (when running behind it) or the process gets into OOM and gets restarted/killed - both of which prevent the client from receiving the response from API.
Giving more memory wouldn’t solve the issue, as it seems to be requiring the amount of memory to be equal to amount of e-mail messages data on disk in order to be able to read everything.
Potentially, configuring MailHog against MongoDB might result into different (more effective) behavior, but that produces additional dependency on MongoDB and maintenance overhead associated with its support.