Zilean scraper is pretty wasteful when it comes to syncing the latest DMM hashlist. From a glance it's downloading the whole hashlist just to see what the latest appends are. It manages to pull on the order of magnitude of 10GB a day.
Looking at the commits on https://github.com/debridmediamanager/hashlists they are append-only. It should be possible for Zilean to keep a local clone of that repo and only do git pulls on it instead of fetching the whole zipped archive on every sync. This will be much faster and and bandwidth-efficient compared to the existing method.
Can this be implemented? Thanks!