Skip to content

[Feature request] Support for incremental DMM pulls #101

@giorgi1324

Description

@giorgi1324

Zilean scraper is pretty wasteful when it comes to syncing the latest DMM hashlist. From a glance it's downloading the whole hashlist just to see what the latest appends are. It manages to pull on the order of magnitude of 10GB a day.

Looking at the commits on https://github.com/debridmediamanager/hashlists they are append-only. It should be possible for Zilean to keep a local clone of that repo and only do git pulls on it instead of fetching the whole zipped archive on every sync. This will be much faster and and bandwidth-efficient compared to the existing method.

Can this be implemented? Thanks!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions