Goals

Each section covers an aspect of the system. These can be subdivided into smaller sections if necessary. Each section contains one or more goals which are suffixed with a priority from LOW to HIGH.

Crawlers

• Should have a centralized interface (fetch, parse, store) to make for easy iterating over all existing crawlers (LOW)
• Should be able to store the data in any standard datastore (look into something like SQLAlchemy) (LOW)

Backend

• TBD

UI

• TBD