baemillion-data

Data source for the BaeMillion site

Publishing the data

On push to main this repo will automatically publish the contents of data/ and public/ to GitHub Pages.

The data will be available as latest.tar.gz and latest.zip which will be available at

• https://data.baemillion.com/latest.tar.gz
• https://data.baemillion.com/latest.zip

Processing data for the site

There are two scripts used for processing CSVs containing Message submissions and Timeline events to be used for the website. They require Python along with two dependencies, requests and pillow.

Installing the Python dependencies

The Python dependencies in this repo are maintained using uv. https://github.com/astral-sh/uv

# This ensures a new virtual environment for the repo is created
uv venv
# Activate the venv
source .venv/bin/activate
# Install all the dependencies
uv sync

A requirements.txt file has been generated by uv and added to the repo if you prefer to install via pip instead.

pip install -r requirements.txt

Running the scripts

Both the scripts take three arguments.

python <script>.py <path to CSV> <path to save images> <path to save JSON data>

So the full commands would look like the following.

python process_messages.py raw/messages.csv public/images/messages data/messages.json

python process_timeline.py raw/timeline.csv public/images/timeline data/timeline.json