Add Modal GPU calibration#279

Open

nwoodruff-co wants to merge 25 commits intomainfrom

feature/modal-gpu-calibration

Commits on Feb 20, 2026

Add Modal GPU calibration to speed up CI
nwoodruff-co
committed
Fix black formatting, dataset.copy(), and add changelog entry
nwoodruff-co
committed
Reduce peak memory in Modal path by building matrices sequentially
nwoodruff-co
committed
ci: re-trigger CI after runner preemption
nwoodruff-co
committed
fix: spawn constituency Modal job before building LA arrays to reduce peak memory
nwoodruff-co
committed
fix: interleave national/local matrix builds with explicit gc.collect()
nwoodruff-co
committed
fix: build national matrix once before app.run() to halve peak memory
nwoodruff-co
committed
fix: pass frs.copy() to create_national_target_matrix to avoid int time_period TypeError
nwoodruff-co
committed
fix: use serialized=True on Modal function to avoid policyengine_uk import in container
nwoodruff-co
committed
feat: return per-epoch weight checkpoints from Modal and write calibration logs
nwoodruff-co
committed
feat: offload imputation pipeline to high-CPU Modal container
nwoodruff-co
committed
Reduce peak memory by serialising matrices before building next one
nwoodruff-co
committed
Trigger CI
nwoodruff-co
committed
Fix Modal CPU image: install system HDF5 deps and pin policyengine-uk-data>=1.40.0
nwoodruff-co
committed
Enable Modal output for image build debugging
nwoodruff-co
committed
Fix Modal CPU image: use Python 3.13 for policyengine-uk-data compatibility
nwoodruff-co
committed
Use uv for CPU image installs; pre-install CPU-only torch to avoid CUDA download
nwoodruff-co
committed
Fix uv_pip_install index_url args; wrap app.run() with modal.enable_output()
nwoodruff-co
committed
Fix CPU image: use run_commands with pip to ensure policyengine-uk installs
nwoodruff-co
committed
Add serialized=True to run_imputation to avoid container importing policyengine_uk_data __init__
nwoodruff-co
committed
Install deps via uv in CPU image for faster cached builds
nwoodruff-co
committed
Copy local policyengine_uk_data source into CPU image instead of installing from PyPI
nwoodruff-co
committed
Fix: use add_local_dir not copy_local_dir
nwoodruff-co
committed
Upgrade Modal specs: 16 CPU/32GB for imputation, A10G GPU for calibration
nwoodruff-co
committed
Revert imputation offloading; Modal GPU for calibration only
nwoodruff-co
committed