library dependencies need to be managed more cleanly

[matthewcornell]

Right now we have Python depedencies pinned using two methods, and the results are contradictory and can cause tests to fail depending on where and how they're run. The files involved are pyproject.toml, uv.lock, and requirements/requirements.txt/ and requirements-dev.txt. As of this writing, the CI tests run correctly b/c they use requirements/requirements-dev.txt, but local test runs using uv failed.

Following are two reports from Claude:

• summary
• recommended solution

Todo:

• incorporate ideas/history from @lshandross 's Slack post

Claude's summary ("three different files, all derived from pyproject.toml, serving different purposes"):

pyproject.toml — the source of truth. Declares direct deps with constraints (pandas~=2.0, etc.). This is what pip/uv reads when someone installs idmodels as a package.

---

requirements/requirements.txt and requirements/requirements-dev.txt — generated by uv pip compile, pip-compatible lockfiles for developing idmodels itself (running tests locally, etc.):

uv pip compile pyproject.toml -o requirements/requirements.txt
uv pip compile pyproject.toml --extra dev -o requirements/requirements-dev.txt

These are the same pattern you just set up for operational-models.

---

uv.lock — generated by uv lock (a different command). This is uv's native lockfile format (like poetry.lock or Cargo.lock). It covers multiple Python versions (note the resolution-markers at the top), includes download URLs and hashes, and is used by uv sync / uv run. It's more comprehensive but only works with uv.

---

The redundancy: uv.lock and requirements/*.txt serve the same conceptual purpose — pinning all transitive deps for reproducible installs. Having both suggests the repo supports two workflows: uv sync for uv users, and pip install -r requirements/requirements.txt for everyone else. The versions also differ (e.g. uv.lock has aiobotocore==2.25.0, requirements.txt has 2.15.2), so they're not in sync and whichever was regenerated last is more current.

Claude's recommended solution ("make uv.lock the single source of truth and derive requirements/*.txt from it using uv export instead of uv pip compile.")

cd /Users/cornell/IdeaProjects/idmodels

uv export --format requirements-txt -o requirements/requirements.txt
uv export --format requirements-txt --extra dev -o requirements/requirements-dev.txt

uv export reads uv.lock and writes pip-compatible output — so the two are guaranteed to match. The headers in the files will change from uv pip compile pyproject.toml ... to uv export ..., which is the correct attribution.

Then whenever you update pyproject.toml:

uv lock                  # update uv.lock
uv export --format requirements-txt -o requirements/requirements.txt
uv export --format requirements-txt --extra dev -o requirements/requirements-dev.txt

Or if you just want to refresh to latest versions:

uv lock --upgrade
# then the two export commands above

This replaces the current uv pip compile workflow for idmodels and eliminates the divergence. The requirements/*.txt files become a pip-compatibility layer on top of uv.lock rather than independently resolved lockfiles.