build: dependency version sweep across PyAuto ecosystem

[Jammy2211]

Overview

All five PyAuto libraries have accumulated stale dependency caps on core packages (numpy, scipy, jax, astropy, scikit-image, scikit-learn). Many caps were originally needed for platform compatibility (JAX macOS ARM64 wheels) or API stability (NumPy 2.0 breaking changes) but those constraints have been resolved upstream. This sweep updates version constraints to current releases while maintaining CI stability.

Plan

• Bump safe caps first: scipy (→1.17), scikit-image (→0.26), scikit-learn (→1.8), astropy (→7.2), remove threadpoolctl cap
• Raise numpy cap past 2.0.1 (verify np.bool_ usage in PyAutoFit)
• Raise JAX cap past 0.5 — update deprecated jax.tree_util.Partial imports in PyAutoGalaxy, raise floor to 0.4.35 for ARM64 macOS wheels
• Bump secondary exact-pinned packages (SQLAlchemy, networkx, psutil) where safe; keep sampler pins stable
• Align PyAutoBuild numpy pin with ecosystem
• Run CI across all repos to validate

Detailed implementation plan

Affected Repositories

• PyAutoConf (primary — defines numpy, jax constraints)
• PyAutoFit
• PyAutoArray
• PyAutoGalaxy
• PyAutoLens
• PyAutoBuild

Work Classification

Library (all five core libraries + build tooling)

Branch Survey

Repository	Current Branch	Dirty?
PyAutoConf	main	clean
PyAutoFit	main	clean
PyAutoArray	main	clean
PyAutoGalaxy	main	clean
PyAutoLens	main	clean
PyAutoBuild	main	clean

Suggested branch: feature/dependency-sweep
Worktree root: ~/Code/PyAutoLabs-wt/dependency-sweep/

Implementation Steps

Tier 1 — Safe cap bumps (pyproject.toml only):

1. PyAutoFit/pyproject.toml: scipy <=1.15.2 → <=1.17.1, threadpoolctl >=3.1.0,<=3.2.0 → >=3.1.0
2. PyAutoArray/pyproject.toml: scipy <=1.15.2 → <=1.17.1, scikit-image <=0.24.0 → <=0.26.0, scikit-learn <=1.5.1 → <=1.8.0, astropy >=5.0,<=6.1.2 → >=5.0,<=7.2.0
3. PyAutoGalaxy/pyproject.toml: astropy >=5.0,<=6.1.2 → >=5.0,<=7.2.0

Tier 2 — NumPy:
4. PyAutoConf/pyproject.toml: numpy >=1.24.0,<=2.0.1 → >=1.24.0,<3.0.0
5. Verify PyAutoFit/autofit/messages/interface.py:195 np.bool_ annotation (np.bool_ exists in NumPy 2.x; np.bool was removed)

Tier 3 — JAX:
6. PyAutoConf/pyproject.toml: jax/jaxlib >=0.4.13,<0.5.0 → >=0.4.35,<0.10.0
7. PyAutoGalaxy/autogalaxy/profiles/mass/total/jax_utils.py:35: replace from jax.tree_util import Partial as partial with from functools import partial
8. PyAutoGalaxy/autogalaxy/operate/lens_calc.py:986: same Partial fix

Tier 4 — Secondary pins:
9. PyAutoFit/pyproject.toml: SQLAlchemy ==2.0.32 → ==2.0.40, networkx ==3.1 → latest, psutil ==6.1.0 → ==6.1.1
10. Keep sampler pins (dynesty, nautilus, zeus, ultranest) unchanged

Tier 5 — PyAutoBuild:
11. Update numpy cap in PyAutoBuild notebook tooling

Key Files

• PyAutoConf/pyproject.toml — numpy, jax, jaxlib constraints
• PyAutoFit/pyproject.toml — scipy, threadpoolctl, SQLAlchemy, secondary pins
• PyAutoArray/pyproject.toml — scipy, astropy, scikit-image, scikit-learn
• PyAutoGalaxy/pyproject.toml — astropy
• PyAutoGalaxy/autogalaxy/profiles/mass/total/jax_utils.py — JAX Partial import
• PyAutoGalaxy/autogalaxy/operate/lens_calc.py — JAX Partial import
• PyAutoBuild/ — numpy pin alignment

Original Prompt

Click to expand starting prompt

The dependencies of many core libraries, for example numpy, scipy, matplotlib and ultimately JAX,
are capped or tied based on the code API. Its probably been like this for a long time, and looking to update
these dependencies is a good idea, but must be balanced against source code tweaks and updates.

Can you do an assessment of whether we can update the version of these libraries whilst maintaining a stable
build where all github actions run and autobuild works ok. Dont just focus on the core libraries listed, but
also do an assessment of other key libraries like astropy, scikit-image, scikit-learn. Prioritize simplicity if
necessary, but ultimately I think a version sweep update is long overdue.

[Jammy2211]

Smoke Test Results — 2026-04-12

Workspace	Passed	Failed	Total	Notes
autofit_workspace	7	0	7
autogalaxy_workspace	3	3	6	failures are stale output/ cache (pre-existing)
autolens_workspace	7	0	7
autolens_workspace_test	5	3	8	failures are pynufft config + JAX tolerance flakes (pre-existing)

All 22 passing scripts confirmed unaffected by dependency cap changes.

The 6 failures are pre-existing and unrelated to this PR:

• autogalaxy start_here.py scripts: stale output/ from prior model structure (PyAutoFit#1179)
• autolens_test interferometer aggregator: transformer_class is None (pynufft not configured)
• autolens_test JAX interferometer/point_source: numerical tolerance flakes

PRs

• PyAutoConf: build: raise numpy, jax, jaxlib version caps #88
• PyAutoFit: build: raise scipy cap, relax threadpoolctl and SQLAlchemy pins PyAutoFit#1197
• PyAutoArray: build: raise caps on scipy, astropy, scikit-image, scikit-learn PyAutoArray#267
• PyAutoGalaxy: build: raise astropy cap, update JAX Partial imports for 0.5+ compat PyAutoGalaxy#344

[Jammy2211]

Shipped — 2026-04-12

PRs

• build: raise numpy, jax, jaxlib version caps #88
• build: raise scipy cap, relax threadpoolctl and SQLAlchemy pins PyAutoFit#1197
• build: raise caps on scipy, astropy, scikit-image, scikit-learn PyAutoArray#267
• build: raise astropy cap, update JAX Partial imports for 0.5+ compat PyAutoGalaxy#344

Summary

Raised version caps across the PyAuto ecosystem: numpy <3.0.0, jax/jaxlib <0.10.0 (floor >=0.4.35 for ARM64 macOS wheels), scipy <=1.17.1, astropy <=7.2.0, scikit-image <=0.26.0, scikit-learn <=1.8.0. Relaxed threadpoolctl and SQLAlchemy pins. Updated deprecated jax.tree_util.Partial imports in PyAutoGalaxy for forward compatibility with JAX 0.5+. All 3,070 unit tests pass, CI green on all 5 repos, 22/28 smoke tests pass (6 failures pre-existing).

Session Notes

• JAX macOS ARM64 concern was valid historically (early 0.4.x lacked wheels) but resolved since 0.4.35
• Sampler pins (dynesty, nautilus, zeus, ultranest) intentionally left unchanged — version changes can alter scientific results
• PyAutoBuild numpy pin also updated (requirements.txt)