Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude/skills/run_all_script_fix_failures/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Match PyAutoBuild behavior instead of inventing a separate policy:
3. Use test mode by default:

```bash
PYAUTOFIT_TEST_MODE=1 python <script.py>
PYAUTO_TEST_MODE=1 python <script.py>
```

4. Build the script list using the PyAutoBuild skip policy and the workspace ordering above. Do not use `run_all_scripts.sh`, because that workflow is designed around continuing after failures and writing failure logs under `failed/`.
Expand Down
2 changes: 1 addition & 1 deletion .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ You are working on the **autogalaxy_workspace**, a tutorial/example repository f

## Testing

`run_all_scripts.sh` sets `PYAUTOFIT_TEST_MODE=1` automatically. Every script should pass in this mode. A script that fails in test mode indicates a real problem (broken import, wrong function name, etc.).
`run_all_scripts.sh` sets `PYAUTO_TEST_MODE=1` automatically. Every script should pass in this mode. A script that fails in test mode indicates a real problem (broken import, wrong function name, etc.).

## Notebook Generation

Expand Down
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,15 @@ python scripts/imaging/modeling.py

## Testing

Set `PYAUTOFIT_TEST_MODE=1` so non-linear searches skip actual sampling (fast test run).
Set `PYAUTO_TEST_MODE=1` so non-linear searches skip actual sampling (fast test run).

Run all scripts:

```
bash run_all_scripts.sh
```

This sets `PYAUTOFIT_TEST_MODE=1` automatically, runs every script in `scripts/`, and reports pass/fail/skipped counts. Failure logs are written to `failed/<relative_path>.log`.
This sets `PYAUTO_TEST_MODE=1` automatically, runs every script in `scripts/`, and reports pass/fail/skipped counts. Failure logs are written to `failed/<relative_path>.log`.

### Script ordering

Expand Down
14 changes: 7 additions & 7 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,19 +22,19 @@ python scripts/interferometer/modeling.py

Jupyter notebooks mirror the scripts in `notebooks/` and use `# %%` markers to separate cells.

**Integration testing / fast mode**: Set `PYAUTOFIT_TEST_MODE=1` to run any example script without performing full non-linear search sampling. The search is skipped and a mock result is returned immediately, making this ideal for high-level integration testing of scripts end-to-end:
**Integration testing / fast mode**: Set `PYAUTO_TEST_MODE=1` to run any example script without performing full non-linear search sampling. The search is skipped and a mock result is returned immediately, making this ideal for high-level integration testing of scripts end-to-end:

```bash
PYAUTOFIT_TEST_MODE=1 python scripts/imaging/modeling.py
PYAUTO_TEST_MODE=1 python scripts/imaging/modeling.py
```

**Fast smoke tests**: For maximum speed, combine test mode with small datasets and fast plots:
**Fast smoke tests**: For maximum speed, combine test mode with all skip flags and small datasets:

```bash
PYAUTOFIT_TEST_MODE=2 PYAUTO_WORKSPACE_SMALL_DATASETS=1 PYAUTO_FAST_PLOTS=1 python scripts/imaging/modeling.py
PYAUTO_TEST_MODE=2 PYAUTO_SKIP_FIT_OUTPUT=1 PYAUTO_SKIP_VISUALIZATION=1 PYAUTO_SKIP_CHECKS=1 PYAUTO_SMALL_DATASETS=1 PYAUTO_FAST_PLOTS=1 python scripts/imaging/modeling.py
```

- `PYAUTO_WORKSPACE_SMALL_DATASETS=1` — caps all grids/masks to 15x15 pixels at 0.6"/px, making simulators and all downstream computations dramatically faster. Delete `dataset/` when toggling this variable.
- `PYAUTO_SMALL_DATASETS=1` — caps all grids/masks to 15x15 pixels at 0.6"/px, making simulators and all downstream computations dramatically faster. Delete `dataset/` when toggling this variable.
- `PYAUTO_FAST_PLOTS=1` — skips `plt.tight_layout()` and critical curve/caustic overlay computation in subplot functions.

**Codex / sandboxed runs**: when running from Codex or any restricted environment, set writable cache directories so `numba` and `matplotlib` do not fail on unwritable home or source-tree paths:
Expand All @@ -55,14 +55,14 @@ Run all scripts end-to-end using the provided runner:
bash run_all_scripts.sh
```

- Sets `PYAUTOFIT_TEST_MODE=1` automatically (skips non-linear sampling)
- Sets `PYAUTO_TEST_MODE=1` automatically (skips non-linear sampling)
- Failure logs written to `failed/<relative_path>.log`; execution continues after each failure
- Final summary shows pass/fail/skipped counts

To run a single script in test mode:

```bash
PYAUTOFIT_TEST_MODE=1 python3 scripts/imaging/modeling.py
PYAUTO_TEST_MODE=1 python3 scripts/imaging/modeling.py
```

### Script ordering rules
Expand Down
11 changes: 7 additions & 4 deletions config/build/env_vars.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,11 @@
# - Patterns without '/' match the file stem exactly

defaults:
PYAUTOFIT_TEST_MODE: "2" # 0=normal, 1=reduced iterations, 2=skip sampler (fastest)
PYAUTO_WORKSPACE_SMALL_DATASETS: "1" # Cap grids/masks to 15x15, reduce MGE gaussians
PYAUTO_TEST_MODE: "2" # 0=normal, 1=reduced iterations, 2=skip sampler (fastest)
PYAUTO_SKIP_FIT_OUTPUT: "1" # Skip pre/post-fit I/O, VRAM profiling, result text
PYAUTO_SKIP_VISUALIZATION: "1" # Skip fit visualization and plotting
PYAUTO_SKIP_CHECKS: "1" # Skip mesh validation, position checks, weight thresholds
PYAUTO_SMALL_DATASETS: "1" # Cap grids/masks to 15x15, reduce MGE gaussians
PYAUTO_DISABLE_JAX: "1" # Force use_jax=False, avoid JIT compilation overhead
PYAUTO_FAST_PLOTS: "1" # Skip tight_layout() + critical curve/caustic overlays
JAX_ENABLE_X64: "True" # Enable 64-bit precision in JAX
Expand All @@ -20,10 +23,10 @@ defaults:
overrides:
# Tutorial loads FITS data at full resolution
- pattern: "howtogalaxy/chapter_1_introduction/tutorial_3_fitting"
unset: [PYAUTO_WORKSPACE_SMALL_DATASETS]
unset: [PYAUTO_SMALL_DATASETS]
# guides/results/start_here.py must produce real samples so the example
# scripts that read from `output/results_folder` afterwards
# (data_fitting, queries, models, samples_via_aggregator) find a
# non-empty aggregator. Covers all scripts under guides/results/.
- pattern: "guides/results/"
unset: [PYAUTOFIT_TEST_MODE]
unset: [PYAUTO_TEST_MODE, PYAUTO_SKIP_FIT_OUTPUT]
7 changes: 5 additions & 2 deletions run_all_scripts.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash
# Run all scripts in scripts/ with PYAUTOFIT_TEST_MODE=1.
# Run all scripts in scripts/ with PYAUTO_TEST_MODE=1.
#
# Rules:
# - start_here.py in a folder runs before all other scripts and subfolders in that folder
Expand All @@ -12,7 +12,10 @@ FAILED_DIR="$SCRIPT_DIR/failed"
NO_RUN_YAML="$SCRIPT_DIR/../PyAutoBuild/autobuild/config/no_run.yaml"
PROJECT_KEY="autogalaxy"

export PYAUTOFIT_TEST_MODE=1
export PYAUTO_TEST_MODE=1
export PYAUTO_SKIP_FIT_OUTPUT=1
export PYAUTO_SKIP_VISUALIZATION=1
export PYAUTO_SKIP_CHECKS=1

# ---------------------------------------------------------------------------
# Build ordered script list: within each directory, start_here.py runs first,
Expand Down
6 changes: 3 additions & 3 deletions scripts/guides/results/start_here.py
Original file line number Diff line number Diff line change
Expand Up @@ -93,9 +93,9 @@

model = af.Collection(galaxies=af.Collection(galaxy=galaxy))

test_mode_was_on = os.environ.get("PYAUTOFIT_TEST_MODE") == "1"
test_mode_was_on = os.environ.get("PYAUTO_TEST_MODE") == "1"
if test_mode_was_on:
os.environ.pop("PYAUTOFIT_TEST_MODE", None)
os.environ.pop("PYAUTO_TEST_MODE", None)

search = af.Nautilus(
path_prefix=Path("results_folder"),
Expand All @@ -111,7 +111,7 @@
result = search.fit(model=model, analysis=analysis)

if test_mode_was_on:
os.environ["PYAUTOFIT_TEST_MODE"] = "1"
os.environ["PYAUTO_TEST_MODE"] = "1"

"""
__Info__
Expand Down
2 changes: 1 addition & 1 deletion smoke_tests.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
imaging/start_here.py
imaging/fit.py
# imaging/modeling.py # disabled: result.max_log_likelihood_instance fails in PYAUTOFIT_TEST_MODE=2 (rhayes777/PyAutoFit#1179)
# imaging/modeling.py # disabled: result.max_log_likelihood_instance fails in PYAUTO_TEST_MODE=2 (rhayes777/PyAutoFit#1179)
interferometer/start_here.py
multi/start_here.py
# ellipse/modeling.py # disabled: bypass mode tuple-path KeyError on ellipse models (rhayes777/PyAutoFit#1179)
Expand Down