Make bypass test mode write a complete on-disk result#1192
Merged
Jammy2211 merged 1 commit intomain_buildfrom Apr 10, 2026
Merged
Make bypass test mode write a complete on-disk result#1192Jammy2211 merged 1 commit intomain_buildfrom
Jammy2211 merged 1 commit intomain_buildfrom
Conversation
Three things were broken for `PYAUTOFIT_TEST_MODE=2` (bypass mode):
1. `SamplesPDF.log_evidence` hard-coded `return None`, so any downstream
arithmetic on the log evidence (grid search comparisons, subhalo
Bayesian model comparison, scraped-aggregator assertions) crashed
with `NoneType - float`. Now it returns `samples_info.get("log_evidence")`
— matching the pattern already used by `SamplesNest`.
2. `_fit_bypass_test_mode` never persisted samples or the samples
summary to disk even though its docstring claimed to write "all
expected output files". Downstream code that reads from the output
folder (database scrape, `paths.load_samples_summary`) saw a
half-populated directory. Now calls `save_samples_summary` and
`save_samples`; both are no-ops on `NullPaths` and safe on
`DatabasePaths`, so all path modes remain correct.
3. `fit()` skipped `pre_fit_output` in bypass mode and only wrote
`model.info`, omitting the `metadata` file that the aggregator uses
to discover search directories. Now calls the full `save_all`
(lightweight JSON dumps only — still skips the expensive
`analysis.save_attributes` / `visualize_before_fit` steps).
Together these make `autofit search_grid_search`, `autolens subhalo
detect`, and `autofit_workspace_test database/scrape/grid_search` work
end-to-end in bypass mode.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three tightly-coupled bugs in
PYAUTOFIT_TEST_MODE=2(bypass mode) were causing the post-API-drift mega-run cluster ofNoneTypearithmetic errors onlog_evidenceand the database scrape test's IndexError.All three ship together because they share the same root cause — bypass mode was supposed to produce a fully-populated result folder, but its implementation was incomplete:
SamplesPDF.log_evidencereturnedNoneunconditionally_fit_bypass_test_modenever calledsave_samplesorsave_samples_summary(despite its docstring)fit()skippedpre_fit_outputin bypass mode, which meant themetadatafile used by the aggregator to discover search directories was never writtenAfter this PR,
autofit features/search_grid_search.py,autolens imaging/features/advanced/subhalo/detect/start_here.py, andautofit_workspace_test database/scrape/grid_search.pyall pass end-to-end in bypass mode.API Changes
SamplesPDF.log_evidencenow returnssamples_info.get("log_evidence")instead ofNone.metadata,samples.csv, andsamples_summary.jsonto the output folder. No public signatures changed; this only affects what files exist on disk after a bypass-mode run.See full details below.
Test Plan
test_autofit/non_linear/search+test_autofit/non_linear/samples— 65 passedFull API Changes (for automation & release notes)
Changed Behaviour
autofit.non_linear.samples.pdf.SamplesPDF.log_evidence— now returnsself.samples_info.get("log_evidence")instead of hardcodedNone. Only affects pureSamplesPDFinstances;SamplesNestandSamplesMCMCoverride and are unchanged.autofit.non_linear.search.abstract_search.NonLinearSearch._fit_bypass_test_mode— now stubslog_evidenceinsamples_info(set to the single evaluatedlog_likelihood) and callsself.paths.save_samples_summary(...)andself.paths.save_samples(...)so downstream readers see a complete result folder.autofit.non_linear.search.abstract_search.NonLinearSearch.fit— whentest_mode_level() >= 2, now callsself.paths.save_all(...)instead of only_save_model_info. This writes themetadatafile that the aggregator uses to discover search directories. Still skipsanalysis.save_attributesandvisualize_before_fitto keep bypass mode cheap.Migration
No migration needed for library consumers. If you have custom code that relied on
SamplesPDF.log_evidencebeingNone, replace with an explicit check againstsamples_info.🤖 Generated with Claude Code