Skip to content

Conversation

@hpparvi
Copy link
Contributor

@hpparvi hpparvi commented Jan 25, 2026

This PR adds uncertainty propagation to the Background class and introduces sigma clipping for outlier rejection in background estimation. It also adds tests for the functionality and updates the documentation.

New Features

Uncertainty Propagation

The Background class now computes and propagates uncertainties through all background operations:

  • bkg_image() - Returns background image with uncertainty
  • bkg_spectrum() - Returns 1D background spectrum with uncertainty
  • sub_image() - Returns background-subtracted image with propagated uncertainty
  • sub_spectrum() - Returns background-subtracted 1D spectrum with propagated uncertainty

Variance Calculation

For the average statistic:

Var(weighted_mean) = Σ(w² × σ²) / (Σw)²

For the median statistic:

Var(median) ≈ (π/2) × σ² / n

Automatic Variance Estimation

When input image has no uncertainty provided, variance is automatically estimated from the sample variance of flux values in the background region.

Uncertainty Type Preservation

The output uncertainty type matches the input:

  • VarianceUncertaintyVarianceUncertainty
  • StdDevUncertaintyStdDevUncertainty
  • InverseVarianceInverseVariance

Sigma Clipping for Outlier Rejection

Added optional sigma parameter for sigma clipping to reject cosmic rays and other outliers in the background region:

# Default: sigma=5.0
bg = Background.two_sided(image, trace, separation=5, width=4)

# Tighter clipping for aggressive outlier rejection
bg = Background.two_sided(image, trace, separation=5, width=4, sigma=3.0)

# Disable sigma clipping
bg = Background.two_sided(image, trace, separation=5, width=4, sigma=None)

The outlier mask is stored separately in _outlier_mask and does not modify the original image mask.

Bug Fixes

  • Fixed incorrect axis reference in trace position calculation: now correctly uses crossdisp_axis instead of disp_axis when computing default trace position.

Documentation

  • Updated docs/background.rst with new sections on outlier rejection and uncertainty propagation
  • Updated docs/extraction.rst with uncertainty propagation documentation
  • Updated CHANGES.rst changelog

…ts for uncertainty in background estimation, subtraction, and propagation methods.
…rovided. Enhanced tests to validate variance computation and propagation.
… `Background` methods. Improved uncertainty type preservation and propagation logic.
…ariance computation to incorporate clipping mask, and implemented new tests for sigma clipping.
@codecov
Copy link

codecov bot commented Jan 25, 2026

Codecov Report

❌ Patch coverage is 97.10145% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.19%. Comparing base (1d6491e) to head (c0950ff).
⚠️ Report is 18 commits behind head on main.

Files with missing lines Patch % Lines
specreduce/background.py 97.10% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #297      +/-   ##
==========================================
+ Coverage   89.90%   90.19%   +0.29%     
==========================================
  Files          17       17              
  Lines        1822     1877      +55     
==========================================
+ Hits         1638     1693      +55     
  Misses        184      184              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@tepickering
Copy link
Contributor

the RTD is failing due to some warnings that are being cast as errors. some of the warnings are related to ArrayLike which i thought i had fixed in a recent PR. not sure what the deal is with the other warnings. would need to dig more deeply to see what's what. the other failure is due to tcl/tk being missing on the windows action runner. i also thought i had that fixed. a more proper fix would be to force the use of a generic Agg backend for the tests rather than relying on some configured default.

@hpparvi
Copy link
Contributor Author

hpparvi commented Jan 27, 2026

the RTD is failing due to some warnings that are being cast as errors. some of the warnings are related to ArrayLike which i thought i had fixed in a recent PR. not sure what the deal is with the other warnings. would need to dig more deeply to see what's what. the other failure is due to tcl/tk being missing on the windows action runner. i also thought i had that fixed. a more proper fix would be to force the use of a generic Agg backend for the tests rather than relying on some configured default.

This is fixed now, but it took some time to identify the issue. Apparently, complex type annotations in data classes can cause problems, but I'm not sure why this became an issue only now. I added some nitpick_ignore_regex rules to the Sphinx configuration that did the trick.

I also added some missing packages to the specreduce-dev conda environment.

Copy link
Contributor

@tepickering tepickering left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great work! this is pretty key functionality to have in place to properly handle spectra and their uncertainties.

nitpick_ignore = [
("py:class", "ArrayLike"),
("py:class", "numpy._typing.ArrayLike"),
("py:class", "specutils.spectra.spectrum1d.Spectrum1D"),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this because we're still referencing Spectrum1D or is it on the specutils end? either way, i think ignoring is the right approach for now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fixes an issue I ran into while testing with specutils < 2. Then again, I believe we can finally pin specutils >= 2 for the next release. I'll clean up this and the remaining compatibility workarounds in a separate PR.

@hpparvi hpparvi merged commit 2bc31c5 into astropy:main Jan 28, 2026
14 checks passed
@hpparvi hpparvi deleted the bkg_sub_error_propagation branch February 1, 2026 19:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants