Discontinuities in unmixed data

[DrCytometer]

Issue: Discontinuities sometimes appear in the unmixed data when using autospectral unmixing.

Cause: Spectral variant optimization utilizes cut-offs based on the unmixed unstained sample, as in TRU-OLS. Cells just above this threshold tend to be "optimized" away from it, creating a gap in some cases.

Temporary fix: Use autospectral unmixing without spectral variant (per-cell fluorophore) optimization. Do not supply anything to spectra.variants.

Possible solution: Weight spectral updates per cell relative to the fluorophore's abundance over the threshold. Cells near the threshold will not move much. Cells far from the threshold will still be optimized for spillover spread and skewing.

[DrCytometer]

Partial fix implemented in dev branch with version 0.9.0.

Part of the problem derives from incorporation of autofluorescence variation into the variant spectral signatures, so the impacts appear on unmixed channels affected by autofluorescence, either due to fluorophore similarity with predominant AF signatures or due to the cell type associated with the fluorophore being particularly autofluorescent (e.g., monocytes, macrophages). This has been addressed with changes to the definition of the variant signatures. AF identification at the single-cell level in the single-colour controls for this process has been altered to use only that single fluorophore in the unmixing, providing more information about AF. The similarity threshold has been raised to 0.99. The number of spectra identified has been dropped (which works a bit better and is considerably faster), which appears to reduce the influence of noisy cells on any single variant. Finally, low-level denoising has been applied to the variant spectra, regularizing towards the single best spectrum in any channels where the signal in the best spectrum is 5% or less of the peak. This retains the information causing spread and skewing with less random variation, I think.

[DrCytometer]

This should be mostly fixed in the next version (tentatively 1.0.0) to be released.

[DrCytometer]

This should be mostly fixed now. If you experience discontinuities in the unmixed data and are willing to share files for troubleshooting, contact me.