Prototype on using the in situ tool Deisa with a radio-interferometric imaging pipeline. The largely same pipeline is used across all examples, where de/gridding is parallelized by EM-frequency across multiple nodes, after which the individual dirties and psfs and reduced, with deconvolution being performed on a single node. The deconvolution algorithm used here is a L1 regularized method using Fista. The main differences between the different pipelines are how the data is being shared with pdi/deisa.
The repository contains a few example applications of using Deisa with the above pipeline:
- A simple case where the reconstructed and residual image is shared with deisa every major cycle. Deisa then performs source finding on the reconstructed image, and a variance estimation on the residual using a sliding window to maintain some locality.
- A case where visibilities are shared from each individual de/gridding node to deisa, which performs a jackknife resampling (e.g., https://www.aanda.org/articles/aa/abs/2025/03/aa51927-24/aa51927-24.html) in parallel across the different workers in order to generate a population of reference residual images. The actual residuals from the reconstructions across the different major cycles are then recuperated and plotted against this population of reference residuals in a series of scatter plots. The ideal case here is that the final residual has roughly the same distribution as the reference residual population. If it has a larger flux, it means that we are not cleaning enough, whereas a smaller flux means we are over cleaning and reconstructing noise.
The datasets used are simulated using RASCIL (https://developer.skao.int/projects/rascil/en/latest/) and can be downloaded using the get_data.sh script.
To run the applications, run setup.sh, which should create a venv inside a spack environment. Afterwards, to run an application, activate both environments and then run the launch.sh in the respective applications' directories. A results folder may need to be created if it does not yet exist, as the output plots and data are written there.