DashPVA: Distributed Analysis and Streaming Hub with Process Variable Access

DashPVA enables distributed analysis and real-time image streaming using PvaPy as its backend. It includes:

• Live Image Visualization
• ROI Monitoring and Statistics from area detector EPIC IOC
• Predetermined and Spontaneous/Random Scans visualization

Features

• Configurable GUIs for setting up detector prefixes, collectors, and more.
• Real-Time Analysis: Supports frame-by-frame image processing.
• Integration with PvaPy: Leverages PvaPy to access process variables efficiently.

Getting Started

Follow the instructions in the RUN_INSTRUCTIONS.md for setup and usage.

FAQ: Area Detector and Data Analysis System

What is the primary function of the software described in the provided sources?

• This software suite provides a comprehensive system for acquiring, visualizing, and analyzing data from area detectors, primarily within a scientific or experimental context. It integrates real-time data streaming, region of interest (ROI) analysis, image display, and data caching capabilities, and is built to interact with EPICS control systems for instrument parameters. It manages the flow of data from a detector, processes it, allows for user-defined ROIs, provides both live and historical data, and saves this data for later analysis.

How does the system handle real-time data from the detector?

• The system uses the PVAccess (PVA) protocol to establish a channel for monitoring the detector's data stream. It employs a callback mechanism that is triggered every time new data is received, allowing it to process the data in real-time. The incoming data is cached, and it also can be processed live using multiple different consumer types. These can be spontaneous which caches data as it is received, or vectorized which processes based on a scan plan. The system is also capable of monitoring motor positions and other metadata associated with the detector. The data is also displayed using the pyqtgraph library for live visualization.

How are Regions of Interest (ROIs) defined and used in the software?

• ROIs are defined as rectangular areas within the detector image. They are configured using an external configuration file that specifies the starting x and y coordinates, as well as the width and height of each ROI. ROIs can be dynamically adjusted through EPICS control PVs and visualized on the main image display, allowing for real-time monitoring of specific regions in the image. The software calculates and displays statistical information like total intensity, mean, and sigma within each ROI, which can be viewed in pop-up dialogs.

What are the different data consumption and analysis modes supported by the software?

• The software supports two main data consumption modes: "spontaneous" and "vectorized." In spontaneous mode, data is processed and cached as it arrives, which is useful for data streams without a fixed scan pattern. Vectorized mode, on the other hand, processes data according to a defined scan plan, which allows users to relate data points to positions in a 2D scan. Additionally, the software allows processing using external scripts written by the user, for custom analysis.

How does the software manage data caching, and what limitations exist?

• The software implements a circular buffer system to manage the caching of detector images and position data. This buffer is limited to a specified maximum size (default is 900 frames), and new data overwrites the oldest data in the cache. The size of the cache can be an important factor for analyzing data, especially for retrospective scans. It also caches the analysis information from the detector. The data is stored and can be saved into an HDF5 file.

What kind of plots and visualizations does the system offer?

• The software provides a variety of visualizations, including a main image display of the detector data, scatter plots for vectorized analysis (showing intensity, center of mass in x and y), and plots of the horizontal and vertical averages. It uses Pyqtgraph for real-time plotting and offers features like adjustable color scaling, region selection, and pixel information readouts. The system allows the display to be frozen which will stop updating the display but continue to collect data.

How does the software interact with EPICS control systems?

• The software heavily relies on EPICS (Experimental Physics and Industrial Control System) for both data acquisition and control of experiment parameters. It uses Channel Access (CA) for monitoring PVs that specify ROIs. The PVAccess (PVA) library is used to subscribe to detector data, and control PVs, and the metadata associated with the detector are also monitored through EPICS. It also is able to read custom PV names from external configuration files.

What is the workflow for setting up and running an analysis?

• The workflow begins by configuring the system using a dialog box, including the prefix for the PVs and the address of the data collector. The user then loads a configuration file defining the PVs to be monitored. The user can define ROIs, and open the analysis window that displays the data, allows setting up scan locations through uploaded .npy files, and gives the ability to save the data into HDF5 files. It is also capable of running both simulated and real data.