This repository contains scripts for processing REALTA data, including generating shell scripts for CDMT processing and other utilities for mass filterbank generation and data management. Along with database generation to keep track of what data has been processed on REALTA.
At present there are two scripts that are used to generate a database of voltages and filterbanks. They both need to be run on REALTA as a sudo user to access all mounted drives and directories. These scripts can be run individually or together using bash update_csv.sh which will save the generated CSV files to the csv_files directory one with a date stamp and another with a generic name to be used in the scripts. REALTA-Voltage-Files.csv and REALTA-Observation-Files.csv.
The gen_CDMT_shell.py script is used to generate shell scripts for CDMT processing. It is a wrapper for David's cdmtProc.py script, nothing more. It takes the CSV of voltage files and generates a shell script for each set of voltage files that has 4 valid lanes. It orders them in size of voltages, with the intent of processing the largest files first.
voltage_cleaner.py can be used to clear out voltages, it reads .csv files generated by update_csv.sh and then grabs relevant file sizes and checks them against expected size and schedule entry. For each matched filterbank and voltage set the user is prompted with a y/n to delete the voltages. Care should be taken when using this script, never delete voltages if you're unsure if their filterbanks are corrupt or incorrectly generated.