Skip to content

hydra

Jump to bottom
Luigi Feriani edited this page May 11, 2021 · 4 revisions

This module contains functions that help in the compilation of metadata from experiments performed in the multi-camera rig (hydra).

Contents

README

This module contains its own readme file with the rescriptions of the digital experimental records that are expected as input in the functions to compile metadata. The readme markup file is accompanied by a flowchart that demonstrates the order in which the functions should be used to compile all the required records and create a single metadata file.

compile_metadata

This module contains all the functions needed to compile the records described in the the README and in the companion flowchart to create a single metadata file that can be readily linked to videos and tierpsy feature data.

This is the list of functions available to the user:

  • populate_96WPs
  • get_day_metadata
  • concatenate_days_metadata
  • get_source_metadata
  • merge_basic_and_source_meta
  • merge_robot_metadata
  • merge_robot_wormsorter

legacy:

  • day_metadata_check
  • number_wells_per_plate

copy_rawvideos_yamls

Script that copies all the metadata.yaml files from a source directory to a destination directory, while preserving the subfolder tree structure. Only useful when working from home: if other code relies on the existence of the yamls files, this is a quick alternative to copying all the RawVideos over the VPN or on a hard drive.

masked_videos_checks

Module with functions useful to perform checks on the number of videos taken during a day. These checks are performed on the MaskedVideos folder, so they are also useful to detect if Tierpsy skipped some videos.

Available functions:

  • get_prestim_videos: scans the directory containing the masked videos for one day of tracking, looking for videos with prestim in their path. Returns a DataFrame
  • missing_videos: checks that all cameras for each run have been processed, creates a csv if any videos are found to be missing
  • check_all_videos: checks that all the cameras were recording. Calls missing_videos.

The module has an example section that shows how to use the functions.

match_bluelight_videos

Module containing functions to group videos in prestim/bluelight/poststim triplets, provided that the videos are respectively 5, 6, and 5 minute long.

Functions available to the user:

  • parse_imaging_datetime: utility for extracting the yyyymmdd_HHMMSS date and time from a videos full path
  • match_bluelight_videos_in_folder: scans a target directory, returns matching prestim/bluelight/poststim triplets of videos. Returns a DataFrame with extra info such as rig, camera serial, date.

Example at the end of the module.

match_wells_annotations

Module with functions for adding the wells annotations from the WellAnnotator GUI to a metadata file. It only works if only the prestim videos were annotated in the GUI

Functions available to the user:

  • import_wells_annotations: reads a wells_annotations.hdf5 file and returns a dataframe with the annotations
  • import_wells_annoations_in_folder: scans a directory for all well_annotations.hdf5 files, reads them with the function above and returns a single dataframe with all the annotations
  • match_rawvids_annotations: given the annotations for the prestim videos, and a folder containing also the bluelight and poststim videos, finds the matching poststim and bluelight videos for each prestim annotation. Return a long format dataframe (one well per row)
  • update_metadata: merges the matched annotations with an existing metadata file.

Example at the end of the module.

platechecker

Module to create an html report that shows whether the imaging plates were well placed in the Hydra rigs, and sensor data over time (light intensity, temperature and humidity inside the imaging rig, temperature outside the rig).

It assumes that the metadata are in place and that there are no empty cells in columns with a name containing serial, date, slot, or number (these columns can be entirely empty, but not have missing values).

This should just be used as a command line tool (in the Terminal, try platechecker --help for instructions, with the tierpsytools environment activated).

read_imgstore_extradata

Module with classes and functions to read the sensor data included in the imgstore containers.

Available:

  • ExtraDataReader: class (wrapper for loopbio's own class), allows to read sensor data for one imgstore
  • check_hydra_sensordata: function that scans a directory, reads all the sensor data, and can create a report in pdf format.

This module should just be used as a command line tool (in the Terminal, try hydra_sensordata_report --help for instructions, with the tierpsytools environment activated)

Clone this wiki locally