Callback Analysis for Breathing

Code for analyzing respiratory traces during callback experiments in the Brainard Lab (UCSF/HHMI). Initially split from callback-analysis repo.

Files

Each of the below correspond to folders.

preprocessing

make data structures

Files usually note which data structure they're pulling from & where to make that structure. I've tried to remain consistent (or at least interpretable) with nomenclature.

• make_breath_dfs_plots.py
  • make dataframe for cbpt metadata
  • also plots kde-threshold for CBPT
• make-preprocessed_traces_npy.py
  • process wav files to filter, center, and normalize breath traces, saving as npy arrays
  • TODO: edit implementations to load & slice from these, rather than (1) making enormous dataframes or (2) reprocessing from raw wav file as needed. (UMAP stuff especially!)
• make-spontaneous_dfs.py
  • makes df & segments breaths for spontaneous data
  • TODO: integrate with make-preprocessed_traces_npy.py for efficiency.
• make-wav_snippets.py
  • save snippets of audio + breath as .wav files for breaths in df all_breaths

plot: for Callback Breathing Plot Tool

• plot-rolling_min_subtracted.py
  • lowpass_trace-rolling_min_seg plots for cbpt
• plot-spectrograms.py
  • spectrogram plots for cbpt

umap

00: callback, first insp only

First pass at umap, using only the first inspiration of each callback trial

• umap-00.00-train_first_insp.py
• umap-00.01-analyze_first_insp.py

01: callback, all breaths

Train UMAP on all of 1 breath segment type in callback data (eg, all insps). Can adjust which segment in files.

• umap-01.00-train_all_cb.py
• umap-01.01-analyze_all_cb.py
• umap-01.02-map_clusters.py
  • Which expirations come from which inspirations? Map insp and exp clusters.
  • I didn't refine expiration embedding/clustering particularly well, so this wasn't especially revealing.

02: spontaneous

Consider how spontaneous breaths fall into callback-trained embedding. No additional UMAP training goes on here.

• umap-02.00-add_spontaneous.py
• umap-02.01-analyze_spontaneous.py

Miscellaneous

• umap-input_walkthrough.py
  • Plot examples of the types of traces used for umap input.
  • Note: nearly all of the later embeddings simply use interpolated - UMAP is most useful for determining shape, since duration, timing, etc. can be added back later.

exploratory

In (rough) order of creation.

• zero_point.py
  • Addressing drift in respiratory trace over the course of a file (ie, non-constant zero point).
  • I wound up giving up on those files, since it was only the case in 1/4 birds
• amplitude_distributions.py
  • plot & fit breath amplitude distribution for many files; important step in determining zero point algorithm
• pca.py
  • How well can PCs explain the shapes of breath traces?
• phase.py
  • Initial implementation of phase
  • Some descriptive stuff
  • First pass at tying into UMAP clusters
• cycle_durations.py
  • Is mean cycle duration a good baseline for phase computation?
• long_calls/
  • See section below.

long calls

Do long calls have a special (HVC-dependent) relationship with ongoing breathing?

• threshold_lc.py: try thresholding long calls directly from breathing data, since not all audio data is usable.
• phase_lc.py: using the determinations from threshold_lc.py, think about phase & long calls
• whisperseg_lc.py: first pass at cross-verifying audio-segmented long calls (easier & better-defined problem) with breath-based threshold
  • whisperseg_long_calls.csv: list of long calls pulled out of ziggy's data by hamish.
  • there were only 2 useful birds in this data (gr56bu23, gr92gr19); many had stimulation, pharmacology, or something else going on.
  • those 2 birds had fairly few calls (557 non-long, 61 long; don't recall how hamish thresholded long calls.)
• spontaneous_cross_verify_lc.py
  • ran whisperseg on the spontaneous and callback files that I'd previously analyzed; ignore the files where whisperseg doesn't find calls.
  • this is well-positioned for expansion