Daily Pattern Classifier

---

Overview

This code is from M.S. research in applying deep learning and recurrent neural networks to eating detection. The M.S. thesis that accompanies this research can be found here, "Detecting eating episodes from daily patterns of wrist motion using recurrent neural networks". The Clemson All-Day (CAD) dataset used in this project is publicly available.

Requirements

This project requires the following:

• Python 3.8.3
• TensorFlow 2.2.0
• NumPy 1.19.5
• Pandas 1.1.1
• tqdm 4.61.2
• scikit-learn 0.23.2

Repository Structure

common/                     // directory for scripts and text files used by other programs
    batch-unix.txt          // filenames for all recordings in CAD dataset
    meals-shimmer.txt       // information for all meals in CAD dataset
    loadfile.py             // contains functions for loading CAD dataset data
    testing.py              // contains functions for evaluation
    training.py             // contains functions for training window-based classifier (Sharma 2020)

GenerateSamples/            // code to generate daily samples for training daily pattern classifier
    GenerateSamples.ipynb   // Jupyter notebook for generating daily samples
    GenerateSamples.py      // Python program for generating daily samples
    LoadFiles.ipynb         // loads many daily sample text files and saves as combined .npy files
    SubmitGenSamplesJob.py  // script to run GenerateSamples on Palmetto cluster as a PBS job

DailyPatternClassifier/     // code to train and evaluate the daily pattern classifier
    DailyPatternRNN.ipynb   // performs k-fold cross validation for training AND testing (Jupyter notebook)
    TrainDailyPatternRNN.py // performs training for k-fold cross validation
    TestDailyPatternRNN.py  // evaluates time and episode metrics post-hoc for k-fold cross validation
    SubmitTrainRNNJob.py    // script to run TrainDailyPatternRNN on Palmetto cluster as a PBS job

Code Description

Code can be found at https://github.com/apatyk/Daily-Pattern-Classifier. Each program or script mentioned above is briefly described here.

1. loadfile.py: This Python file contains a variety of functions used to load the Shimmer data from the Clemson All-Day (CAD) dataset. This code is adapted from previous work (Sharma 2020).
2. testing.py: This Python file contains several functions and their related auxiliary functions for testing the window-based classifier from Sharma as well as the daily pattern classifier.
3. training.py: This Python file encapsulates training code for the window-based classifier.
4. GenerateSamples.ipynb/.py: These programs perform data augmentation and generate a collection of daily samples needed to train the daily pattern classifier.
5. SubmitGenSamplesJob.py: This Python script creates a PBS script to submit a GenerateSamples job to the Palmetto cluster. This is the preferred use of these scripts as GenerateSamples takes more than 24 hours to run to generate 200,000 samples for a 6 minute window.
6. LoadFiles.ipynb: This Jupyter notebook loads all of the daily sample text files generated by GenerateSamples and saves them collectively to binary NumPy files .npy. Data is divided into files for samples, labels, and filenames. Cells are included in the notebook to load and export a single directory or multiple directories (in parallel).
7. TrainDailyPatternRNN.py: This Python program trains daily pattern classifiers for k-fold cross validation using bundled daily sample arrays from LoadFiles. The number of training epochs, batch size, and number of recurrent units in the model architecture can be defined in command line arguments. This is utilized by the SubmitTrainRNNJob script.
8. TestDailyPatternRNN.py: This Python program evaluates time and episode metrics post-hoc for k-fold cross validation and saves the results to a CSV file. Varying thresholds can be used for thresholding eating episode segmentation.
9. SubmitTrainRNNJob.py: This Python script creates a PBS script to submit a TrainDailyPatternRNN job to the Palmetto cluster.
10. DailyPatternRNN.ipynb: This Jupyter notebook trains AND tests daily pattern classifiers with k-fold cross validation. This notebook performs the functions of TrainDailyPatternRNN and TestDailyPatternRNN together at a fixed post-processing threshold.