Deception Detection in Dyadic Exchanges Using Multimodal Machine Learning

This repository contains the code accompanying the paper:

"Deception Detection in Dyadic Exchanges Using Multimodal Machine Learning: A Study on a Swedish Cohort"
Preprint available on arXiv.

The goal of this repository is to increase the reproducibility of our experiments and provide a reference implementation for future research on multimodal deception detection, particularly in dyadic contexts.

⚠️ Note: The original data (audio and video recordings) used in this study is protected and cannot be shared due to ethical and privacy restrictions. However, we share the derived feature-level data used in all analyses, consisting of CSV files extracted from the recordings using OpenFace and openSMILE.

---

🧠 Overview

The paper explores multimodal machine learning techniques to detect deception in dyadic interactions, using a unique Swedish-language dataset. It compares early vs. late fusion strategies and examines the impact of incorporating features from both participants in a conversation.

The codebase includes:

• Feature extraction and preprocessing
• Model training and evaluation
• Comparison of fusion strategies
• Scripts for reproducibility and persistent use

---

📁 Repository Structure

• notebooks/: Jupyter notebooks to run experiments interactively and explore results
• src/: Python modules for feature extraction, data loading, training, and evaluation
• requirements.txt: All required packages for running the project

---

⚙️ Setup

1. Clone the repository

git clone https://github.com/your-username/deception-detection-mm.git cd deception-detection-mm

2. Install dependencies

python -m venv venv source venv/bin/activate # or .\venv\Scripts\activate on Windows pip install -r requirements.txt

3. Usage

You can either:

🔬 Run the experiments interactively: Use the Jupyter notebooks to run and visualize the experiments step-by-step.

🧪 Run persistent scripts: Use the Python scripts to run full training/evaluation pipelines.

📖 Citation

If you use this code in your research, please cite the following paper:

Samuels, T., et al. (2025).
Deception Detection in Dyadic Exchanges Using Multimodal Machine Learning: A Study on a Swedish Cohort.
arXiv:2506.21429

@article{samuels2025deception,
  title={Deception detection in dyadic exchanges using multimodal machine learning: A study on a Swedish cohort},
  author={Samuels, Thomas Jack and Rugolon, Franco and Hau, Stephan and H{\"o}gman, Lennart},
  journal={arXiv preprint arXiv:2506.21429},
  year={2025}
}

Acknowledgments

This work was conducted as a collaboration between the Department of Psychology and the Department of Computer and Systems Sciences, Stockholm University. The project was made possible by funding from the Marcus and Amalia Wallenberg Memorial Foundation (grant MAW 2022.0062).