Experiments with deep learning models and optimization.
A fundamental principle in machine learning and neuroscience is chaos dynamics, which hypothesizes an "edge of chaos" that enables optimal information flow and capacity. It balances the chaos (exploding gradients) with order (vanishing gradients) to support rich yet controllable systems. This can be naturally studied through the lense of Lyapunov exponents.
This repository hosts code to research this space.
- Setup virtual environment
python3 -m venv .venv
source .venv/bin/activate- Install dependencies
pip install --upgrade pip
pip install -r requirements.txt