A Python library for real-time audio time-scale modification using advanced DSP techniques including Phase Vocoder (PV) and Overlap-Add (OLA) methods.
- Real-time audio processing with interactive controls
- Multiple time-stretching algorithms:
- Phase Vocoder (PV) with baseline and lookup methods
- Overlap-Add (OLA) processing
- Hybrid approaches combining PV and OLA
- Harmonic/Percussive source separation using librosa
- Interactive keyboard controls for real-time tempo adjustment
- Modular design with reusable TSM functions
- Cross-platform support (Windows, macOS, Linux)
For full audio playback functionality, you must install the PortAudio library separately:
System-specific PortAudio installation:
- macOS:
brew install portaudio - Ubuntu/Debian:
sudo apt-get install portaudio19-dev - Windows: Usually included with PyAudio wheels
then,
# Install audio dependencies
pip install tsm-realtimegit clone https://github.com/HMC-MIR/TSMRealTime.git
cd TSMRealTime
# See system-specific PortAudio installations
pip install -e - Install system audio libraries (see Installation section above)
import tsm_realtime
# Create TSM processor instance
tsm = tsm_realtime.TSMRealTime()
# Play audio with real-time controls
tsm.play_hps_full("path/to/your/audio.wav")When running the audio processing, use these keyboard controls:
- ↑ (Up Arrow): Increase time-stretch factor (alpha)
- ↓ (Down Arrow): Decrease time-stretch factor (alpha)
- Ctrl+C: Stop playback
# Use lookup-based method for better performance
tsm.play_hps_lookup("audio.wav", beta=0.25)
# The beta parameter controls the overlap factor for lookup analysis
# Lower values (0.1-0.3) provide better quality
# Higher values (0.4-0.8) provide faster performanceplay_hps_full(filename): Play audio using hybrid baseline methodplay_hps_lookup(filename, beta=0.25): Play audio using hybrid lookup methodgenerate_lookup(beta, xh): Generate lookup tables for efficient processingphase_vocoder_full(xh, Ha_PV, prev_phase): Complete phase vocoder analysisphase_vocoder_lookup(...): Phase vocoder using precomputed tablesola_process(xp, Ha_ola): Overlap-Add processing for percussive components
alpha: Time-stretch factor (1.0 = normal speed, >1.0 = faster, <1.0 = slower)beta: Overlap factor for lookup analysis (default: 0.25)sr: Sampling rate (default: 22050 Hz)
- Python: 3.8 or higher
- Local machine: Cannot run in headless environments (Colab, remote servers without display)
- Audio system: Requires PortAudio for audio I/O (system-specific installation)
Core dependencies (included automatically):
numpy>=1.20.0scipy>=1.7.0librosa>=0.9.0pydub>=0.25.0pynput>=1.8.1(for keyboard controls)pyaudio>=0.2.11(requires PortAudio system library)
PortAudio cannot be bundled with the package due to:
- System-specific compilation: PortAudio must be compiled for each operating system
- Hardware dependencies: Audio drivers vary by machine
- Platform-specific libraries: Different audio backends (ALSA, CoreAudio, DirectSound)
- Processes harmonic components of audio
- Maintains phase continuity for natural sound
- Supports both real-time and lookup-based processing
- Handles percussive components efficiently
- Provides good quality for transient sounds
- Lower computational complexity
- Combines PV for harmonics and OLA for percussives
- Achieves optimal balance of quality and performance
- Automatic source separation using median filtering
- Lookup method: Faster processing, affects the audio quality
- Full method: Higher quality, more computational overhead
- Real-time performance: Optimized for interactive use
- Memory usage: Moderate, depends on audio length and beta parameter
import tsm_realtime
# Initialize processor
tsm = tsm_realtime.TSMRealTime()
# Play with real-time tempo control
tsm.play_hps_full("sample.wav")# Use lookup method for better performance
tsm.play_hps_lookup("sample.wav", beta=0.2)import librosa
import tsm_realtime
# Load and preprocess audio
x, sr = librosa.load("sample.wav", mono=True, sr=22050)
tsm = tsm_realtime.TSMRealTime()
# Generate lookup tables for efficient processing
xh, xp = tsm._harmonic_percussive_separation(x, sr)
S_phase, S_mag, w_if, Ha_lookup = tsm.generate_lookup(0.25, xh)
# Process with custom parameters
# ... (advanced usage)Import fails in headless environments (Colab, remote servers):
ImportError: this platform is not supported: failed to acquire X connection
Solution: This package requires a local machine with display capabilities. Cannot run in:
- Google Colab
- Remote servers without X11
- Docker containers without display forwarding
PyAudio installation fails:
# macOS
brew install portaudio
pip install tsm-realtime[audio]
# Ubuntu/Debian
sudo apt-get install portaudio19-dev
pip install tsm-realtime[audio]
# Windows
pip install tsm-realtime[audio]Audio playback issues:
- Ensure audio drivers are properly installed
- Check that the audio file format is supported
- Verify PortAudio is correctly installed
- Try different audio backends if available
Performance issues:
- Use lookup method (
play_hps_lookup) for better performance - Reduce
betaparameter for faster processing - Ensure sufficient system resources
We welcome contributions! Please see our GitHub repository for:
- Issue reporting
- Feature requests
- Pull requests
- Development guidelines
If you use this library in your research, please cite:
@software{tsm_realtime,
title={TSM Real-Time: Real-time Audio Time-Scale Modification},
author={Lubis, Sayema and Peng, Clark and Carreno, Jared},
year={2025},
url={https://github.com/HMC-MIR/TSMRealTime}
}This project is licensed under the MIT License - see the LICENSE.md file for details.
- Sayema Lubis - Development and Initial work
- Clark Peng - Development
- Jared Carreno - Initial work
- TJ Tsai - Project Advisor
- Built on top of excellent libraries: NumPy, SciPy, librosa, and PyAudio
- Inspired by classic DSP research in time-scale modification
- Developed at Harvey Mudd College (HMC)
For more information, visit our GitHub repository or PyPI page.