Yolo Touch

YOLO Pose Detection component for TouchDesigner. Real-time human pose estimation with skeleton overlay, optimized for Apple Silicon (MPS) with CUDA fallback.

Features

• Real-time multi-person pose detection using Ultralytics YOLO pose models
• 17-keypoint COCO skeleton with built-in rendering
• Keypoint smoothing via moving average
• Structured keypoint data output (dict and Table DAT formats)
• Metal Performance Shaders (MPS) acceleration on Apple Silicon, with CUDA and CPU fallbacks
• Performance monitoring with FPS and inference time logging

Requirements

• Python 3.11
• uv package manager
• TouchDesigner (tested with the included .tox component)

Setup

# Install dependencies into a local venv
make install

# Verify installation
make test

The Makefile will create a .venv with Python 3.11 and install ultralytics, torch, torchvision, opencv-python, and numpy.

After install, the output will show the venv site-packages path and project path — use these to configure TouchDesigner's Python paths.

Usage

1. Open TouchDesigner and import yolo_pose_component.tox.
2. Point TouchDesigner's Python path to the .venv/lib/python3.11/site-packages directory.
3. The script auto-selects the best available device (MPS > CUDA > CPU).

API

process_frame(bgr_frame, draw_skeleton=True, confidence=0.3) Process a BGR frame and return the rendered output with skeleton overlay.

get_keypoints_data() Returns a list of dicts with per-person keypoint coordinates and confidence values.

get_keypoints_table() Returns keypoint data as a list of lists suitable for a TouchDesigner Table DAT.

Makefile Commands

Command	Description
make install	Create venv and install packages
make test	Verify all dependencies load
make clean	Remove the virtual environment
make all	Install and test