feat: add GestureMapper abstraction and gesture implementations#3
Open
iliasmahboub wants to merge 2 commits intom2b3:mainfrom
Open
feat: add GestureMapper abstraction and gesture implementations#3iliasmahboub wants to merge 2 commits intom2b3:mainfrom
iliasmahboub wants to merge 2 commits intom2b3:mainfrom
Conversation
…pers Decouples gesture detection from the consumer process by introducing a GestureMapper ABC (configure/process/reset) that parallels the existing VideoInput pattern. Extracts the hardcoded tap logic into TapMapper and adds PinchMapper (thumb-index distance) and VelocityMapper (hand speed tracking) as new gesture types. Includes a generic gesture_consumer drop-in for latency_mp.py, a webcam preview script for FLIR-free testing, and 14 unit tests (first in this repo) covering all three mappers.
CompositeMapper runs multiple gestures simultaneously on the same hand data — the main payoff of the abstraction (can't do this with hardcoded approach). Two PD patches demonstrate pinch->synth and velocity->noise filter. Equivalence test replays hand sequences through both TapMapper and the original latency_mp.py consumer logic to prove identical output. 19 tests total, all passing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Decouples gesture detection from the consumer process by introducing a
GestureMapperABC that parallels the existingVideoInputpattern. Enables running multiple gestures simultaneously — something you can't do with the hardcoded tap logic.New files
Gesture framework:
gestures/gesture_mapper.py— ABC withconfigure(),process(),reset()gestures/tap_mapper.py— Extracts tap detection fromlatency_mp.py(same state machine, same landmarks, same threshold)gestures/pinch_mapper.py— Thumb-index distance → continuous/pinch/distance+ binary trigger/releasegestures/velocity_mapper.py— Hand speed → continuous/velocity/speed+/velocity/swipewith hysteresisgestures/composite_mapper.py— Runs multiple mappers on the same hand data, merges OSC outputPipeline integration:
latency_measurement/gesture_consumer.py— Drop-in consumer replacement that accepts anyGestureMapperpreview_gestures.py— Webcam preview for testing without FLIR:python preview_gestures.py --gesture pinchPureData patches:
puredata/pinch_synth.pd— Pinch distance → oscillator frequency (200–1000Hz), trigger/release → volume envelopepuredata/velocity_noise.pd— Hand speed → lowpass filter cutoff (100–5000Hz) on noise, swipe → visual bangTests (19 total, first in this repo):
tests/test_gesture_mappers.py— 16 unit tests for all 4 mappers including CompositeMappertests/test_tap_equivalence.py— Replays hand sequences through both TapMapper and the originallatency_mp.pyconsumer logic, asserts identical trigger behavior12 new files, 0 existing files modified.
How this connects to the README's future work
The README proposes separating MediaPipe workers from OSC signal production: "A separate process that takes landmarks from the worker pool, maps them to gestures, and sends OSC signals." The
GestureMapperinterface is the mapping layer that architecture needs —gesture_consumer.pyis the consumer side, and the mapper implementations handle the landmark→gesture→OSC translation.Test plan
python -m pytest tests/ -v— 19/19 passing in <0.3stest_tap_equivalence.pyproves TapMapper == original latency_mp.py logicpython preview_gestures.py --gesture tap --camera 0python preview_gestures.py --gesture pinch --camera 0python preview_gestures.py --gesture velocity --camera 0puredata/pinch_synth.pdin PD, run preview with--gesture pinchpuredata/velocity_noise.pdin PD, run preview with--gesture velocity