Skip to content

Refactor face tracking control pipeline for smoother arbitration#7

Merged
Sa1koro merged 7 commits intomainfrom
cursor/-bc-9423ef01-f95d-4c6c-a830-8ff169c47f81-63c9
Mar 25, 2026
Merged

Refactor face tracking control pipeline for smoother arbitration#7
Sa1koro merged 7 commits intomainfrom
cursor/-bc-9423ef01-f95d-4c6c-a830-8ff169c47f81-63c9

Conversation

@Sa1koro
Copy link
Copy Markdown
Member

@Sa1koro Sa1koro commented Mar 23, 2026

Summary

  • Reworked esp32_firmware_refactored/face_tracking control core to reduce servo twitch and improve reliability:
    • STA-only WiFi flow (no AP fallback)
    • automatic reconnect loop + serial manual retry (wifi retry <n>)
    • tracking/manual control-mode arbitration with smooth blend transitions
    • per-servo-pair targets with bounded smoothing and max-step limiting
    • configurable angle limits with default pan/tilt range 20..160
    • new OSC/serial controls for /track/mode, /track/limits, and updated diagnostics
  • Upgraded Python host face tracking pipeline:
    • multi-face area-weighted centroid output for tracking target
    • preserved primary face output for UI diagnostics
    • integrated control-priority API (/api/control/mode) for face_tracking vs emotion_manual
  • Enhanced web panel Face Tracking node:
    • added XY 2D drive pad for servo pairs
    • added multi-select checkboxes for pairs 1..4 and concurrent group control
    • added live editable pan/tilt limit inputs and apply-to-node action
    • added control-priority mode toggle buttons
    • completed missing tracking/serial helper actions in panel JS

Verification

  • python3 -m compileall python_host
  • node -e "...new Function(script)..." inline JS syntax check for python_host/ui/templates/index.html

Notes

  • pytest is not available in this environment (No module named pytest), so automated test suite execution was not run.
Open in Web Open in Cursor 

cursoragent and others added 6 commits March 23, 2026 08:06
Co-authored-by: Huanrui Cao <kaminodice@outlook.com>
Co-authored-by: Huanrui Cao <kaminodice@outlook.com>
- Added configurable retry attempts and delays for WiFi connection.
- Implemented functions for automatic and manual WiFi reconnection.
- Enhanced mDNS setup to ensure service availability upon WiFi connection.
- Updated serial command handling to include WiFi status and manual retry options.
- Refactored network setup functions for improved clarity and functionality.
…n features

- Added gaze limit parameters for X and Y axes to control eye movement range.
- Implemented pupil auto-spin functionality for dynamic pupil movement based on eye state.
- Updated eye state structure and related functions to accommodate new features.
- Enhanced OSC message routing for gaze limits and pupil auto-spin control.
- Improved UI elements for setting gaze limits and toggling pupil auto-spin.
- Introduced a camera mirror toggle in the UI to preview mirrored video feed.
- Added an option to invert X tracking for enhanced user control over face tracking.
- Updated the CoordinatePublisher to support invert_x configuration.
- Enhanced API tracking configuration to include invert_x parameter.
- Improved UI elements for emotion model selection and tracking state display.
@vercel
Copy link
Copy Markdown

vercel Bot commented Mar 25, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
v0-react-esp-32-controller Error Error Mar 25, 2026 6:21pm

- Added functionality to clear OSC history via a new API endpoint and corresponding UI button.
- Implemented history capacity tracking and limit settings for OSC messages.
- Enhanced OSC sender to include additional metadata (reason and source) in history logs.
- Updated UI to allow users to set history limits and enable auto-refresh for the OSC log.
- Improved tests to cover new history management features and ensure proper functionality.
@Sa1koro Sa1koro marked this pull request as ready for review March 25, 2026 18:22
@Sa1koro Sa1koro merged commit 5eb9419 into main Mar 25, 2026
6 of 8 checks passed
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

servo1.write(pos);
int pupilX = map(pos, 0, 180, 110, 130);
drawPupil(pupilX);
delay(2);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Servo sweep loops are asymmetric and comments wrong

Medium Severity

The first sweep comment says "sweep 0 → 180" but the loop only iterates to 120. The second sweep comment says "sweep 180 → 0" but starts at 100. This causes a hard jump from servo position 120 to 100 between sweeps. Additionally, map(pos, 0, 180, 110, 130) uses an input range of 0–180, but pos only reaches 120 max, so pupilX never reaches 130 — the pupil eye animation won't use its full travel range.

Fix in Cursor Fix in Web

3: { pan: 90, tilt: 90 },
4: { pan: 90, tilt: 90 },
};
let faceAngleLimits = { pan_min: 20, pan_max: 160, tilt_min: 20, tilt_max: 160 };
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tilt limit defaults mismatch between firmware and panel

Medium Severity

The JS panel defaults faceAngleLimits to tilt_min: 20, tilt_max: 160, but the firmware uses TILT_MIN_DEFAULT = 50 and TILT_MAX_DEFAULT = 180. When a user clicks "Apply Limits to Node" without editing values, the panel sends 20/160 to a node initialized at 50/180, causing an unexpected range change. The PR description states the intended default is 20..160, suggesting the firmware tilt constants are wrong.

Additional Locations (1)
Fix in Cursor Fix in Web

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants