Skip to content

Complete redesign of AprilTag integration and support for ArUco families#1919

Open
fspindle wants to merge 18 commits intolagadic:masterfrom
fspindle:feat_apriltag_system
Open

Complete redesign of AprilTag integration and support for ArUco families#1919
fspindle wants to merge 18 commits intolagadic:masterfrom
fspindle:feat_apriltag_system

Conversation

@fspindle
Copy link
Contributor

@fspindle fspindle commented Feb 27, 2026

This PR provides a major overhaul of the Apriltag integration within ViSP. The goal is to modernize the build system, improve compatibility with system-wide installations, and extend the features available to the end-user (ArUco markers, Pose ambiguity handling).

1. Modernized Build System & Dependency Management

The AprilTag 3rd-party integration has been redesigned to support three flexible usage modes:

  • System Package: Automatically detects and uses the library installed via package managers (e.g., libapriltag-dev).
  • External Installation: Supports official Apriltag installations built from source outside of ViSP.
  • Embedded Build: Reliable fallback to the built-in version in 3rdparty/apriltag.
  • Configuration Verification: Added vp_verify_config() and ENABLE_CONFIG_VERIFICATION to ensure build options (USE_XXX) match actual detection results, preventing silent feature loss.

2. Enhanced Features & ArUco Support

  • ArUco Integration: Native support for ArUco tag families (4x4 to 7x7 and MIP) directly within vpDetectorAprilTag.
  • Extended API Support: Detects and exposes new capabilities from AprilTag 3.4+, including deep-copy utilities and the secondary pose solution for handling pose ambiguity.
  • Improved Robustness: Implemented version-aware internal wrappers (my_matd_destroy(), etc.) to handle memory layout changes in matd_t across different Apriltag versions and bypass non-exported symbols.

3. Cleanup & Maintenance

  • API Sanitization: Removed support for deprecated and non-functional families (36h10, 25h7, 36ARTOOLKIT).
  • Standardization: Renamed CMake options from WITH_APRILTAG to USE_APRILTAG to align with modern ViSP naming conventions.
  • Tutorials & Examples: Fully synchronized all robot examples (Franka, UR, Afma6, Flir-PTU) and tutorials with the new API and indexed tag families.

4. Remaining work

  • Submission of a PR to https://github.com/AprilRobotics integrating changes done in 3rdparty/apriltag
  • Synchronize 3rdparty/apriltag with the PR. Will be done in a next pull-request
  • Add GHA with libapriltag-dev package
  • When libapriltag-dev package is installed, check if 3rdparty/apriltag folder can be removed
  • Test build-in on WinRT
  • Investigate is pthread is needed for build-in
  • Test using conda apriltag package
  • Add pixi conda package in GHA

- useful to detect libapriltag 3.3.0 installed on ubuntu 24.04 with apt
- libapriltag 3.3.0 has a corrupted apriltagTargets.cmake file that makes find_package(apriltag)
  crashing
This commit refactors the AprilTag integration within ViSP and introduces
new CMake utilities for better dependency tracking and build robustness.

AprilTag Integration Improvements:
- Implement detection of 'Extended API' (copy functions and 2nd pose solution).
- Add support for ArUco tag families through header check and compile tests.
- Modernize version handling and property settings for the built-in 3rdparty.
- Add support for system-installed AprilTag with a fallback to built-in source.

CMake System Enhancements:
- Introduce VISP_OPTION and vp_verify_config() to ensure that requested
  build options (USE_XXX) match the actual detection results (HAVE_XXX).
- Add ENABLE_CONFIG_VERIFICATION option to fail the build on configuration
  mismatch, preventing "silent" feature loss.
- Update vpConfig.h template to export AprilTag version and feature flags
  (EXTENDED_API, ARUCO, BIG_FAMILY).
- Improve vp_get_interface_link_libraries to better handle imported
  targets and library locations on Windows/Unix.

Refactoring:
- Rename WITH_APRILTAG to USE_APRILTAG to follow modern ViSP naming conventions.
- Standardize big family exclusion logic.
- Fix various CMake list manipulation macros for better stability.
…stic wrappers

This commit updates the detection module to support the new Apriltag
features while maintaining compatibility with older or system-provided
versions of the library.

Key Changes:
- Robust Build Logic: Updated CMakeLists.txt to correctly handle public
  vs private dependencies depending on whether Apriltag is built from
  source or found on the system.
- Version-Agnostic Internal Wrappers: Added a 'my_' namespace in
  vpDetectorAprilTag.cpp containing wrappers for core AprilTag functions
  (matd, zarray, detections). This bypasses issues where some symbols
  (like matd_destroy) are not exported in certain binary distributions.
- Matd Allocation Fix: Implemented conditional allocation logic for
  matd_t structures to handle the memory layout change introduced in
  AprilTag 3.4.3 (contiguous vs separate data pointer).
- API Guarding: Used VISP_HAVE_APRILTAG_EXTENDED_API and
  VISP_HAVE_APRILTAG_ARUCO macros to conditionally enable/disable
  pose estimation methods (like Orthogonal Iteration) and ArUco families.
- Cleanup: Removed deprecated and non-functional tag families
  (TAG_36h10, TAG_36ARTOOLKIT, TAG_25h7) and synchronized Python bindings.
- Testing: Updated catchAprilTag and catchArUco tests with adjusted
  precision thresholds for newer AprilTag versions and added guards
  for ArUco-specific test cases.
This commit synchronizes the documentation and tutorials with the
recent AprilTag integration refactoring.

Changes:
- Doxygen Configuration: Added VISP_HAVE_APRILTAG_ARUCO and
  VISP_HAVE_APRILTAG_EXTENDED_API to the PREDEFINED macros in
  config-doxygen.in. This ensures that conditionally compiled
  code (like ArUco tags or new pose methods) is correctly
  documented in the generated API reference.
- Tutorial Updates:
    * Updated AprilTag detection tutorial to use the new naming
      convention for CMake variables (USE_APRILTAG_BIG_FAMILY
      instead of WITH_...).
    * Cleaned up tracking tutorials by removing deprecated tag
      families (36h10, 25h7, ARTOOLKIT) from the supported lists.
    * Re-indexed the tag family IDs in tutorial examples to match
      the updated vpAprilTagFamily enum, ensuring command-line
      examples remain accurate.
This commit synchronizes all ViSP code with the updated
vpDetectorAprilTag class and the new AprilTag 3.4+ capabilities.

Changes:
- Robust Pose Defaults: Updated calibration apps and servoing examples
  (AFMA6, Franka, UR, Flir-PTU) to use BEST_RESIDUAL_VIRTUAL_VS as the
  default pose estimation method when the AprilTag Extended API is
  not available.
- UI & CLI Cleanup:
    * Removed deprecated tag families (36h10, 25h7, ARTOOLKIT)
      from help menus (usage()) and command-line parsing across all
      C++ tutorials.
    * Re-aligned tag family indices to match the new enum sequence.
    * Wrapped ArUco-specific help strings with VISP_HAVE_APRILTAG_ARUCO guards.
- Mobile & Java Synchronization:
    * Updated the Android CameraPreview sample and Java tutorials to
      reflect the corrected tag family indices and added compatibility
      notes for Homography-based methods.
- Bug Fixes in Tutorials: Adjusted epsilon thresholds and improved
  variable naming (poseEstimationMethod -> pose_estimation_method)
  for better coding style consistency.
@s-trinh
Copy link
Contributor

s-trinh commented Feb 28, 2026

When comparing AprilRobotics/apriltag PR and ViSP/AprilTag code, if relevant, these files have some discrepancies compared to the AprilTag PR code:

Most likely related to MinGW support, but maybe no more relevant now.


With these recent changes, USE_APRILTAG_BIG_FAMILY should no more be needed. The memory usage has been drastically reduced.
I have not tested these changes nor benchmarked the impact, if any, on the computation.
If correct, I believe USE_APRILTAG_BIG_FAMILY should be forced to TRUE/ignored if the correct AprilTag version is detected.


In the case of the official AprilRobotics/apriltag declining the responsability to maintain the ArUco changes, this would lead to a main difference between ViSP and AprilRobotics AprilTag version.
What would be the default order of priority between:

  • System lib AprilTag / git clone AprilRobotics/apriltag
  • ViSP built-in AprilTag

when building ViSP? And in general (I guess there should be no difference when speaking about performance between system lib and built-in)?

If the PR is accepted, I guess there is no other fundamental difference between ViSP and AprilRobotics version?


In the future, the GHA should be updated to add the support of the AprilRobotics/apriltag version.


On another topic, what would be the correct approach when dealing with the planar pose ambiguity? Does a trivial approach such as a vector of the 30 recently estimated rotations (with solutions from the lowest reprojection error) and choose for the next frame the one corresponding to the "consensus rotation"?
Or would something more elaborated mathematically speaking be needed, and could be interesting as a feature in ViSP.
Note that it is only when the tag appears small in the image this ambiguity arises, which is not problematical with the general usage.

@codecov
Copy link

codecov bot commented Feb 28, 2026

Codecov Report

❌ Patch coverage is 71.87500% with 36 lines in your changes missing coverage. Please review.
✅ Project coverage is 48.45%. Comparing base (b03c98f) to head (1adacae).
⚠️ Report is 3 commits behind head on master.

Files with missing lines Patch % Lines
modules/detection/src/tag/vpDetectorAprilTag.cpp 72.44% 21 Missing and 14 partials ⚠️
...ction/include/visp3/detection/vpDetectorAprilTag.h 0.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1919      +/-   ##
==========================================
- Coverage   48.68%   48.45%   -0.24%     
==========================================
  Files         532      532              
  Lines       69293    69289       -4     
  Branches    32403    32406       +3     
==========================================
- Hits        33735    33573     -162     
- Misses      24976    25027      +51     
- Partials    10582    10689     +107     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants