New Embodiment : support for Agibot-X2#112
Open
devsolotech wants to merge 5 commits intoNVlabs:mainfrom
Open
Conversation
Made-with: Cursor
- URDF, MJCF, and STL meshes under robot_description/ - Robot config with actuator groups, joint limits from vendor specs, and IsaacLab/MuJoCo index mappings (round-trip verified) - X2UltraConverter in order_converter.py - Registered in robot_mapping and experiment config (sonic_x2_ultra.yaml) - KP/KD use H2 motor constants as initial estimates pending vendor data Made-with: Cursor
Generalize SONIC codebase to support robots beyond Unitree G1 (29 DOF): - Refactor convert_soma_csv_to_motion_lib.py for multi-robot support with --robot flag, per-robot DOF axis/joint constants, and MJCF selection - Add factory functions to order_converter.py (get_converter_for_skeleton, get_converter_for_mjcf) and move shared properties to base class - Replace hardcoded G1 DOF check in motion_lib_base.py with dynamic num_dof - Use dynamic converter selection in commands.py and token_losses.py instead of hardcoded G1Converter - Add X2 Ultra experiment config with correct body names and motion file - Add X2-specific smoke test configs: tokenizer (unitoken_x2_teleop_noz), actor-critic (x2_teleop_mlp_v1), aux losses (x2_recon_and_teleop_latent), and experiment YAML (sonic_x2_ultra_smoke) for 2-encoder training without SMPL data Made-with: Cursor
…versal
Isaac Lab sorts sibling bodies alphabetically within each BFS level, so
head_yaw_link ("h") precedes left/right_shoulder_pitch_link ("l"/"r") at
the same depth. The previous mapping arrays assumed URDF XML order, which
caused head and shoulder DOFs to be cross-contaminated — arms appeared
bent inward while legs and waist looked correct.
Also documents the alphabetical BFS pitfall in conventions.md and
new_embodiments.md to prevent the same issue when onboarding future robots.
Made-with: Cursor
eval_x2_mujoco.py runs a trained SONIC checkpoint in MuJoCo with Reference State Initialization, IsaacLab-faithful proprioception (term order from PolicyCfg dataclass, history primed via broadcast- fill), correct tokenizer obs in body frame, FSQ quantization in the actor forward path, and auto-reset on fall. dump_isaaclab_step0.py captures IsaacLab's step-0 obs/encoder/FSQ/decoder/action for side-by-side debugging. The biggest sim-to-sim bug fixed during bring-up: MuJoCo free-joint qvel[3:6] is body-local, not world. Both the RSI write and the per-step base_ang_vel read had to be corrected to stop double- rotating the angular velocity, which had been silently corrupting the proprioception every step. docs/source/user_guide/sim2sim_mujoco.md catalogs ten such IsaacLab vs. MuJoCo gotchas with symptoms, debugging recipe, and a symptom-to-cause table so the next embodiment doesn't have to rediscover them. Made-with: Cursor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add Agibot X2 Ultra (31 DOF) Embodiment to GEAR-SONIC
This PR integrates the Agibot X2 Ultra humanoid (31 DOF) into the GEAR-SONIC whole-body control pipeline, covering the full path from robot description to SONIC training to MuJoCo sim-to-sim deployment.
What's included
Robot description
robot_description/x2_ultra.py) with actuator groups, joint limits from vendor specs, and IsaacLab/MuJoCo index mappings (round-trip verified)X2UltraConverterinorder_converter.pyfor joint-order translation between URDF, IsaacLab, and MuJoCo conventionsSONIC training pipeline
convert_soma_csv_to_motion_lib.pywith--robotflag and per-robot DOF axis/joint constantsget_converter_for_skeleton,get_converter_for_mjcf) toorder_converter.py; replaced hardcoded G1 DOF checks with dynamicnum_dofsonic_x2_ultra.yaml), smoke test (sonic_x2_ultra_smoke.yaml), tokenizer, actor-critic, and aux loss configsJoint ordering fix
head_yaw_link("h") precedesleft/right_shoulder_pitch_link("l"/"r") at the same depth. The previous mapping assumed URDF XML order, causing head and shoulder DOFs to be cross-contaminated.conventions.mdandnew_embodiments.mdMuJoCo sim-to-sim deployment
eval_x2_mujoco.py-- runs a trained SONIC checkpoint in MuJoCo with Reference State Initialization, IsaacLab-faithful proprioception, FSQ quantization, and auto-reset on falldump_isaaclab_step0.py-- captures IsaacLab's step-0 obs/encoder/FSQ/decoder/action for side-by-side debuggingqvel[3:6]is body-local (not world), which was silently double-rotating angular velocity and corrupting proprioception every stepdocs/source/user_guide/sim2sim_mujoco.md-- documents 10 IsaacLab-vs-MuJoCo gotchas with symptoms, debugging recipes, and a symptom-to-cause tableSim-to-sim deployment guide
Added
docs/source/user_guide/sim2sim_mujoco.md-- a comprehensive guide for deploying trained SONIC checkpoints from IsaacLab to MuJoCo. Documents 10 IsaacLab-vs-MuJoCo gotchas discovered during this bring-up, 8 of which correspond to real bugs fixed in the code. Includes a symptom-to-cause lookup table and a step-by-step debugging recipe usingdump_isaaclab_step0.py.Results