Joint Angle Coordinate Mismatch Between GR00T Training Data and Isaac Sim USD Model

[alex-luci]

I trained GR00T N1.5 on a SO100 robot using specific motor angle ranges (e.g., shoulder_lift: [0°, 180°], elbow_flex: [0°, 180°]), but the Isaac Sim USD model uses different coordinate systems and ranges (e.g., shoulder_lift: [-100°, 100°], elbow_flex: [-100°, 90°]). This causes incorrect robot behavior during inference, as angles from GR00T are directly applied to the simulation without coordinate transformation.

This is my dataset on which i trained GR00T: https://huggingface.co/datasets/VoicAndrei/so100_kitchen_hackathon

Current Behavior

When running GR00T inference in Isaac Sim, the robot exhibits incorrect joint behavior:

• Joints move in reversed directions (e.g., shoulder_lift extends instead of resting at initial position)
• Joints hit limits unexpectedly
• The initial pose in simulation doesn't match the initial pose GR00T was trained on
• Some joints don't move at all or only move in one direction

Root Cause

The GR00T model was trained on real robot motor angles with these ranges:

MOTOR LIMITS:

'shoulder_pan': (-120.0, 120.0),
'shoulder_lift': (0.0, 180.0),      # 180° = resting, 0° = fully extended
'elbow_flex': (0.0, 180.0),         # 180° = resting, 0° = fully extended
'wrist_flex': (-90.0, 90.0),        # 90° = resting, -90° = fully extended
'wrist_roll': (-165.0, 100.0),
'gripper': (0.0, 100.0),

However, the Isaac Sim USD model uses different coordinate systems:

USD_JOINT_LIMITS:

"shoulder_pan": (-110.0, 110.0),
"shoulder_lift": (-100.0, 100.0),   # -100° = resting
"elbow_flex": (-100.0, 90.0),       # 90° = resting
"wrist_flex": (-95.0, 95.0),
"wrist_roll": (-160.0, 160.0),
"gripper": (-10.0, 100.0),

Is there a way to map the angles that GR00T was trained on to the simulation? Or is there a way to change how the simulation controls the robot? I tried to create a mapping between these two and i went down a never-ending rabbit hole of adjustments, without success.

[EverNorif]

In fact, our code already includes a mapping for aligning the real leader with the simulated follower. You can refer to our implementation for alignment.

We use MotorNormMode.RANGE_M100_100 to get the joint position from the real leader, so its range is

SO101_FOLLOWER_MOTOR_LIMITS = {
    'shoulder_pan': (-100.0, 100.0),
    'shoulder_lift': (-100.0, 100.0),
    'elbow_flex': (-100.0, 100.0),
    'wrist_flex': (-100.0, 100.0),
    'wrist_roll': (-100.0, 100.0),
    'gripper': (0.0, 100.0),
}

The range of USD in the simulation is

SO101_FOLLOWER_USD_JOINT_LIMLITS = {
    "shoulder_pan": (-110.0, 110.0),
    "shoulder_lift": (-100.0, 100.0),
    "elbow_flex": (-100.0, 90.0),
    "wrist_flex": (-95.0, 95.0),
    "wrist_roll": (-160.0, 160.0),
    "gripper": (-10, 100.0),
}

We use a linear mapping to align the maximum and minimum values ​​of these two ranges.
https://github.com/LightwheelAI/leisaac/blob/main/source/leisaac/leisaac/devices/action_process.py#L93

Of course, this assumes that you align the states of both, for example, that their behavior is the same when both the real and simulated joints reach their maximum values.