[BUG]The observation in the mujoco wrapper is not updated in the correct frequency, resulting bad sim2sim performance.

[EatingHe]

The sim2sim performance deteriorates if we reduce the passive damping parameter in the mujoco robot config to a real world value, like 0.01.

https://github.com/NVlabs/HOVER/blob/c14bb7e56a691da7f1f906997d966567ef138108/neural_wbc/data/data/mujoco/models/h1.xml#L8C2-L8C42

The reason comes from a bug in the mujoco environment wrapper. Event though the PD control loop in running at 200HZ, the actual observation is updated at 50HZ.

https://github.com/NVlabs/HOVER/blob/c14bb7e56a691da7f1f906997d966567ef138108/neural_wbc/inference_env/inference_env/neural_wbc_env.py#L252C1-L263C1

Add the following line at the end of the pd control loop solves this problem.

        self._pre_physics_step(actions)

        for _ in range(self.cfg.decimation):
            self._apply_action()

            # Convert it to numpy and apply it to the simulator.
            processed_action_np = self._processed_action.detach().cpu().numpy()
            self.robot.step(processed_action_np)
            self.robot.update({})

        # Forward the current episode step buffer to keep track of current number of steps into the motion reference.
        self.episode_length_buf += 1