[FEATURE] Add public API to Scene for drawing cameras frustum and trajectories.

[Mehak261124]

Description

Added two new debug visualization methods to scene.py:

• draw_debug_frustum(camera): Visualizes the frustum of any camera including sensor cameras, by reusing the existing create_camera_frustum() from utils/mesh.py. Previously this was only available for the interactive viewer camera via show_cameras=True in vis_options.

• draw_debug_trajectory(positions): Draws a trajectory as a series of connected lines by connecting a list of 3D positions. Useful for visualizing robot paths and motion history.

Both methods follow the same pattern as existing draw_debug_* methods and render as markers — visible in the interactive viewer but invisible to robot cameras (depth, segmentation, RGB renders are unaffected).

Related Issue

Resolves #1049

Motivation and Context

draw_debug_frustum reuses the existing create_camera_frustum() from utils/mesh.py which was previously only used internally for the viewer camera. This change makes frustum visualization available for any camera including sensor cameras, without duplicating code.

draw_debug_trajectory was missing entirely from the debug visualization API and is a commonly needed tool for visualizing robot paths during development and debugging.

How Has This Been / Can This Be Tested?

Tested manually on macOS Apple Silicon (M2), Python 3.11.

Test draw_debug_frustum:

import genesis as gs

gs.init(backend=gs.cpu)
scene = gs.Scene(show_viewer=True)
scene.add_entity(gs.morphs.Plane())
cam = scene.add_camera(res=(640, 480), pos=(3.5, 0.0, 2.5), lookat=(0, 0, 0.5), fov=30, GUI=False)
scene.build()
scene.draw_debug_frustum(cam, color=(0.0, 1.0, 0.0, 0.3))
while scene.viewer.is_alive():
    scene.step()

Test draw_debug_trajectory:

import genesis as gs
import numpy as np

gs.init(backend=gs.cpu)
scene = gs.Scene(show_viewer=True)
scene.add_entity(gs.morphs.Plane())
scene.build()
t = np.linspace(0, 2 * np.pi, 50)
positions = np.column_stack([np.cos(t), np.sin(t), np.ones_like(t) * 0.5])
scene.draw_debug_trajectory(positions, radius=0.005, color=(1.0, 0.5, 0.0, 1.0))
while scene.viewer.is_alive():
    scene.step()

Screenshots:

Checklist:

• I read the CONTRIBUTING document
• I followed the Submitting Code Changes section
• I tagged the title correctly
• I updated the documentation accordingly
• I tested my changes and added instructions on how to test it for reviewers

[duburcqa]

Remove this.

[duburcqa]

All PRs adding features or fixing bugs must come with accompanying unit tests. In this case you have no other choice than resorting on screenshot-based validation.

[Mehak261124]

Hey @duburcqa, I have addressed your feedback and added the screenshot-based unit tests for draw_debug_frustum and draw_debug_trajectory. Could you take a look?

[duburcqa]

Hey @duburcqa, I have addressed your feedback and added the screenshot-based unit tests for draw_debug_frustum and draw_debug_trajectory. Could you take a look?

Look at tests/test_render.py. The corresponding fixture is png_snapshot. One of the most relevant example is probably test_sensors_draw_debug. See snippet:

    pyrender_viewer = scene.visualizer.viewer._pyrender_viewer
    assert pyrender_viewer.is_active
    rgb_arr, *_ = pyrender_viewer.render_offscreen(
        pyrender_viewer._camera_node,
        pyrender_viewer._renderer,
        rgb=True,
        depth=False,
        seg=False,
        normal=False,
    )

    if sys.platform == "darwin":
        glinfo = pyrender_viewer.context.get_info()
        renderer = glinfo.get_renderer()
        if renderer == "Apple Software Renderer":
            pytest.xfail("Tile ground colors are altered on Apple Software Renderer.")

    assert rgb_array_to_png_bytes(rgb_arr) == png_snapshot

[duburcqa]

The procedure to quite painful unfortunately, but there is no way around if you are planning to working on rendering features for quite some time. It is based on a thin custom wrapper around pytest-snapshot to support pixel-matching validation:

• Add assertion in your unit test: assert rgb_array_to_png_bytes(rgb_arr) == png_snapshot
• Run your unit test in isolation, with distributed framework disabled, and snapshot update enabled: pytest -xsv -n 0 --snapshot-update tests/test_render.py::test_rasterizer_env_separate
• Fork our internal HuggingFace-hosted Git repository snapshots that is used to manage snapshots.
• Copy the newly created snapshots that are stored under tests/__snapshots__ in your local clone of snapshots repository, commit them and push a branch on your fork.
• Open a Pull Request on HuggingFace and ping me, on this PR so that I can review and merge the HuggingFace. Note that I will only merge once every comments in Genesis' PR is approved for merging.
• Update HUGGINGFACE_SNAPSHOT_REVISION hash that is hardcoded in tests/utils.py to update the commit being used for downloading snapshots.
• If Genesis' CI pass we are good to merge.

[duburcqa]

By the way, the title of the PRs must focus on what the end user should be aware of. Not implementation details. In this case, add scene helper methods for drawing cameras frustum and trajectories.

[github-actions]

🔴 Benchmark Regression Detected ➡️ Report

[duburcqa]

Ignore performance regression report. They are a bit unstable at the moment unfortunately.

[Mehak261124]

Hi @duburcqa, I've uploaded the snapshot to my HuggingFace fork.

Here's the commit: https://huggingface.co/datasets/Mehak261124/snapshots/commit/c947da0356cb88838242a484d8763086bf50a35c

Please review and merge into Genesis-Intelligence/snapshots when ready.
I'll update HUGGINGFACE_SNAPSHOT_REVISION in tests/utils.py once it's merged.

[duburcqa]

This snapshot is not right. The camera frustum is not visible because the camera is misplaced. It may be necessary to do at least one simulation step to avoid camera artefacts.

You should already update the PR with corresponding png_snapshot.

[Mehak261124]

Hey @duburcqa, I've addressed all your feedback:

• Fixed camera placement so the frustum is now clearly visible in the snapshot
• Added scene.step() to avoid camera artifacts
• Merged the duplicate test into a single one using the png_snapshot fixture
• Updated the snapshot on my HuggingFace fork: https://huggingface.co/datasets/Mehak261124/snapshots/tree/add-debug-visualization-snapshot
• Updated HUGGINGFACE_SNAPSHOT_REVISION in tests/utils.py

Could you please review and merge the HuggingFace snapshot PR?

[duburcqa]

By the way, do not forget to remove doc update.

[duburcqa]

Can you add an object in the scene inside the camera frustum to make sure it renders transparent?

[Mehak261124]

Hi @duburcqa, I've addressed all your review comments:

• Restored test_sensors_draw_debug as a separate unchanged test. test_draw_debug_frustum_and_trajectory is now a separate independent test as suggested.
• Restored the original "Flaky on Apple Software Renderer." message in test_sensors_draw_debug and removed the "Tile ground colors are altered" change.
• Added # FIXME: comment on the double viewer.update() call explaining why two calls are necessary.
• Restored all original comments (# Force screen-independent..., # Enable running in background thread..., # Disable shadows systematically...).
• Added a semi-transparent box inside the sensor camera frustum to verify transparency rendering.
• Removed the doc submodule update.
• Updated HuggingFace snapshot revision to f4edfa66608b016457350aa2e97e4cdffec469ff.