You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
After this ships, pip install strands-robots[sim] is the fastest path from zero to a robot arm grasping a cube in MuJoCo. GR00T N1.7 support lands in the same release so real-hardware users get the latest Cosmos-Reason2-2B backbone.
What the user gets
# ═══════════════════════════════════════════════════════════════# 1. One-line sim or real — auto-detected# ═══════════════════════════════════════════════════════════════fromstrands_robotsimportRobot, list_robotsfromstrandsimportAgentrobot=Robot("so100") # USB detected → real; else → simagent=Agent(tools=[robot])
agent("Pick up the red cube") # ← the 5-line promise# Explicit mode overridesim=Robot("so100", mode="sim")
hw=Robot("so100", mode="real", cameras={"wrist": {"index_or_path": "/dev/video0"}})
# Discovery — 68 robots registered (was 38 in v0.3.8)list_robots(mode="sim")
# ═══════════════════════════════════════════════════════════════# 2. Simulation is a standalone AgentTool — no Robot needed# ═══════════════════════════════════════════════════════════════fromstrands_robots.simulationimportSimulationsim=Simulation() # MuJoCo backend, lazy-importedagent=Agent(tools=[sim])
agent("Create a world with an so100 and a red cube, then step 100 times")
# 35 actions exposed through the AgentTool interface:# create_world, step, reset, render, run_policy, record_dataset,# raycast, jacobian, energy, apply_force, save_checkpoint, …# ═══════════════════════════════════════════════════════════════# 3. Pluggable physics backends via SimEngine ABC# ═══════════════════════════════════════════════════════════════fromstrands_robots.simulationimportcreate_simulation, register_backend# Bring your own physics engine — as long as it implements SimEngineregister_backend("bullet", lambda: BulletSim, aliases=["pb"])
sim=create_simulation("bullet")
# ═══════════════════════════════════════════════════════════════# 4. GR00T N1.7 — new Cosmos-Reason2-2B backbone# ═══════════════════════════════════════════════════════════════robot=Robot(
"unitree_g1",
policy={"name": "gr00t", "server": "thor.local:5555"},
embodiment="REAL_G1", # new N1.7 embodiment tag
)
# Detection order: N1.7 → N1.6 → N1.5 (newest-first)# Verified live against nvidia/GR00T-N1.7-3B on Jetson AGX Thor.
The big one. Three headline features land together in v0.4.0:
Robot Mesh (#98) — every Robot() auto-joins a Zenoh peer network. Agent-to-agent tool calls over a LAN with zero config.
Newton backend (#96) — NewtonSimulation(SimEngine) on NVIDIA Warp + Newton 1.x. GPU-native, 4096+ envs per GPU, differentiable.
Isaac Sim backend (#97) — IsaacSimulation(SimEngine) with USD + IsaacLab 3.0. Photorealistic, Replicator synthetic data.
Together these turn strands-robots from "run one robot in MuJoCo" into "run a fleet across sim + real + photoreal, all coordinated through a mesh."
What the user gets
# ═══════════════════════════════════════════════════════════════# 1. Every Robot is automatically on the mesh — #98# ═══════════════════════════════════════════════════════════════fromstrands_robotsimportRobotstriker=Robot("unitree_g1", peer_id="striker")
mid_1=Robot("unitree_g1", peer_id="mid_1")
# Mesh auto-starts in __init__. Verify:assertstriker.mesh.aliveassertstriker.mesh.peer_id=="striker"# Peer discovery (local network multicast, no broker, no config)print(striker.mesh.peers)
# → [{'peer_id': 'mid_1', 'peer_type': 'robot', 'hostname': '…',# 'caps': {…}, 'last_seen': 1716…}]# ═══════════════════════════════════════════════════════════════# 2. Agent-to-agent tool calls — #98# ═══════════════════════════════════════════════════════════════# Natural language instruction — peer's agent interprets and executesresp=striker.mesh.tell("mid_1", "pass the ball to me")
# → {'status': 'done', 'responder_id': 'mid_1', 'result': …}# Structured command (bypass NL — faster, typed)resp=striker.mesh.send(
"mid_1",
{"action": "execute", "instruction": "cross to box"},
timeout=30.0,
)
# Broadcast to every peer on the meshresponses=striker.mesh.broadcast({"action": "status"}, timeout=3.0)
forrinresponses:
print(r["responder_id"], r["result"])
# Safety — always broadcasts, always winsstriker.mesh.emergency_stop()
# ═══════════════════════════════════════════════════════════════# 3. GPU-native training at scale — #96 (Newton)# ═══════════════════════════════════════════════════════════════fromstrands_robots.simulationimportcreate_simulationsim=create_simulation("newton", solver="mujoco", num_envs=4096)
sim.create_world()
sim.add_robot("so100")
sim.replicate(4096)
sim.step(100) # 4096 envs × 100 steps, one GPU# Differentiable simulation for gradient-based policy optimizationsim.run_diffsim(num_steps=100, loss_fn=…, optimize_params=…)
# ═══════════════════════════════════════════════════════════════# 4. Photorealistic synthesis — #97 (Isaac Sim)# ═══════════════════════════════════════════════════════════════sim=create_simulation("isaac", rtx_mode="path_traced")
sim.create_world()
sim.add_robot("unitree_g1")
sim.add_terrain("rough")
frame=sim.render(camera_name="wrist") # RTX path-traced RGBsim.record_video("demo.mp4", duration=10) # for DreamGen dataset gen# ═══════════════════════════════════════════════════════════════# 5. Mixed sim + real fleets — same API across all three backends# ═══════════════════════════════════════════════════════════════planner=Robot("so100", mode="sim", backend="newton", peer_id="planner")
executor=Robot("so100", mode="real", peer_id="executor")
# Planner trains the policy in 4096-env Newton, then tells the real robotplanner.mesh.tell("executor", "approach the red cube and grasp it")
# ═══════════════════════════════════════════════════════════════# 6. Live observability — mesh publishes state + events# ═══════════════════════════════════════════════════════════════# Every Robot publishes joint state, camera frames, and events to# strands/<peer_id>/... at up to 50 Hz. Any peer (or the dashboard)# can subscribe.python-mstrands_robots.dashboard# live gauges, camera feeds,# teleop, E-STOP button
Full example — 2-robot coordinated pick-and-place
fromstrands_robotsimportRobotimporttimepicker=Robot("so100", peer_id="picker")
placer=Robot("so100", peer_id="placer")
# Wait for discoverywhilelen(picker.mesh.peers) <1:
time.sleep(0.1)
# Orchestrate through the meshpicker.execute("pick up the red cube")
picker.mesh.tell("placer", "open gripper and hold still")
picker.mesh.tell("placer", "gripper closing — release on contact")
picker.execute("release gripper")
placer.execute("move to target pose and place cube")
Full example — 5v5 humanoid coordination (the agentic_soccer teaser)
# 10 G1 humanoids — 5 per team, each running its own Strands agentstriker=Robot("unitree_g1", peer_id="striker")
mid_1=Robot("unitree_g1", peer_id="mid_1")
def_1=Robot("unitree_g1", peer_id="def_1")
# … and so on for 10 peers# Striker initiates a play via the meshstriker.mesh.tell("mid_1", "pass diagonal")
mid_1.mesh.tell("striker", "cross to box")
striker.execute("shoot")
If you see something missing, mislabeled, or mistimed - drop a comment. This issue is the pinned source of truth for where every PR and issue is headed across the next two releases.
Last updated: 2026-04-21 (added #96 Newton, #97 Isaac Sim, #98 Robot Mesh tracking issues)
TL;DR
v0.3.9 — "The 5-line promise"
After this ships,
pip install strands-robots[sim]is the fastest path from zero to a robot arm grasping a cube in MuJoCo. GR00T N1.7 support lands in the same release so real-hardware users get the latest Cosmos-Reason2-2B backbone.What the user gets
Blocking PRs (merge order matters)
[sim])Robot()factory + lazy importsgr00t_inferenceinput validationSupporting issues
devbranch withmain(24 commits behind)sleep()with readiness checks + time-window RTC latencygrootdirectory/module togr00tDefinition of Done
pip installmainv0.4.0 — Robot Mesh + Newton + Isaac Sim
The big one. Three headline features land together in v0.4.0:
Robot()auto-joins a Zenoh peer network. Agent-to-agent tool calls over a LAN with zero config.NewtonSimulation(SimEngine)on NVIDIA Warp + Newton 1.x. GPU-native, 4096+ envs per GPU, differentiable.IsaacSimulation(SimEngine)with USD + IsaacLab 3.0. Photorealistic, Replicator synthetic data.Together these turn
strands-robotsfrom "run one robot in MuJoCo" into "run a fleet across sim + real + photoreal, all coordinated through a mesh."What the user gets
Full example — 2-robot coordinated pick-and-place
Full example — 5v5 humanoid coordination (the
agentic_soccerteaser)Blocking PRs & tracking issues
feat(mesh): Zenoh-based Robot Meshfeat(newton): NVIDIA Warp/Newton backendSimEngineon Warp 1.x. ~2.1K LOC across 7 PRsfeat(isaac): Isaac Sim backendrobot_meshprimitive; +4,273 / −142up_axisfix +[isaac]extrassamples/fleet_orchestration.py,samples/live_stream.py,samples/emergency_stop.pyexamples/newton_fleet_training.py,examples/isaac_photo_realistic_so100.pyExecution ordering
All three headline issues depend on the
SimEngineABC (#84) and the MuJoCo reference backend (#85) landing first. Once v0.3.9 ships:Supporting issues
robot_meshtool APIlerobot_teleoperate.pyModalityTransformmethods inDataConfigsDefinition of Done
Robot Mesh (#98)
robot.mesh.tell()/send()/broadcast()/emergency_stop()— stable public APIexamples/Newton backend (#96)
create_simulation("newton")registered + lazy-importedsim.add_robot("so100")+sim.step(100)stable at 4096 envs on one GPUsim.replicate(num_envs=4096)— >50k aggregate steps/sec targetsim.run_diffsim(...)+sim.solve_ik(...)working end-to-enddocs/backends/newton.md+ 2 examples shippedIsaac Sim backend (#97)
create_simulation("isaac")registered + lazy-importedsim.is_available()runtime check works without Isaac Sim installedurdf_to_usd/mjcf_to_usdasset converter merges as independent sub-PRsim.render(rtx_mode="path_traced")produces photorealistic framesadd_terrain+get_contact_forces+record_videoall workingdocs/backends/isaac.md+ 3 examples shipped🗺️ Beyond v0.4 — research directions
Not scoped to a release yet, but tracked for later:
📚 Full snapshot (2026-04-21)
All 10 open PRs
gr00t_inferenceinput validationRobot()factoryup_axisfixAll 16 open issues
feat(mesh): Zenoh-based Robot Meshfeat(isaac): Isaac Sim backendfeat(newton): Newton backenddevwithmainsleep()with readiness checksgroot→gr00tlerobot_teleoperate.py📬 Feedback
If you see something missing, mislabeled, or mistimed - drop a comment. This issue is the pinned source of truth for where every PR and issue is headed across the next two releases.
Last updated: 2026-04-21 (added #96 Newton, #97 Isaac Sim, #98 Robot Mesh tracking issues)