feat: Robot() factory + top-level lazy imports#86
feat: Robot() factory + top-level lazy imports#86cagataycali wants to merge 22 commits intostrands-labs:mainfrom
Conversation
1e2d93b to
253c01a
Compare
yinsong1986
left a comment
There was a problem hiding this comment.
All review comments addressed. LGTM.
|
For all comments in this PR, we should examine common themes and include corrections for them in AGENTS.md so that future agent runs benefit from their lessons. |
Review Thread Triage (14 unresolved)Already fixed in code (4 threads from @yinsong1986) — need thread resolution:
New from @awsarron (10 threads, Apr 10) — need code work:
Additional blockers:
Recommendation: Wait for PR #84 to merge, then rebase and address @awsarron's 10 threads in one pass. Items 7 and 9 may need a design discussion before implementing. 🤖 Pipeline analysis by AI agent. Strands Agents. Feedback welcome! |
1. Rename factory.py → robot.py, robot.py → hardware_robot.py Eliminates two 'Robot' classes in different files. The factory function now lives where users expect: strands_robots.robot.Robot 2. Default mode='sim' instead of mode='auto' Using real hardware should be an explicit decision since it affects the physical world. Robot('so100') now always returns simulation. Use mode='real' to explicitly opt into hardware control. 3. Fix ThreadPoolExecutor leak in _async_utils.py Register atexit.shutdown(wait=False) to clean up the module-level executor on interpreter exit. 4. Remove redundant list_robots() wrapper Was a 1-line passthrough to registry.list_robots(). Now __init__.py points directly to strands_robots.registry.list_robots. 5. Use module names in dataset_recorder docstring 'robot.py' → 'strands_robots.hardware_robot', 'simulation.py' → 'strands_robots.simulation' 6. Make camera shape configurable in dataset_recorder Added camera_shapes parameter to _build_features() instead of hardcoding (3, 480, 640). Default preserved for backward compat. 7. Add mode validation — invalid mode raises ValueError 8. Update __init__.py lazy imports for renamed modules Tests: 230 passed, 10 skipped, 0 failures Lint: ruff check + ruff format clean
7c9cc11 to
3803e26
Compare
📋 Review Status SummaryHi @awsarron — this PR has 11/14 threads resolved. Here's a summary of the 3 remaining unresolved threads to help focus the re-review: Unresolved Thread 1: "Document all env vars in README"
➔ Cross-PR dependency: This is being addressed in PR #87 (docs rewrite), which includes comprehensive env var documentation. Suggest resolving this thread with a note that #87 covers it, or merging #87 first. Unresolved Thread 2: "Where is
|
…s-labs#86 - Add Environment Variables table to README documenting all 6 env vars used across the project (STRANDS_ROBOT_MODE, STRANDS_ASSETS_DIR, STRANDS_URDF_DIR, STRANDS_TRUST_REMOTE_CODE, GROOT_API_TOKEN, MUJOCO_GL) plus cache directory documentation - Add module-level docstring to dataset_recorder.py explaining why it lives at package root (shared by both hardware and simulation paths, avoids circular dependency) - Add docstring to load_lerobot_episode() documenting that it is consumed by simulation.mujoco.policy_runner for replay_episode
…ssets (#84) Simulation foundation layer for `strands-robots`. Pure Python, no MuJoCo dependency. Unblocks #85 (MuJoCo backend) and #86 (Robot factory). ## What's in **Simulation abstractions** (`strands_robots/simulation/`) - `models.py` — `SimWorld`, `SimRobot`, `SimObject`, `SimCamera`, `TrajectoryStep`, `SimStatus` dataclasses. Backend-agnostic: engine handles live in `_model`/`_data`, everything else in `_backend_state: dict`. - `base.py` — `SimEngine` ABC. 12 required abstract methods + 4 optional (raise `NotImplementedError`). Context-manager protocol. `__del__` logs cleanup errors at warning level. - `factory.py` — `create_simulation()` + `register_backend()` with duplicate/alias shadow protection (raises `ValueError`; `force=True` for intentional overrides). Descriptive `ImportError` when a built-in backend module isn't installed. - `model_registry.py` — URDF/MJCF resolution: user-registered → `STRANDS_ASSETS_DIR` → `~/.strands_robots/assets/` → CWD → `robot_descriptions` fallback. Resolves search paths at call time (no import-time `Path.cwd()` snapshot). - `__init__.py` — thin re-exports with lazy `__getattr__`. **Assets** (`strands_robots/assets/`) - `__init__.py` — thin exports only (repo convention). - `manager.py` — path resolution with `safe_join()` traversal protection. `_has_meshes()` uses `os.scandir` + early-exit, cached by `(path, mtime)`. Module-level guard for optional `[sim]` extra — no circular import with `download.py`. - `download.py` — all download logic (`robot_descriptions` → git clone fallback). `_shallow_clone()` enforces `_ALLOWED_CLONE_URL_RE` (HTTPS github.com only). `_copy_and_clean` filters ignored patterns at `copytree()` time so user files in the cache aren't clobbered. **Tools** (`strands_robots/tools/`) - `download_assets.py` — thin `@tool` wrapper (~78 lines) that delegates to `assets.download.download_robots()`. No duplicated logic. **Registry** (`strands_robots/registry/`) - `user_registry.py` — `register_robot()` / `unregister_robot()` persisted to `~/.strands_robots/user_robots.json`. Fails closed on missing asset dir. Warns on alias collisions at registration time. Docstring warns this must not be exposed as an agent `@tool` without `STRANDS_TRUST_REMOTE_CODE` gating (MJCF → MuJoCo plugin code-exec risk). - `loader.py` — merges user-local registry on top of package `robots.json`. Public `invalidate_cache()` API (no private imports from callers). - `robots.json` — 38 → 68 robots (adds aerial, expressive, mobile_manip categories). - `__init__.py` — re-exports `register_robot`, `unregister_robot`, `list_user_robots`, `invalidate_cache`. **Utils** (`strands_robots/utils.py`) - `get_base_dir()` reads `STRANDS_BASE_DIR` — decoupled from `STRANDS_ASSETS_DIR` so setting the assets path no longer drops `user_robots.json` into an unexpected parent. - `get_assets_dir()`, `resolve_asset_path()`, `safe_join()`, `get_search_paths()` — single source of truth; consumed by model_registry, user_registry, assets/manager. **Docs & packaging** - `README.md` — environment variables table (`STRANDS_BASE_DIR`, `STRANDS_ASSETS_DIR`, `GROOT_API_TOKEN`) + cache directory docs. - `AGENTS.md` — documents nested-asset-path convention (e.g. `xmls/asimov.xml` matching upstream layout) and the `auto_download` strategy invariant. - `pyproject.toml` — new `[sim]` extra (`robot_descriptions>=1.11.0,<2.0.0`); included in `[all]`. ## Design decisions **SimEngine ABC contract.** 12 required methods every physics engine must implement; 4 optional (`load_scene`, `run_policy`, `randomize`, `get_contacts`) raise `NotImplementedError` so unimplemented features are explicit during development. `get_observation`/`send_action` are deliberately facade methods bridging Sim ↔ Policy — the agent tool sees a single interface without needing to know the Robot vs Sim split. **Asset resolution order.** Customer assets always win over defaults: `STRANDS_ASSETS_DIR` → `~/.strands_robots/assets/` → `CWD/assets/` → `robot_descriptions` fallback. Single env var for the asset tree (`STRANDS_ASSETS_DIR`); separate `STRANDS_BASE_DIR` for the base dir that holds `user_robots.json`. **Backend registration.** `register_backend()` rejects duplicates by default and blocks shadowing of built-in aliases (`mj`, `mjc`, `mjx`) unless `force=True`. Alias conflicts caught at both the `name` and `aliases` parameters. **Security.** - `safe_join()` applied everywhere registry values flow into filesystem paths (manager + download + user registry). - `_shallow_clone()` URL regex rejects `ssh://`, `git://`, `file://`, non-github hosts. - `register_robot()` is library-only; not surfaced as `@tool`. Docstring spells out the MJCF-plugin exec risk. ## Testing - 338 unit tests pass, 6 skipped, 0 failures - `ruff check` + `ruff format --check`: clean (57 files) - `mypy`: 0 issues in 57 source files - New test files: - `tests/test_simulation_foundation.py` — ABC contracts, factory round-trip, context-manager cleanup - `tests/test_simulation_factory.py` — duplicate rejection, alias shadowing, missing-backend ImportError - `tests/test_user_registry.py` — register/unregister, persistence, validation, path traversal; asserts `STRANDS_ASSETS_DIR` does NOT move the base dir / registry - `tests/test_registry_integrity.py` — auto-download invariant, alias uniqueness, canonical-shadow protection, lerobot_type presence on hardware-only robots ## Review history | Reviewer | Status | Threads | |-------------------|---------------------------------------|-----------------| | @yinsong1986 | APPROVED | 3/3 resolved | | @awsarron | CHANGES_REQUESTED → all addressed | 50/50 addressed | | @max-rattray-aws | COMMENTED → all addressed | 3/3 resolved | Closes #84. --------- Co-authored-by: cagataycali <cagataycali@icloud.com> Co-authored-by: strands-agent <217235299+strands-agent@users.noreply.github.com>
Complete MuJoCo simulation backend composed of focused mixins:
Simulation(AgentTool)
├── PhysicsMixin # raycasting, jacobians, energy, forces,
│ # mass matrix, checkpoints, inverse dynamics
├── PolicyRunnerMixin # run_policy, eval_policy, replay_episode
├── RenderingMixin # RGB/depth offscreen rendering, observations
├── RecordingMixin # LeRobot dataset recording
└── RandomizationMixin # domain randomization (colors, lighting, physics)
Supporting modules:
- backend.py: lazy mujoco import + headless GL auto-config (EGL/OSMesa/GLFW)
- mjcf_builder.py: procedural MJCF XML generation from dataclasses
- scene_ops.py: XML round-trip for runtime object/camera injection
- simulation.py: orchestrator dispatching 35 actions via tool_spec.json
- dataset_recorder.py: LeRobot v3 format recorder (parquet + video)
Key design decisions:
- Simulation extends AgentTool directly: Agent(tools=[Simulation()]) works
- Lazy MuJoCo import via _ensure_mujoco() — only when first needed
- XML round-trip for scene modification (standard: dm_control, robosuite)
- Same Policy ABC for sim and real — zero code changes for transfer
Tests: 47 new tests (12 E2E + 35 physics unit tests)
All use self-contained inline XML robots (no external files needed).
…anup HIGH: - Simulation now inherits SimulationBackend ABC (isinstance works) - start_policy rejects concurrent execution per robot (thread-safety) - XML injection protection via _sanitize_name() in MJCFBuilder MEDIUM: - overwrite defaults to False in start_recording - Silent frame dropping now respects strict=True (AGENTS.md strands-labs#5) LOW: - Remove dead _numpy_ify code - Replace insecure tempfile.mktemp with NamedTemporaryFile - Remove unimplemented total_reward from eval_policy - Reuse ThreadPoolExecutor in _async_utils (50Hz perf fix)
- mjcf_builder.py: move 'import re' after docstring into import block - simulation.py: sort SimulationBackend import alphabetically - dataset_recorder.py: add strict param to __init__ signature - Run ruff format on both files All checks pass: ruff check ✅, ruff format ✅, 335 tests ✅
- Wrap data.ctrl writes + mj_step calls with self._lock in run_policy and eval_policy to prevent concurrent MuJoCo data access - Apply _sanitize_name() to ALL user-provided names interpolated into MJCF XML (geom, joint, mesh, camera), not just body names - Import _sanitize_name in scene_ops for camera name validation Addresses review comments on thread-safety and XML injection. ruff check ✅, ruff format ✅, 335 tests ✅
Adds mujoco>=3.0.0,<4.0.0 to the [sim] optional-dependencies group, and includes it in [all] so CI installs it via 'uv sync --extra all --extra dev'.
Address yinsong1986's feedback to namespace the optional dependency group as [sim-mujoco] for clarity when additional sim backends are added.
…pdate lazy imports
…t stubs - Add mujoco.* to third-party ignore-missing-imports list - Add mypy override for simulation.mujoco.* with disable_error_code for attr-defined (cooperative mixin pattern), assignment (implicit Optional), override (extended signatures), and misc (MRO conflicts) - Add mypy override for _async_utils and dataset_recorder (pre-existing) - Fix add_robot/add_object/add_camera/move_object signatures: use X | None - Fix set_gravity, cleanup, __enter__/__exit__/__del__ return annotations - Fix randomization seed: int → int | None - Fix backend _ensure_mujoco return type annotation - Fix __init__.py __getattr__ type annotation
Removed 4 error categories from disable_error_code (assignment, typeddict-item, index, return-value) by fixing the actual code: - physics.py: Fix 16 implicit Optional params (X = None → X | None = None) - rendering.py: Fix 3 implicit Optional params - recording.py: Fix 2 implicit Optional params + inline ignore for fallback - policy_runner.py: Fix 5 implicit Optional params + inline ignore for narrowed arg - simulation.py: Fix send_action/create_world signatures to match base, fix variable name reuse bug (result → recompile_result), inline ignore for TypedDict ** expansion Remaining suppressed (all legitimate): - attr-defined (137): cooperative mixin pattern (self._world on mixins) - misc (3): MRO conflicts + import fallback redefinition - override (1): add_object extends base with orientation/mesh_path params - import-not-found (1): imageio optional dep - import-untyped (1): internal zenoh_mesh - has-type (1): dynamic renderer cache
…able_error_code Replace blanket disable_error_code with proper type fixes: - Add TYPE_CHECKING attribute declarations to all 5 mixins (PhysicsMixin, RenderingMixin, RecordingMixin, PolicyRunnerMixin, RandomizationMixin) so mypy can verify self._world, self._lock, etc. - Add _push_to_hub field to SimWorld dataclass (was missing) - Add orientation + mesh_path params to SimEngine.add_object base signature - Add **kwargs to RandomizationMixin.randomize to match base - Simplify SimEngine.randomize to **kwargs (backends define own params) - Add assert guards for _world None checks in rendering methods - Restructure recording.py import fallback to avoid redefinition errors - Fix _apply_sim_action Protocol stubs to match real signatures Result: 0 mypy errors, 0 disable_error_code, only 2 inline type: ignore with specific codes (arg-type for narrowed var, typeddict-item for ** expansion)
- Replace bare with for consistent optional dependency handling per project conventions - Add imageio and imageio-ffmpeg to sim-mujoco extras in pyproject.toml - Add type: ignore comment for dynamic imageio writer attribute
…ng, tests, XML parsing Addresses all 8 unresolved review threads from @awsarron (Apr 10): 1. pyproject.toml: Remove empty [sim] extra, move robot_descriptions into [sim-mujoco]. Update extra= reference in backend.py. (yinsong1986 thread) 2. pyproject.toml: Keep sim-mujoco naming (not just mujoco) for consistency with future sim-isaac, sim-pybullet extras. (awsarron nit — reply only) 3. mujoco/__init__.py: Stop exporting private functions (_configure_gl_backend, _ensure_mujoco, _is_headless). Internal callers already import from backend directly. (awsarron thread) 4. simulation.py: Centralize _ensure_mujoco() to __init__ — fail fast at construction time. Store as self._mj, use throughout Simulation methods. Mixins retain their own _ensure_mujoco() calls since they may be used independently. (awsarron thread) 5. backend.py: Add docstring explaining why _is_headless() is Linux-only — Windows uses WGL, macOS uses CGL, both support offscreen natively. (awsarron thread) 6. policy_runner.py: Replace duplicated private function TYPE_CHECKING stubs with a shared SimulationProtocol in new types.py module. Eliminates coupling via signature duplication. (awsarron thread) 7. test_mujoco_e2e.py: Add TestToolSpecActionCoverage — iterates every action enum in tool_spec.json and asserts hasattr(Simulation, method) via the alias map. Catches drift between spec and implementation. (awsarron thread) 8. scene_ops.py: Standardize on ElementTree for all XML manipulation. Converted inject_object_into_scene, inject_camera_into_scene, and _patch_xml_paths from regex/string.replace to ET. Kept regex fallback in _patch_xml_paths for malformed fragments. (awsarron thread)
…olicyRunnerMixin The `_: SimulationProtocol` pattern declares a class variable named `_` but does NOT propagate Protocol member declarations to mypy's understanding of the class. This caused 34 attr-defined errors in policy_runner.py. Fix: Replace with direct attribute declarations under TYPE_CHECKING, matching the pattern used by PhysicsMixin, RenderingMixin, RecordingMixin, and RandomizationMixin. The SimulationProtocol in types.py is preserved for runtime checks and documentation — it's the TYPE_CHECKING usage pattern that was incorrect. Lint: 0 errors (ruff check + ruff format + mypy) Tests: 323 passed, 2 skipped, 0 failures
Post-strands-labs#84 merge: SimWorld no longer carries MuJoCo-specific private fields (_xml, _robot_base_xml, _recording, _trajectory, _dataset_recorder, _tmpdir, _push_to_hub). These are MuJoCo backend implementation details and now live in world._backend_state, as the SimWorld docstring requests (prefer _backend_state over new fields). Migrated call sites: - mjcf_builder.py: tmpdir - policy_runner.py: recording, trajectory, dataset_recorder - recording.py: recording, trajectory, dataset_recorder, push_to_hub - scene_ops.py: robot_base_xml - simulation.py: xml, robot_base_xml, recording, trajectory Reads use dict[] where preceded by a guard that guarantees initialization (e.g. start_recording() sets before policy_runner reads), and .get() with sensible defaults where the key may be unset. Tests: 392 passed, 2 skipped (5 pre-existing test_path_validation failures are on main too — unrelated). Lint: ruff + mypy clean on 75 source files.
Add unified Robot() factory function that auto-detects sim vs real:
Robot('so100') → auto-detect → MuJoCo sim (default)
Robot('so100', mode='sim') → Simulation AgentTool
Robot('so100', mode='real') → HardwareRobot AgentTool
Auto-detect priority:
1. STRANDS_ROBOT_MODE env var (explicit override)
2. USB probe for servo controllers (Feetech/Dynamixel)
3. Default to sim (safest — never accidentally send to hardware)
Also adds list_robots(mode='all'|'sim'|'real'|'both') for discovery.
Updates __init__.py with lazy imports:
- Robot (→ factory), list_robots
- Simulation, SimWorld, SimRobot, SimObject, SimCamera
- Auto-configures MuJoCo GL backend for headless environments
Tests: 22 new factory tests covering name resolution, aliases,
list_robots filtering, auto-detect mode, Robot() factory, imports.
… test - sim.destroy() on partial factory failure (prevents resource leak) - except (ImportError, OSError) instead of bare Exception for USB probing - Added comment explaining why GL backend config must run eagerly - Added happy-path MuJoCo test gated behind pytest.importorskip
…detect p.description and p.manufacturer can both be None on some platforms. Guard with 'or ""' to prevent TypeError on string concatenation. All checks pass: ruff check ✅, ruff format ✅, 358 tests ✅
The test was calling Robot('so100') which requires downloaded URDF/mesh
assets not available in CI. Now uses a minimal inline MJCF XML via
tmp_path + urdf_path param — tests MuJoCo physics without external deps.
1. Rename factory.py → robot.py, robot.py → hardware_robot.py Eliminates two 'Robot' classes in different files. The factory function now lives where users expect: strands_robots.robot.Robot 2. Default mode='sim' instead of mode='auto' Using real hardware should be an explicit decision since it affects the physical world. Robot('so100') now always returns simulation. Use mode='real' to explicitly opt into hardware control. 3. Fix ThreadPoolExecutor leak in _async_utils.py Register atexit.shutdown(wait=False) to clean up the module-level executor on interpreter exit. 4. Remove redundant list_robots() wrapper Was a 1-line passthrough to registry.list_robots(). Now __init__.py points directly to strands_robots.registry.list_robots. 5. Use module names in dataset_recorder docstring 'robot.py' → 'strands_robots.hardware_robot', 'simulation.py' → 'strands_robots.simulation' 6. Make camera shape configurable in dataset_recorder Added camera_shapes parameter to _build_features() instead of hardcoding (3, 480, 640). Default preserved for backward compat. 7. Add mode validation — invalid mode raises ValueError 8. Update __init__.py lazy imports for renamed modules Tests: 230 passed, 10 skipped, 0 failures Lint: ruff check + ruff format clean
…s-labs#86 - Add Environment Variables table to README documenting all 6 env vars used across the project (STRANDS_ROBOT_MODE, STRANDS_ASSETS_DIR, STRANDS_URDF_DIR, STRANDS_TRUST_REMOTE_CODE, GROOT_API_TOKEN, MUJOCO_GL) plus cache directory documentation - Add module-level docstring to dataset_recorder.py explaining why it lives at package root (shared by both hardware and simulation paths, avoids circular dependency) - Add docstring to load_lerobot_episode() documenting that it is consumed by simulation.mujoco.policy_runner for replay_episode
Post-rebase on top of clean PR-85 (which enforces zero mypy errors), hardware_robot.py needed the same strictness: - __init__, cleanup, __del__, stop: add -> None - task_runner (nested): add -> None - **kwargs: add : Any annotation (3 methods) Result: 77/77 source files mypy clean.
449aa05 to
0917c24
Compare
|
Rebased onto the updated #85 (which itself is now rebased on main). Stack status:
What changed:
Conflict resolution:
Quality gate:
🤖 AI agent response. Strands Agents. Feedback welcome! |
TL;DR
Add
Robot()factory function that auto-detects sim vs real mode, plus lazy imports in__init__.pyfor all simulation types.What changed
strands_robots/factory.pyRobot()factory +list_robots()strands_robots/__init__.pytests/test_factory.pyUsage
Auto-detect logic
STRANDS_ROBOT_MODEenv var (explicit override)"sim"(safest — never accidentally send commands to hardware)Discussion point
Per Arron's feedback: "Robot always returned Robot. robot.backend returned HardwareRobot or Simulator instance" — current implementation is a factory function (returns the backend directly). Consider if
Robotshould be a wrapper class with.backendattribute for nicer typing.Testing
Part 5 of 6 in the MuJoCo simulation PR decomposition