v0.4.0-release

Release Highlights

• HoloBrain-0: A foundation model for general embodied manipulation. Including training code and evaluation code.

• New Research Projects: We have added comprehensive implementation scripts for notable research projects, including Monodream

New Features

• Project Implementations:

  • Added training and evaluation scripts for the HoloBrain-0 project.
  • Added evaluation scripts for the Monodream paper.

• RoboOrchard Dataset

  • Features
    • Add tf_graph datatypes as dataset features.
    • Add PickleFeature for arrow dataset.
  • Transform
    • Add image transforms.
    • Add transform to adapt arrow_dataset
  • Dataset
    • Support dataset index based repacking.
    • Add ConcatRODataset and sampler.
    • Add dataset merging method and support merge multiple datasets into single one.
    • Add meta query by statement
    • Support upgrade meta database in RODataset.
    • Upgrade Episode table schema to store more information.
  • Sampler
    • Add ColumnIndexOffsetSampler.
    • Support customized column index sampler.
    • Add batch row index sampler method for future.
  • Docs
    • Add tutorial about the RoboOrchard Dataset

• Inference and Model APIs:

  • Refactor TorchModel save and load for better compatibility of accelerate.
  • Refactor inference pipeline to be simpler and support batched inference.
  • Add model api tutorial and model zoo docs
  • ModelMixin and InferenceMixin support hf hub sub directory

• Evaluation

  • Add libero env.
  • Add distributed robotwin evaluator

Bug Fixes

v0.2.0-release

Release Highlights

• New Research Projects: We have added comprehensive implementation scripts for notable research projects, including AuxThink and FineGrasp.

• Streamlined Inference & Model Sharing: A new set of APIs, including TorchModelMixin and InferencePipelineMixin, dramatically simplifies the process of saving, loading, and sharing models. This system is fully compatible with the Hugging Face Hub, enabling seamless import and export of trained models and entire inference pipelines.

• Unified Data Format: Introducing the RoboOrchard Dataset, a sophisticated data structure designed to bridge the entire robotics workflow. It is natively compatible with both ROS bags for data collection and Hugging Face Datasets for training. Its design supports multi-frequency signals, efficient metadata querying, and direct visualization, solving many common data management challenges in robotics.

New Features

• Project Implementations:

  • Added evaluation scripts for the AuxThink paper.
  • Added training and evaluation scripts for the FineGrasp paper.
  • Added a rule-based pick-and-place demo.

• Inference and Model APIs:

  • TorchModelMixin: An API for easily saving and loading model weights and configurations, compatible with the Hugging Face Hub.
  • InferencePipelineMixin: An integrated API that packages a model and its data processor into a single, portable inference pipeline, similar to the transformers.pipeline and compatible with the Hugging Face Hub.

• RoboOrchard Dataset

  • Compatibility: Natively designed to be compatible with both ROS (mcap) message definitions and the arrow serialization format of Hugging Face Datasets, enabling a smooth transition from rosbag -> dataset.
  • High-Fidelity Data: Each row in the dataset table represents a "message chunk," preserving multi-frequency sensor data without downsampling.
  • Efficient Metadata Querying: Non-timeseries metadata (e.g., episode, task, robot info) is managed via an embedded database (like DuckDB/SQLite) for fast and convenient querying.
  • Easy Visualization: Supports direct export of episodes to mcap format for visualization in tools like Foxglove.

• Simulation Environments: Added apis for the Maniskill and RoboTwin simulation environments.

Improvements

• SEM: Updated to fully support the RoboTwin 2.0 environment.

Bug Fixes

v0.1.0-release

Framework

1. Add docs, including architecture, tutorials and api reference
2. Deprecate SimpleTrainer, migrate to HookBasedTrainer

Embodied AI Algorithm

1. Add BIP3D and SEM