Robocodec

Robocodec is a robotics data format library for reading, writing, and converting MCAP and ROS1 bag files. It provides a unified API with automatic format detection, parallel processing, and support for multiple message encodings (CDR, Protobuf, JSON) and schema types (ROS .msg, ROS2 IDL, OMG IDL).

Why Robocodec?

• Clean API - Only RoboReader, RoboWriter, RoboRewriter exposed at top level
• Auto-Detection - Format detected from file extension or URL scheme
• Fast - Parallel processing with rayon, zero-copy memory-mapped files
• S3-Native - First-class support for s3:// URLs (AWS S3, MinIO, Alibaba OSS, etc.)
• Transformations - Topic/type renaming and format conversion built-in

Quick Start

Rust

# Cargo.toml
[dependencies]
robocodec = "0.1"

use robocodec::RoboReader;

// Format auto-detected from extension
let reader = RoboReader::open("data.mcap")?;
println!("Found {} channels", reader.channels().len());

Python (from source)

Python bindings are available but must be built from source:

git clone https://github.com/archebase/robocodec.git
cd robocodec
make build-python-dev

from robocodec import RoboReader

reader = RoboReader("data.mcap")
print(f"Found {len(reader.channels)} channels")

Note: PyPI release is coming soon. For now, build from source using the instructions above.

Common Tasks

Read messages from a file

use robocodec::RoboReader;

let reader = RoboReader::open("file.mcap")?;

// List all channels
for channel in reader.channels() {
    println!("{}: {} messages", channel.topic, channel.message_count);
}

// Get message count
println!("Total messages: {}", reader.message_count());

Write messages to a file

use robocodec::RoboWriter;

let mut writer = RoboWriter::create("output.mcap")?;
let channel_id = writer.add_channel("/topic", "MessageType", "cdr", None)?;
// ... write messages ...
writer.finish()?;

Read decoded messages

use robocodec::RoboReader;

let reader = RoboReader::open("file.mcap")?;

for result in reader.decoded()? {
    let msg = result?;
    println!("Topic: {}", msg.topic());
    println!("Data: {:?}", msg.message);
    println!("Log time: {:?}", msg.log_time);
}

Read from S3

Robocodec supports reading directly from S3-compatible storage using s3:// URLs:

use robocodec::RoboReader;

// Format and S3 access auto-detected
let reader = RoboReader::open("s3://my-bucket/path/to/data.mcap")?;
println!("Found {} channels", reader.channels().len());

S3-compatible services (AWS S3, Alibaba Cloud OSS, MinIO, etc.) require credentials via environment variables:

# AWS S3
export AWS_ACCESS_KEY_ID="your-access-key"
export AWS_SECRET_ACCESS_KEY="your-secret-key"
export AWS_REGION="us-east-1"  # optional, defaults to us-east-1

# For Alibaba Cloud OSS, MinIO, or other S3-compatible services
export AWS_ACCESS_KEY_ID="your-oss-access-key"
export AWS_SECRET_ACCESS_KEY="your-oss-secret-key"

Note: While we use AWS-standard environment variable names for compatibility, robocodec works with any S3-compatible storage service.

Read from HTTP/HTTPS

Robocodec also supports reading directly from HTTP/HTTPS URLs:

use robocodec::RoboReader;

// Format detected from URL path, access via HTTP
let reader = RoboReader::open("https://example.com/data.mcap")?;
println!("Found {} channels", reader.channels().len());

Note: HTTP reading supports range requests for efficient access to large files.

HTTP Authentication

For authenticated HTTP endpoints, robocodec supports Bearer tokens and Basic authentication via ReaderConfig:

use robocodec::io::{RoboReader, ReaderConfig};

// Bearer token (OAuth2/JWT)
let config = ReaderConfig::default().with_http_bearer_token("your-token-here");
let reader = RoboReader::open_with_config("https://example.com/data.mcap", config)?;

// Basic authentication
let config = ReaderConfig::default().with_http_basic_auth("username", "password");
let reader = RoboReader::open_with_config("https://example.com/data.mcap", config)?;

Alternatively, you can provide authentication via URL query parameters:

use robocodec::RoboReader;

// Bearer token via URL
let reader = RoboReader::open("https://example.com/data.mcap?bearer_token=your-token")?;

// Basic auth via URL (user:pass encoded)
let reader = RoboReader::open("https://example.com/data.mcap?basic_auth=user:pass")?;

Write to S3

use robocodec::RoboWriter;

// Format detected from .mcap extension, S3 from s3:// URL
let mut writer = RoboWriter::create("s3://my-bucket/output.mcap")?;
let channel_id = writer.add_channel("/topic", "MessageType", "cdr", None)?;
// ... write messages ...
writer.finish()?;
}

Custom S3 endpoints

For S3-compatible services with custom endpoints:

Option 1: Environment variable (global)

export S3_ENDPOINT="http://localhost:9000"  # MinIO
export S3_ENDPOINT="https://oss-cn-hangzhou.aliyuncs.com"  # Alibaba OSS

Option 2: URL query parameter (per-request)

use robocodec::RoboReader;

// MinIO running locally
let reader = RoboReader::open("s3://bucket/data.mcap?endpoint=http://localhost:9000")?;

// Alibaba Cloud OSS (Hangzhou region)
let reader = RoboReader::open(
    "s3://bucket/data.mcap?endpoint=https://oss-cn-hangzhou.aliyuncs.com"
)?;

Convert between formats

use robocodec::RoboRewriter;

let rewriter = RoboRewriter::open("input.bag")?;
rewriter.rewrite("output.mcap")?;

Rename topics during conversion

use robocodec::{RoboRewriter, TransformBuilder};

let transform = TransformBuilder::new()
    .with_topic_rename("/old/topic", "/new/topic")
    .build();

let rewriter = RoboRewriter::with_options(
    "input.mcap",
    robocodec::RewriteOptions::default().with_transforms(transform)
)?;
rewriter.rewrite("output.mcap")?;

Installation

Rust Users

Add to Cargo.toml:

[dependencies]
robocodec = "0.1"

Optional features:

robocodec = { version = "0.1", features = ["jemalloc"] }

Feature	Description	Default
s3	S3-compatible storage support (AWS S3, MinIO, etc.)	✅ Yes
python	Python bindings	❌ No
jemalloc	Use jemalloc allocator (Linux only)	❌ No

Python Users

Build from source (PyPI release coming soon):

git clone https://github.com/archebase/robocodec.git
cd robocodec
make build-python-dev

Supported Formats

Format	Read	Write
MCAP	✅	✅
ROS1 Bag	✅	✅
RRF2 (Rerun)	✅	✅

Note: RRF2 support is compatible with Rerun 0.27+. Earlier versions use a different format and are not supported.

Message Encodings

Encoding	Description
CDR	Common Data Representation (ROS1/ROS2)
Protobuf	Protocol Buffers
JSON	JSON encoding

Schema Support

• ROS .msg files (ROS1)
• ROS2 IDL (Interface Definition Language)
• OMG IDL (Object Management Group)

License

MulanPSL v2 - see LICENSE

Development

Testing

make test              # Run all tests
make test-rust         # Run Rust tests only
make test-python       # Run Python tests only

Fuzzing

Robocodec includes comprehensive fuzzing infrastructure for parser security and robustness testing:

./scripts/fuzz_init.sh  # Initialize fuzzing infrastructure (one-time setup)
make fuzz               # Quick fuzzing check (30s per target)
make fuzz-all           # Extended fuzzing (1min per target)
make fuzz-mcap          # Fuzz MCAP parser only

For detailed fuzzing documentation, see docs/FUZZING.md.

Benchmarks

make bench              # Run performance benchmarks
make bench-compare      # Compare against baseline

Links

• Issue Tracker
• Security Policy
• Fuzzing Guide