docs: update crates.io metadata and simplify public API

[zhexuany]

Summary

• Add keywords, categories, and documentation links to Cargo.toml for better crates.io discoverability
• Enhance lib.rs documentation with features table, supported formats, feature flags, and usage examples
• Add examples/s3_example.rs demonstrating S3 remote file access
• Add examples/transform.rs demonstrating topic/type transformations
• Update examples/convert_format.rs to clarify same-format rewriting (cross-format not supported)
• Remove HTTP/HTTPS references (not supported)
• Fix ReaderConfig documentation (parallel processing config, not authentication)
• Simplify public API: remove internal types from top-level exports
• Update internal code to use explicit crate::core::* paths

Public API Changes

Previously exported (now internal):

• CodecValue, DecodedMessage, Encoding, PrimitiveType - now at robocodec::core::*
• FormatRewriter, MultiTransform, TopicRenameTransform, etc. - internal details

Still exported (public API):

• RoboReader, RoboWriter, RoboRewriter
• ChannelInfo, DecodedMessageResult
• TransformBuilder, TransformError
• ReaderConfig, WriterConfig, RewriteOptions, RewriteStats
• CodecError, Result

Test Plan

• cargo check --lib passes
• cargo test passes
• cargo doc --no-deps generates without warnings

[greptile-apps]

Greptile Overview

Greptile Summary

This PR improves crates.io metadata and significantly expands crate-level documentation and examples. It also narrows the top-level public API by removing several core/transform types from root re-exports, updating internal modules and tests to reference robocodec::core::* explicitly.

Key additions include new examples for S3 access and rewrite-time transformations, and updated docs clarifying that RoboRewriter rewrites within the same format (not cross-format conversion).

Confidence Score: 3/5

• This PR is close to mergeable but has doc/example correctness issues that should be fixed first.
• Core code changes are mostly straightforward path updates after the API simplification, but at least one new example likely won’t compile due to incorrect imports, and crate-level docs contain a concrete inconsistency about HTTP/HTTPS transport support versus other docs in the PR.
• src/lib.rs, examples/s3_example.rs, examples/convert_format.rs

Important Files Changed

Filename	Overview
Cargo.toml	Adds crates.io metadata fields (keywords/categories/repository/docs/readme) and shortens package description.
README.md	Removes unsupported HTTP/HTTPS reading docs; clarifies rewriter is same-format and updates examples accordingly.
benches/decoder_bench.rs	Updates benchmark references to core types (CodecValue) after top-level re-export removal.
examples/README.md	Reorganizes example docs; adds S3 and transform examples; clarifies rewriting vs conversion.
examples/convert_format.rs	Renames example to same-format rewriting and adds extension check; potential false-negative for URLs/paths without extension and misleading ext parsing.
examples/s3_example.rs	New S3 example imports FormatReader but uses reader.format(); likely doesn’t compile due to missing trait in scope / unused import depending on API.
examples/transform.rs	New transform example demonstrates RewriteOptions + TransformBuilder; does not enforce same-format output and prints format from extension only.
src/encoding/cdr/plan.rs	Adjusts PrimitiveType paths to crate::core::PrimitiveType after API surface change.
src/encoding/json/decoder.rs	Moves imports for core types (CodecError/CodecValue/DecodedMessage/Result) under crate::core.
src/encoding/protobuf/decoder.rs	Moves imports for core types (CodecError/CodecValue/DecodedMessage/Result) under crate::core.
src/lib.rs	Large docs rewrite and public re-export reduction; docs claim HTTP support via remote feature but README says HTTP unsupported—likely doc mismatch.
src/rewriter/engine.rs	Updates Encoding type references to crate::core::Encoding after export change.
src/schema/ast.rs	Updates schema PrimitiveType conversion to target crate::core::PrimitiveType and fixes corresponding tests.
tests/common/mod.rs	Adjusts tests to import CodecValue from robocodec::core after API simplification.
tests/mcap_integration_tests.rs	Adjusts tests to import CodecValue from robocodec::core; keeps FormatReader usage.
tests/schema_resolution_test.rs	Updates pattern matches to use robocodec::core::CodecValue after re-export removal.

Sequence Diagram

sequenceDiagram
    participant User
    participant RoboRewriter
    participant RewriteEngine
    participant TransformBuilder
    participant Transform
    participant Decoder
    participant Encoder

    User->>RoboRewriter: open(input_path)
    RoboRewriter->>RewriteEngine: init(format auto-detected)

    User->>TransformBuilder: with_topic_rename / with_type_rename
    TransformBuilder->>Transform: build()

    User->>RoboRewriter: with_options(input_path, RewriteOptions{transforms})
    RoboRewriter->>RewriteEngine: set transforms

    User->>RoboRewriter: rewrite(output_path)
    RoboRewriter->>RewriteEngine: rewrite()

    loop for each channel/message
        RewriteEngine->>Decoder: decode(message_bytes, schema, type)
        Decoder-->>RewriteEngine: DecodedMessage
        RewriteEngine->>Transform: apply(channel/topic/type)
        Transform-->>RewriteEngine: transformed metadata + message
        RewriteEngine->>Encoder: encode(transformed)
        Encoder-->>RewriteEngine: output bytes
    end

    RewriteEngine-->>RoboRewriter: RewriteStats
    RoboRewriter-->>User: stats

Loading

[greptile-apps]

_{16 files reviewed, 3 comments}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Non-compiling example import
use robocodec::{FormatReader, RoboReader}; looks incorrect for this crate’s API surface: FormatReader is re-exported under robocodec::io::traits::FormatReader (and robocodec::FormatReader is #[doc(hidden)]), and the example never calls any FormatReader methods anyway. As written, this example is likely to fail to compile due to the wrong import path (or at minimum triggers an unused import warning).

Prompt To Fix With AI

This is a comment left during a code review.
Path: examples/s3_example.rs
Line: 22:24

Comment:
**Non-compiling example import**
`use robocodec::{FormatReader, RoboReader};` looks incorrect for this crate’s API surface: `FormatReader` is re-exported under `robocodec::io::traits::FormatReader` (and `robocodec::FormatReader` is `#[doc(hidden)]`), and the example never calls any `FormatReader` methods anyway. As written, this example is likely to fail to compile due to the wrong import path (or at minimum triggers an unused import warning).

How can I resolve this? If you propose a fix, please make it concise.

[greptile-apps]

Extension check is brittle
The same-format validation uses rsplit('.').next() on the full path. For paths with no extension (or containing dots in directories / query strings), input_ext/output_ext become misleading and can reject valid inputs or accept invalid ones. Since RoboRewriter already auto-detects format, consider validating via the detected format (or at least use std::path::Path::extension() for local paths) to avoid false failures.

Prompt To Fix With AI

This is a comment left during a code review.
Path: examples/convert_format.rs
Line: 39:41

Comment:
**Extension check is brittle**
The same-format validation uses `rsplit('.').next()` on the full path. For paths with no extension (or containing dots in directories / query strings), `input_ext`/`output_ext` become misleading and can reject valid inputs or accept invalid ones. Since `RoboRewriter` already auto-detects format, consider validating via the detected format (or at least use `std::path::Path::extension()` for local paths) to avoid false failures.

How can I resolve this? If you propose a fix, please make it concise.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!