Skip to content

Native Layer 1: Zero-copy COSE primitives with streaming parse/sign/verify#181

Merged
JeromySt merged 1 commit intousers/jstatia/native_ports_finalfrom
users/jstatia/native_primitives
Mar 30, 2026
Merged

Native Layer 1: Zero-copy COSE primitives with streaming parse/sign/verify#181
JeromySt merged 1 commit intousers/jstatia/native_ports_finalfrom
users/jstatia/native_primitives

Conversation

@JeromySt
Copy link
Copy Markdown
Member

@JeromySt JeromySt commented Mar 19, 2026

Native Layer 1 — Primitives

Foundational Rust crates for COSE_Sign1 message handling with a zero-copy, streaming-first architecture.

Architecture

CoseSign1Message owns a single Arc<[u8]> backing buffer. All fields are byte-range slices — no payload/signature copies on parse. Header values use ArcSlice/ArcStr to share the same buffer. LazyHeaderMap defers parsing until first access via OnceLock.

Memory profiles:

Operation Buffered Streamed (10GB file)
Parse 1x input + ~700B ~1.4KB
Sign (detached) ~64KB peak ~64KB peak
Verify (ECDSA/RSA) ~64KB peak ~64KB peak
Verify (Ed25519) 1x payload 1x payload (fallback)

Rust Crates (8)

CBOR Layer:

  • cbor_primitives — Encoding/decoding traits (CborDecoder, CborStreamDecoder)
  • cbor_primitives_everparse — EverParse backend + streaming decoder from Read+Seek

COSE Layer:

  • cose_primitives — RFC 9052 types, CoseData (Buffered/Streamed), LazyHeaderMap, ArcSlice/ArcStr
  • cose_sign1_primitives — Sign1 message, builder, sig_structure, streaming parse/sign/verify

Crypto Layer:

  • crypto_primitives — Signer/verifier traits, JWK types, SigningContext/VerifyingContext
  • cose_sign1_crypto_openssl — OpenSSL EC/RSA/EdDSA/ML-DSA provider

FFI Projections:

  • cose_sign1_primitives_ffi — C-ABI for Sign1 operations
  • cose_sign1_crypto_openssl_ffi — C-ABI for crypto provider

C/C++ Projections

  • cose.h / cose.hpp — Shared COSE types
  • sign1.h / sign1.hpp — Sign1 primitives
  • crypto/openssl.h / crypto/openssl.hpp — Crypto provider

Key Design Decisions

  • Single-buffer ownership: CoseData owns the raw bytes; CoseSign1 slices into it
  • Streaming CBOR: CborStreamDecoder reads from Read+Seek, skips large payloads by offset
  • Single-pass sign+embed: Embedded payloads read once into a shared buffer used for both signing and .Content
  • MemoryPayload uses Arc<[u8]>: open() is a pointer copy, not a data clone
  • Lazy headers: OnceLock defers BTreeMap allocation until first typed access
  • EverParse streaming note: Stream decoder parses manually; headers are re-validated via EverParse on lazy parse

Documentation

  • native/rust/docs/memory-characteristics.md — Full memory profiles and scenarios
  • native/docs/ARCHITECTURE.md — Layered dependency graph
  • Per-crate READMEs, FFI guide, getting started

Quality

  • 221+ tests, 90%+ line coverage (collect-coverage-asan.ps1 passes)
  • Zero cargo clippy warnings
  • cargo fmt clean

@JeromySt JeromySt force-pushed the users/jstatia/native_primitives branch 3 times, most recently from 0f67df0 to ca206c3 Compare March 19, 2026 15:45
@JeromySt JeromySt closed this Mar 19, 2026
@JeromySt JeromySt reopened this Mar 19, 2026
@JeromySt JeromySt force-pushed the users/jstatia/native_primitives branch from ca206c3 to 4aa03a5 Compare March 19, 2026 17:31
@JeromySt JeromySt closed this Mar 19, 2026
@JeromySt JeromySt reopened this Mar 19, 2026
@JeromySt JeromySt force-pushed the users/jstatia/native_primitives branch 3 times, most recently from 4ac8695 to 17c4a03 Compare March 19, 2026 19:00
@JeromySt JeromySt changed the title Native Primitives: CBOR, Crypto, COSE, Sign1, DID:x509 + FFI Native Layer 1: Primitive crates, OpenSSL crypto, C/C++ projections, and docs Mar 19, 2026
@JeromySt JeromySt force-pushed the users/jstatia/native_primitives branch 2 times, most recently from b9ba184 to 1bb6943 Compare March 20, 2026 22:21
@JeromySt JeromySt changed the title Native Layer 1: Primitive crates, OpenSSL crypto, C/C++ projections, and docs Native Layer 1: Zero-copy COSE primitives with streaming parse/sign/verify Mar 20, 2026
@JeromySt JeromySt force-pushed the users/jstatia/native_primitives branch 6 times, most recently from 0a2c616 to af8eac5 Compare March 20, 2026 23:09
@JeromySt JeromySt requested a review from Copilot March 20, 2026 23:11

This comment was marked as resolved.

@JeromySt
Copy link
Copy Markdown
Member Author

@copilot can you review the remaining files you didn't already review?

Copy link
Copy Markdown

Copilot AI commented Mar 20, 2026

@JeromySt I've opened a new pull request, #183, to work on those changes. Once the pull request is ready, I'll request review from you.

@JeromySt JeromySt force-pushed the users/jstatia/native_primitives branch 2 times, most recently from 54082a8 to 743a956 Compare March 21, 2026 00:06
JeromySt pushed a commit that referenced this pull request Mar 21, 2026
- builder.rs: Add payload_len validation in streaming sign detached path
  with total_read counter and LengthMismatch error. Add usize::try_from
  guard for embedded path capacity. Keep CborProvider import (needed for
  trait method .encoder()).
- ffi/types.rs: Replace unsound 'static lifetime on handle conversions
  with explicit lifetime parameter 'a to prevent use-after-free.
- ffi/error.rs: Replace unsound 'static lifetime on handle_to_inner with
  explicit 'a. Add FFI_ERR_PAYLOAD_ERROR (-7) to distinguish PayloadError
  from PayloadMissing. Map PayloadError to new code.
- ffi/cbindgen.toml: Remove unused prefix to avoid doubled symbol names.
- ffi/Cargo.toml: Document why rlib is needed (integration tests).
- sign1.h: Fix RFC 9338 -> RFC 9052, add COSE_SIGN1_ERR_PAYLOAD_ERROR.
- cose.h: Fix doc to reference cose_string_free() not cose_sign1_string_free().
- README.md: Update to match actual API (cose_sign1_primitives, compile-time
  CBOR provider, CryptoSigner/CryptoVerifier traits).
- troubleshooting.md: Fix constant name to LARGE_PAYLOAD_THRESHOLD.
- dotnet.yml #15: Intentional - detect-changes gates all events including
  workflow_dispatch via dorny/paths-filter default branch comparison.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@JeromySt JeromySt force-pushed the users/jstatia/native_primitives branch 3 times, most recently from 7607237 to 72bfc78 Compare March 21, 2026 06:02
…ing support

Rust crates (8): cbor_primitives, cbor_primitives_everparse,
crypto_primitives, cose_primitives, cose_sign1_primitives,
cose_sign1_crypto_openssl, cose_sign1_primitives_ffi,
cose_sign1_crypto_openssl_ffi

Architecture: Single Arc<[u8]> backing buffer, ArcSlice/ArcStr
zero-copy headers, LazyHeaderMap via OnceLock, CoseData
Buffered/Streamed variants, CborStreamDecoder for large files.
Streaming parse (~1.4KB), sign (~64KB), verify (~64KB).

Includes C/C++ header projections, documentation, CI caching,
all review comments addressed, 90%+ coverage gate passing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@JeromySt JeromySt force-pushed the users/jstatia/native_primitives branch from 72bfc78 to 0c5e7ff Compare March 23, 2026 19:52
@JeromySt JeromySt marked this pull request as ready for review March 30, 2026 16:37
@JeromySt JeromySt merged commit dab33a8 into users/jstatia/native_ports_final Mar 30, 2026
14 checks passed
@JeromySt JeromySt deleted the users/jstatia/native_primitives branch March 30, 2026 16:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants