Skip to content

Extract -core library crates from WASM operators to enable host-side unit testing and coverage #455

@WilliamBerryiii

Description

@WilliamBerryiii

Problem

Four WASM operator crates under src/500-application/**/operators/** cannot be unit tested or coverage-measured on the host:

  • src/500-application/507-ai-inference/operators/onnx-inference
  • src/500-application/512-avro-to-json/operators/avro-to-json
  • src/500-application/513-pre-onnx-inference/operators/pre-onnx-inference
  • src/500-application/513-pre-onnx-inference/operators/post-onnx-inference

These crates are crate-type = ["cdylib"] targeting wasm32-wasip2, depend on wit-bindgen generated bindings, and import wasm_graph_sdk from the private aio-sdks cargo registry. Together this means:

  • cargo test cannot run them on the host (x86_64-unknown-linux-gnu) — bindings reference WASI component-model symbols that do not link.
  • cargo llvm-cov cannot produce coverage for them on CI runners.
  • They are currently excluded from coverage in codecov.yml under ignore: and absent from the rust-tests.yml matrix, leaving real business logic untested.

This is the documented exclusion path called out in .github/instructions/rust-crate-registration.instructions.md ("Coverage Opt-Out"), but the long-term intent is to bring this logic under test.

Proposed Solution

Refactor each WASM operator into two crates following the standard Rust pattern for WASM components:

  1. <operator>-core — pure-Rust library (crate-type = ["rlib"]), no wit-bindgen, no wasm_graph_sdk, no WASI/component-model dependencies. Holds all transformation, parsing, inference orchestration, and validation logic. Fully testable on the host with cargo test and measurable with cargo llvm-cov.
  2. <operator> (existing) — thin cdylib shim that wires wit-bindgen exports to <operator>-core functions and handles only the WASM ABI surface.

Then:

  • Register each -core crate in .github/workflows/rust-tests.yml coverage matrix.
  • Add each -core crate path to flags.rust.paths in codecov.yml.
  • Keep the cdylib shim crates listed under codecov.yml ignore: (they remain untestable on the host and their logic is now empty).
  • Update .github/instructions/rust-crate-registration.instructions.md to reflect that WASM operators follow the split-crate pattern and only the -core half participates in coverage.

Acceptance Criteria

  • Each of the four operators is split into <operator> (cdylib shim) + <operator>-core (rlib).
  • <operator>-core crates contain all non-binding logic and have unit tests covering it.
  • All four -core crates are added to the coverage matrix in .github/workflows/rust-tests.yml, with vuln-scan matrix.index length updated accordingly.
  • All four -core crate path globs are added to flags.rust.paths in codecov.yml.
  • The four cdylib shim entries remain under codecov.yml ignore: (or are converted to a path glob that excludes only the shim subdir if cleaner).
  • pwsh ./scripts/Validate-RustCrateRegistration.ps1 passes.
  • CI on a representative PR shows non-zero coverage uploaded for each new -core crate.
  • WASM build of the shim crates (wasm32-wasip2) still succeeds and produced .wasm artifacts pass existing operator integration checks.

Out of Scope

  • Changing the operator runtime contract or wit interface.
  • Migrating off wasm_graph_sdk or the aio-sdks registry.
  • Host-side execution of the WASM components themselves (e.g., via wasmtime) — this issue covers only library-level logic extraction.
  • Test coverage for the thin shim crates.

Risks

  • wit-bindgen-generated types may leak into what looks like "pure logic," requiring careful boundary design or small adapter types in -core.
  • wasm_graph_sdk types used in operator signatures may need to be abstracted behind -core-owned traits to keep the SDK dependency on the shim side only.
  • Splitting could surface previously hidden coupling and require minor refactors to make logic testable.

References

  • PR feat(ci): enforce rust crate registration in codecov coverage (#155) #449 — Rust coverage + Codecov rollout that established the current ignore list.
  • codecov.yml — current ignore: block listing the four operator crates.
  • .github/workflows/rust-tests.yml — coverage matrix to extend.
  • .github/instructions/rust-crate-registration.instructions.md — registration contract that documents the opt-out path being closed by this work.

Metadata

Metadata

Labels

refactorCode refactoring, no version bumprustPull requests that update rust codetech-debttestingTest infrastructure and test files

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions