feat(build): add multi-language fuzzing infra (CFLite + Codecov flags)#453
Open
WilliamBerryiii wants to merge 33 commits intomainfrom
Open
feat(build): add multi-language fuzzing infra (CFLite + Codecov flags)#453WilliamBerryiii wants to merge 33 commits intomainfrom
WilliamBerryiii wants to merge 33 commits intomainfrom
Conversation
* Wire ClusterFuzzLite Dockerfile, build scripts, and PR-only workflow * Add cargo-fuzz harness for 501-rust-telemetry; Atheris stubs for 505/506/509/510 * Extend Detect-Folder-Changes.ps1 + matrix-folder-check.yml with fuzz outputs * Add 11 per-component Codecov flags (carryforward); SHA-pin CFLite actions * Wave 2 soft-fail (continue-on-error: true) until two clean sprints 🧪 - Generated by Copilot
added 6 commits
April 28, 2026 13:05
The receiver fuzz crate inherits the parent nightly toolchain pin which uses profile=minimal and lacks the clippy component. Skip any crate path ending in /fuzz during clippy crate discovery so the workflow stays green. Also add a top-level doc comment to parse_telemetry.rs to trigger the fuzz-pr.yml path filter and exercise the fuzz pipeline on this PR.
…startup_failure on PRs without fuzz changes
…dation/main - Pin ClusterFuzzLite actions to v1 SHA 884713a6c30a92e5e8544c39945cd7cb630abcd1 to fix startup_failure - Convert fuzz-pr.yml to reusable workflow (workflow_call) - Add fuzz job to pr-validation.yml and fuzz-main job to main.yml - All fuzz jobs use continue-on-error: true (Wave 2 soft-fail)
🔒 - Generated by Copilot
- emit lowercase 'true'/'false' for changesInFuzz{Rust,Python,Js} so case-sensitive GHA gating matches
- harden default fuzz folder outputs to {"folderName":[]} for matrix compatibility
- broaden JS fuzz regex to match .mjs/.cjs and /tests/fuzz/ paths
🐛 - Generated by Copilot
added 3 commits
April 29, 2026 15:42
…seBranch - rename -IncludeIaCFolders to -IncludeAllIaC - rename -IncludeApplications to -IncludeAllApplications - explicitly pass -BaseBranch to prevent positional binding corruption 🐛 - Generated by Copilot
- replace array splat with hashtable to bind named parameters correctly - positional binding caused BaseBranch="-BaseBranch" and ApplicationPath empty - unblocks fuzz Rust/Python/JS jobs on PR #453 🐛 - Generated by Copilot
- add software-properties-common and deadsnakes PPA to Dockerfile - install python3.12-venv and bootstrap pip via get-pip.py - works around OSS-Fuzz base-builder-rust:v1 being Ubuntu 20.04 focal 🐍 - Generated by Copilot
4 tasks
added 8 commits
April 29, 2026 19:51
🐛 - Generated by Copilot
…back on noble - add build-essential, cmake, python3 apt packages so node-gyp can compile - install prebuild, cmake-js, node-gyp globally before @jazzer.js/core@2.1.0 - resolves exit 127 from missing prebuild CLI when napi prebuilt binary is unavailable for noble + Node 20 🐛 - Generated by Copilot
- detect cargo-fuzz layout by requiring both fuzz/Cargo.toml and parent Cargo.toml - skip Python pytest fixture dirs named 'fuzz' (e.g. sensor-simulator/tests/fuzz) - emit skip message for visibility in CI logs 🛠️ - Generated by Copilot
- drop ubuntu-24-04 base tag (glibc 2.38) that broke bad_build_check on ubuntu-20.04 runner (glibc 2.31); use default base-builder-rust - remove Python (Atheris) and Node.js (Jazzer.js) toolchain layers and COPY of build_python.sh/build_js.sh from Dockerfile - comment out Py/JS dispatch in build.sh - gate fuzz-python and fuzz-js workflow jobs with if: false; preserve original change-detect expressions as comments for restore - preserve build_python.sh/build_js.sh and existing harnesses for the follow-up per-language builder containers (split from #150) Refs #150 Refs #453 🐛 - Generated by Copilot
…tion Wrap Invoke-ScriptAnalyzer in try/catch so PSSA internal crashes surface the offending file as a warning instead of failing the entire lint job. 🛡️ - Generated by Copilot
12 tasks
…ners (#459) Extend the ClusterFuzzLite image to fuzz Python and JavaScript harnesses alongside the existing Rust scaffolding by parameterizing the base image and dispatching to language-specific build scripts at runtime. Container changes: - Parameterize Dockerfile with ARG LANGUAGE (default rust); FROM now resolves to gcr.io/oss-fuzz-base/base-builder-${LANGUAGE} so the ClusterFuzzLite action's language: input drives the base layer. - Copy build_rust.sh, build_python.sh, and build_js.sh into the image and dispatch from build.sh via case on ${LANGUAGE:-rust}. Python harness (services 505, 509, 510; 506 deferred to WI-02): - build_python.sh installs per-service requirements.txt when present, builds each harness with PyInstaller --onefile, and emits an ASAN wrapper script that exports LD_PRELOAD for libasan before exec. JavaScript harness (service 513): - build_js.sh runs npm ci (falling back to npm install) before emitting the Jazzer.js wrapper so @jazzer.js/core resolves at fuzz time. - Add @jazzer.js/core ^2 as devDependency in 513 package.json. CI and docs: - Re-enable fuzz-python and fuzz-js jobs in fuzz-pr.yml (drop if: false). - Remove orphan fuzz-py-506 flag from codecov.yml. - Add .clusterfuzzlite/README.md describing the polyglot architecture. - Add docs/build-cicd/clusterfuzzlite.md with troubleshooting section. Decisions: - ID-02=A: bash-only build scripts (rejected pwsh shim to keep image size down and avoid pulling the PowerShell supply chain into a fuzzing base image). - ID-03=A: fold into existing PR #150 by stacking on the parent feat/issue-150-fuzzing-infrastructure branch rather than opening a new PR. Closes #459 🐛 - Generated by Copilot
added 3 commits
May 1, 2026 11:00
…rFuzzLite CI (#459) - DD-05: hadolint DL3006 waiver in .clusterfuzzlite/Dockerfile + README note; codecov.yml gains fuzz-js-513 flag (and re-adds fuzz-py-506 per DD-04). - DD-06: add fuzz_smoke.py harnesses for services 505/509/510 and a fuzz_smoke.mjs harness for service 513; extend HARNESSES arrays in build_python.sh and build_js.sh so the new harnesses are built. 🧪 - Generated by Copilot
📚 Documentation Health ReportGenerated on: 2026-05-01 20:37:40 UTC 📈 Documentation Statistics
🏗️ Three-Tree Architecture Status
🔍 Quality Metrics
This report is automatically generated by the Documentation Automation workflow. |
…rds, fix shell lint - set python and js fuzz sanitizer to none in fuzz-pr.yml - install atheris before harness build in build_python.sh - add distpath/preinstalls/symbolizer and other words to .cspell.json - fix case-arm spacing in .clusterfuzzlite/build.sh 🛠️ - Generated by Copilot
📚 Documentation Health ReportGenerated on: 2026-05-01 23:04:31 UTC 📈 Documentation Statistics
🏗️ Three-Tree Architecture Status
🔍 Quality Metrics
This report is automatically generated by the Documentation Automation workflow. |
auyidi1
approved these changes
May 1, 2026
📚 Documentation Health ReportGenerated on: 2026-05-01 23:31:10 UTC 📈 Documentation Statistics
🏗️ Three-Tree Architecture Status
🔍 Quality Metrics
This report is automatically generated by the Documentation Automation workflow. |
🐛 - Generated by Copilot
📚 Documentation Health ReportGenerated on: 2026-05-02 00:14:28 UTC 📈 Documentation Statistics
🏗️ Three-Tree Architecture Status
🔍 Quality Metrics
This report is automatically generated by the Documentation Automation workflow. |
…o none - install atheris+pyinstaller in .clusterfuzzlite/Dockerfile when LANGUAGE=python so compile.py import succeeds - switch fuzz-pr.yml javascript sanitizer from address to none (CFL JS rejects address) - format fuzz_smoke harnesses (PEP8 blank line after docstring) 🐛 - Generated by Copilot
📚 Documentation Health ReportGenerated on: 2026-05-02 00:56:05 UTC 📈 Documentation Statistics
🏗️ Three-Tree Architecture Status
🔍 Quality Metrics
This report is automatically generated by the Documentation Automation workflow. |
- Dockerfile: CFLite does not forward language: as a build-arg, so the LANGUAGE
ARG always defaulted to rust at image-build time, leaving atheris uninstalled
for python harnesses. Switch to a fixed base-builder-rust base and install
atheris+pyinstaller unconditionally.
- fuzz-pr.yml: javascript jobs require sanitizer in {address,memory,undefined,
coverage}; replace 'none' with 'coverage'.
- build_python.sh: prefer 'python3 -m pip' over the bare pip3 shim.
🛠 - Generated by Copilot
📚 Documentation Health ReportGenerated on: 2026-05-03 03:01:57 UTC 📈 Documentation Statistics
🏗️ Three-Tree Architecture Status
🔍 Quality Metrics
This report is automatically generated by the Documentation Automation workflow. |
…arnesses The OSS-Fuzz compile step exports FUZZING_LANGUAGE (canonical), not LANGUAGE, inside the build container. build.sh fell through to its rust default in every language job, so python and js jobs built rust binaries that then failed bad_build_check (.pkg architecture probe). 🛠 - Generated by Copilot
📚 Documentation Health ReportGenerated on: 2026-05-03 03:40:45 UTC 📈 Documentation Statistics
🏗️ Three-Tree Architecture Status
🔍 Quality Metrics
This report is automatically generated by the Documentation Automation workflow. |
…modules for Python harnesses
📚 Documentation Health ReportGenerated on: 2026-05-04 17:51:27 UTC 📈 Documentation Statistics
🏗️ Three-Tree Architecture Status
🔍 Quality Metrics
This report is automatically generated by the Documentation Automation workflow. |
…r failures in CFLite
📚 Documentation Health ReportGenerated on: 2026-05-04 19:01:21 UTC 📈 Documentation Statistics
🏗️ Three-Tree Architecture Status
🔍 Quality Metrics
This report is automatically generated by the Documentation Automation workflow. |
…heck - build_js.sh: replace 'npm ci' with 'npm install --no-audit --no-fund' to tolerate lockfile drift in 513-tiered-notification-service. - build_python.sh: add 'LLVMFuzzerTestOneInput' comment marker to the generated wrapper so OSS-Fuzz bad_build_check recognizes the PyInstaller-backed Atheris targets as fuzz targets.
📚 Documentation Health ReportGenerated on: 2026-05-05 00:11:51 UTC 📈 Documentation Statistics
🏗️ Three-Tree Architecture Status
🔍 Quality Metrics
This report is automatically generated by the Documentation Automation workflow. |
…types files in PyInstaller
📚 Documentation Health ReportGenerated on: 2026-05-06 01:10:07 UTC 📈 Documentation Statistics
🏗️ Three-Tree Architecture Status
🔍 Quality Metrics
This report is automatically generated by the Documentation Automation workflow. |
…g-infrastructure # Conflicts: # .github/workflows/matrix-folder-check.yml # codecov.yml # scripts/build/Detect-Folder-Changes.ps1
📚 Documentation Health ReportGenerated on: 2026-05-06 04:43:04 UTC 📈 Documentation Statistics
🏗️ Three-Tree Architecture Status
🔍 Quality Metrics
This report is automatically generated by the Documentation Automation workflow. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #453 +/- ##
=======================================
Coverage 32.41% 32.41%
=======================================
Files 40 40
Lines 5902 5902
=======================================
Hits 1913 1913
Misses 3989 3989
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds multi-language fuzzing infrastructure to edge-ai using ClusterFuzzLite, cargo-fuzz, Atheris, and Jazzer.js, wired into a PR-only workflow with per-component Codecov flags.
continue-on-error: true) until two clean sprintsRelated Issue
Closes #150
Type of Change
Implementation Details
22 files / 860 insertions:
.clusterfuzzlite/): Dockerfile,build.sh, language-specific build helperscflite_pr.yml(SHA-pinned to82652fb4...); fuzz outputs added toDetect-Folder-Changes.ps1andmatrix-folder-check.ymlv4.5.0(e28ff129...) using OIDC (id-token: write)parse_telemetryfor501-rust-telemetry(libfuzzer-sys 0.4.12, nightly-2026-04-01)505,506,509,510Wave 2 components run with
continue-on-error: trueuntil two clean sprints.Testing Performed
parse_telemetryran 55.02s without crashesValidation Steps
.clusterfuzzlite/scaffold and language build scriptscflite_pr.ymltriggers only on PR and uses SHA-pinned actionscontinue-on-error: trueChecklist
Security Review
Not triggered — no changes under
deploy/,SECURITY.md, orsrc/000-cloud/010-security-identity/. New CI workflow uses SHA-pinned third-party actions and OIDC for Codecov.Additional Notes
🧪 - Generated by Copilot
Note: Python 3.11 (temporary)
ClusterFuzzLite Python harnesses run on Python 3.11, not 3.12:
atheris3.0.0 (latest) only supports Python 3.8–3.11.gcr.io/oss-fuzz-base/base-builder-rust:v1base image, which is Ubuntu 20.04 (focal). ThedeadsnakesPPA on focal does not shippython3.12-dev/python3.12-venv, so 3.12 cannot be installed there even if atheris supported it.