-
Notifications
You must be signed in to change notification settings - Fork 618
📖 Scorecard v6: OSPS Baseline conformance proposal and 2026 roadmap #4952
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
28 commits
Select commit
Hold shift + click to select a range
4851dd4
:seedling: Add OpenSpec scaffolding for PVTR integration
justaugustus c0c5185
:seedling: Rename proposal from pvtr-integration to osps-baseline-con…
justaugustus 2aacfb1
:seedling: Rewrite OSPS Baseline conformance proposal to address full…
justaugustus 21c1319
:seedling: Add remaining openspec scaffolding and .gitignore update
justaugustus 7a5c7fd
:seedling: Add maintainer review feedback to OSPS conformance proposal
justaugustus 007cd74
:seedling: Address maintainer feedback on OSPS conformance proposal
justaugustus d67011b
:seedling: Add OSPS Baseline coverage analysis and public roadmap
justaugustus 4b6f19e
:seedling: Remove roadmap-ideas.md reference from public docs; add CQ…
justaugustus ea0e067
:seedling: Fix Mermaid diagram newlines in proposal
justaugustus 4723c01
:seedling: Fix Darn→Darnit references; add Minder to ecosystem compar…
justaugustus c28a674
:seedling: Map existing issues/PRs to OSPS Baseline gaps; add CQ-15
justaugustus ef670de
:seedling: Remove premature openspec scaffolding and roadmap-ideas gi…
justaugustus 307448f
:seedling: Add Allstar to ecosystem comparison and ORBIT diagram; add…
justaugustus 101b0b1
:seedling: Integrate Eddie Knight's ORBIT WG feedback into proposal
justaugustus 5bf8487
Apply suggestions from Eddie's code review
justaugustus 69424eb
:seedling: Add stakeholder annotations and decision priorities
justaugustus 5f68422
:seedling: Integrate PR feedback; elevate AMPEL; remove spec.md
justaugustus 4e09cbc
:seedling: Split proposal into proposal.md + decisions.md
justaugustus cc7c74e
:seedling: Revise proposal with evidence engine framing
justaugustus 9332599
:seedling: Add Steering Committee responses to decisions.md
justaugustus b203a86
:seedling: Update ROADMAP.md with evidence engine framing
justaugustus 9905816
:seedling: Clean up ROADMAP and remove CI gating deliverable
justaugustus 5122d87
:seedling: Fix diagrams in proposal.md
justaugustus c5c1678
:seedling: Integrate AMPEL maintainer feedback (AP-5 through AP-8)
justaugustus 0ae83ed
:seedling: Frame proposal as Scorecard v6; add user feedback
justaugustus 3d3deb7
:seedling: Integrate Spencer and Adam's PR feedback; refine predicate…
justaugustus d4596bf
:seedling: Make proposal.md self-contained; remove dependencies on de…
justaugustus 1889ff5
ROADMAP: Add Best Practices Badge automation proposals integration
justaugustus File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -64,3 +64,6 @@ newRelease.json | |
| # Ignore golang's vendored files | ||
| /vendor/ | ||
| /tools/vendor/ | ||
|
|
||
| # AI tooling instructions | ||
| AGENTS.md | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,150 @@ | ||
| # OpenSSF Scorecard Roadmap | ||
|
|
||
| ## 2026 | ||
|
|
||
| ### Theme: Scorecard v6 — Open Source Security Evidence Engine | ||
|
|
||
| **Mission:** Scorecard produces trusted, structured security evidence for the | ||
| open source ecosystem. | ||
|
|
||
| Scorecard v6 evolves Scorecard from a scoring tool to an evidence engine. The | ||
| primary initiative for 2026 is adding | ||
| [OSPS Baseline](https://baseline.openssf.org/) conformance evaluation as the | ||
| first use case that proves this architecture. Scorecard accepts diverse inputs | ||
| about a project's security practices, normalizes them through probe-based | ||
| analysis, and packages the resulting evidence in interoperable formats for | ||
| downstream tools to act on. | ||
|
|
||
| Check scores (0-10) and conformance labels (PASS/FAIL/UNKNOWN) are parallel | ||
| evaluation layers over the same probe evidence, produced in a single run. | ||
| Existing checks, probes, and scores are unchanged — v6 is additive. The | ||
| conformance layer is a new product surface aligned with the | ||
| [ORBIT WG](https://github.com/ossf/wg-orbit) ecosystem. (While Scorecard is not | ||
| part of ORBIT WG, ecosystem interoperability with ORBIT tools is an overarching | ||
| OpenSSF goal, and Scorecard interoperates through published output formats.) | ||
|
|
||
| **Target Baseline version:** [v2026.02.19](https://baseline.openssf.org/versions/2026-02-19) | ||
|
|
||
| **Current coverage:** See [docs/osps-baseline-coverage.md](osps-baseline-coverage.md) | ||
| for a control-by-control analysis. | ||
|
|
||
| ### Phased delivery | ||
|
|
||
| Phases are ordered by outcome. Maintainer bandwidth dictates delivery timing. | ||
|
|
||
| #### Phase 1: Conformance foundation and Level 1 coverage | ||
|
|
||
| **Outcome:** Scorecard produces a useful OSPS Baseline Level 1 conformance | ||
| report for any public GitHub repository, available across CLI, Action, and | ||
| API surfaces. | ||
|
|
||
| Deliverables: | ||
|
|
||
| - Evidence model and output formats: | ||
| - Enriched JSON (Scorecard-native) | ||
| - In-toto evidence predicate (`scorecard.dev/evidence/v0.1`) | ||
| - Gemara output (via [security-baseline](https://github.com/ossf/security-baseline) | ||
| dependency) | ||
| - OSCAL Assessment Results (via | ||
| [go-oscal](https://github.com/defenseunicorns/go-oscal)) | ||
justaugustus marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| - OpenSSF Best Practices Badge [Automation Proposals](https://github.com/coreinfrastructure/best-practices-badge/blob/main/docs/automation-proposals.md) | ||
| - Unified framework abstraction for OSPS Baseline v2026.02.19: | ||
| - Checks and OSPS Baseline both use the same internal probe composition interface | ||
| - Probe-to-control mappings maintained in Scorecard | ||
| - Applicability engine detecting preconditions (e.g., "has made a release") | ||
| - Map existing probes to OSPS controls where coverage exists today | ||
| - New probes for Level 1 gaps: | ||
| - Governance and documentation presence | ||
| - Dependency manifest presence | ||
| - Security policy deepening | ||
| - Secrets detection — consuming platform signals where available | ||
| - Metadata ingestion layer — Security Insights as first supported source; | ||
| architecture supports additional metadata sources | ||
| - Scorecard control catalog extraction plan | ||
justaugustus marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| #### Phase 2: Release integrity and Level 2 core | ||
|
|
||
| **Outcome:** Scorecard evaluates release-related OSPS controls, covering the | ||
| core of Level 2 and becoming useful for downstream due diligence workflows. | ||
|
|
||
| Deliverables: | ||
|
|
||
| - Release asset inspection layer | ||
| - Signed manifest support | ||
| - Release notes and changelog detection | ||
| - Evidence bundle output (conformance results + in-toto statement) | ||
justaugustus marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| - Additional metadata sources for the ingestion layer | ||
|
|
||
| **Note:** Phase 1 focuses on automatically verifiable controls. Design of | ||
| attestation mechanisms (for non-automatable controls) is deferred to Phase 3 or | ||
| beyond. | ||
|
|
||
| #### Phase 3: Enforcement detection, Level 3, and multi-repo | ||
|
|
||
| **Outcome:** Scorecard covers Level 3 controls including enforcement detection | ||
| and project-level aggregation. | ||
|
|
||
| Deliverables: | ||
|
|
||
| - SCA policy and enforcement detection (Scorecard detects enforcement mechanisms without being an enforcement tool itself) | ||
| - SAST policy and enforcement detection (Scorecard detects enforcement mechanisms without being an enforcement tool itself) | ||
| - Multi-repo project-level conformance aggregation | ||
| - Attestation mechanism for non-automatable controls (deferred from Phase 2) | ||
| - Attestation integration GA | ||
|
|
||
| ### Ecosystem alignment | ||
justaugustus marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| Scorecard operates within the ORBIT WG ecosystem as an evidence engine. All | ||
justaugustus marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| downstream tools consume Scorecard evidence on equal terms through published | ||
| output formats. | ||
|
|
||
| [Allstar](https://github.com/ossf/allstar), a Scorecard sub-project, | ||
| continuously monitors GitHub organizations and enforces Scorecard check | ||
| results as policies. OSPS conformance output could enable Allstar to enforce | ||
| Baseline conformance at the organization level. | ||
|
|
||
| Scorecard SHOULD NOT (per [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119)) | ||
| duplicate evaluation that downstream tools handle: | ||
|
|
||
| - **[Privateer](https://github.com/ossf/pvtr-github-repo-scanner)** — Baseline evaluation powered by Gemara and Security Insights | ||
| - **[Minder](https://github.com/mindersec/minder)** — Policy enforcement and remediation platform (OpenSSF Sandbox, ORBIT WG) | ||
| - **[AMPEL](https://github.com/carabiner-dev/ampel)** — Attestation-based policy enforcement; already consumes Scorecard probe results via [policy library](https://github.com/carabiner-dev/policies/tree/main/scorecard) | ||
| - **[Darnit](https://github.com/kusari-oss/darnit)** — Compliance audit and remediation | ||
|
|
||
| Scorecard's role is to produce deep, probe-based security evidence that these | ||
| tools and downstream consumers can use through interoperable output formats | ||
| (JSON, in-toto, Gemara, SARIF, OSCAL). | ||
|
|
||
| ### Design principles | ||
|
|
||
| 1. **Evidence is the product.** Scorecard's core output is structured, | ||
| normalized probe findings. Check scores and conformance labels are parallel | ||
| evaluation layers over the same evidence. | ||
| 2. **Probes normalize diversity.** Each probe understands multiple ways a | ||
| control outcome can be satisfied. | ||
| 3. **UNKNOWN-first honesty.** If Scorecard cannot observe a control, the | ||
| status is UNKNOWN with an explanation — never a false PASS or FAIL. | ||
| 4. **All consumers are equal.** Downstream tools consume Scorecard evidence | ||
| through published output formats. | ||
| 5. **No metadata monopolies.** Probes may evaluate multiple sources for the | ||
| same data. No single metadata file is required for meaningful results, | ||
| though they may enrich results. | ||
| 6. **Formats are presentation.** Output formats (JSON, in-toto, Gemara, | ||
| SARIF, OSCAL) are views over the evidence model. No single format is | ||
| privileged. | ||
|
|
||
| ### Open questions | ||
|
|
||
| The following design questions are under active discussion among maintainers: | ||
|
|
||
| - **Attestation identity model** — How non-automatable controls are attested | ||
| (repo-local metadata vs. signed attestations via Sigstore/OIDC). Decomposed | ||
| into identity (who signs) and tooling (what generates, when) sub-questions. | ||
justaugustus marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| - **Enforcement detection scope** — How Scorecard detects enforcement | ||
| mechanisms without being an enforcement tool itself | ||
|
|
||
| ### How to contribute | ||
|
|
||
| See the [proposal](../openspec/changes/osps-baseline-conformance/proposal.md) | ||
| for detailed requirements and open questions. Discussion and feedback are | ||
| welcome via GitHub issues and the Scorecard community meetings. | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.