📖 Scorecard v6: OSPS Baseline conformance proposal and 2026 roadmap

[justaugustus]

What kind of change does this PR introduce?

Documentation: Scorecard v6 proposal and 2026 roadmap.

Scorecard v6 evolves Scorecard from a scoring tool to an open source security
evidence engine. The primary initiative for 2026 is adding
OSPS Baseline conformance evaluation as the
first use case that proves this architecture.

Mission: Scorecard produces trusted, structured security evidence for the
open source ecosystem.

Scorecard accepts diverse inputs about a project's security practices,
normalizes them through probe-based analysis, and packages the resulting
evidence in interoperable formats for downstream tools to act on. Check scores
(0-10) and conformance labels (PASS/FAIL/UNKNOWN) are parallel evaluation
layers over the same probe evidence, produced in a single run. v6 is additive —
existing checks, probes, scores, and output formats are preserved.

The goal of this PR is to create a collaboration/decision-making nexus for
Scorecard and WG ORBIT tooling maintainers to ensure that we build interfaces
that easily interact with other tools and minimize duplication of work across
our maintainers and others in the OpenSSF ecosystem.

Key changes that warrant a major version:

1. New evaluation layer — conformance labels alongside existing check scores
2. Framework-agnostic architecture — probe evidence can be composed against
   OSPS Baseline or other frameworks via pluggable mapping definitions
3. Interoperable output formats — in-toto, Gemara, OSCAL Assessment Results
   alongside existing JSON and SARIF
4. Probe catalog as public interface — probe definitions become a consumable
   artifact for external tools

Key design decisions addressed in the proposal:

• Architecture — Scorecard owns probe execution and conformance evaluation;
  interoperability at the output layer only

• Predicate strategy — New scorecard.dev/evidence/v0.1 predicate type
  (framework-agnostic, probe-based evidence) alongside preserved
  scorecard.dev/result/v0.1 (existing check-based scores)

• Unified framework abstraction — Checks and OSPS Baseline both use the
  same internal probe composition interface

• Attestation mechanism — Phase 1 focuses on automatically verifiable
  controls only; attestation design deferred to Phase 3

• Enforcement detection boundary — Scorecard detects signals of enforcement
  (e.g., "SCA tool configured," "SAST results required") but does not itself
  enforce policies

• PR title follows the guidelines defined in our pull request documentation

What is the current behavior?

Scorecard produces 0-10 check scores and structured probe findings. There is no
OSPS Baseline conformance evaluation capability and no public 2026 roadmap.

What is the new behavior (if this is a feature change)?

This PR adds documentation only (no code changes):

• docs/ROADMAP.md — Public 2026 roadmap with phased delivery plan

• openspec/changes/osps-baseline-conformance/proposal.md — Detailed proposal
  covering architecture, scope, phased delivery, and ecosystem positioning

• openspec/changes/osps-baseline-conformance/decisions.md — Reviewer feedback,
  maintainer responses, and decision rationale

• Tests for the changes have been added (for bug fixes/features)

N/A — documentation only.

Which issue(s) this PR fixes

NONE

Special notes for your reviewer

The proposal (proposal.md) is self-contained and includes all key design
decisions inline. The decisions.md file provides supplementary context with
the full reviewer feedback log and detailed decision rationale.

The control-by-control coverage analysis
is maintained separately.

Feedback from Eddie Knight (ORBIT WG TSC Chair), Adolfo García Veytia (AMPEL),
Mike Lieberman, Felix Lange, Spencer Schrock, and Adam Korczynski has been
incorporated. See decisions.md for the complete feedback log and responses.

Does this PR introduce a user-facing change?

NONE

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 69.67%. Comparing base (353ed60) to head (1889ff5).
⚠️ Report is 337 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4952      +/-   ##
==========================================
+ Coverage   66.80%   69.67%   +2.87%     
==========================================
  Files         230      251      +21     
  Lines       16602    15654     -948     
==========================================
- Hits        11091    10907     -184     
+ Misses       4808     3873     -935     
- Partials      703      874     +171     

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[justaugustus]

@ossf/scorecard-maintainers @ossf/scorecard-fe-maintainers @eddie-knight @puerco @evankanderson @mlieberman85 — based on conversations from this week with various WG ORBIT-adjacent maintainers, I'm tossing this early draft up for review.

Feel free to comment away while I work through this!

[eddie-knight]

Hey @justaugustus, thanks for leading this collaboration! Looking forward to hammering this out.

Some things to clarify:

1. Regarding mappings between Baseline catalog<->Scorecard checks, it is possible to easily put that into a new file with Scorecard maintainers as codeowners, pending approval on that proposal from baseline maintainers.
2. There is not an "OSPS output format," and even the relevant Gemara schemas (which are quite opinionated) are still designed to support output in multiple output formats within the SDK, such as SARIF. I would expect that you'd keep your current output logic, and then maybe add basic Gemara json/yaml as another option.
3. There is a stated goal of not duplicating the code from the plugin ossf/pvtr-github-repo-scanner, but the implementation plan as it's currently written does require duplication. In the current proposal, there would not be a technical relationship between the two codebases.
4. There is cursory mention of a scorecard catalog extraction, which I'm hugely in favor of, but I don't see an implementation plan for that.

An alternative plan would be to for us to spend a week consolidating checks/probes into the pvtr plugin (with relevant CODEOWNERS), then update Scorecard to selectively execute the plugin under the covers.

This would allow us to:

• Extract the Scorecard control catalog for independent versioning and still easily connect it to the code
• Instantly integrate Gemara to scorecard
• Allow Scorecard to optionally run the existing Baseline checks from Scorecard
• Allow LFX Insights and other pvtr users to optionally run Scorecard checks from Privateer
• Simplify contribution overhead for each individual scorecard check
• And also improve the quality of both codebases through shared logic

[justaugustus]

@eddie-knight — Thanks for the thoughtful response and yes! Looking forward to working on this with you all!

I've integrated your feedback as open questions in 2ee759f.

Can you take a quick look and see if that commit accurately captures your questions and concerns before I continue?

[puerco]

This is really cool to see. Some initial thoughts.