CANCELLED check runs from superseded workflow runs cause false failures

[sirandreww-starkware]

Description

When multiple workflow runs are triggered concurrently on the same commit (e.g. by different event types), GitHub cancels the older runs as newer ones start. The cancelled jobs leave CANCELLED check runs attached to the commit. In certain timing windows, allcheckspassed picks up these stale cancelled checks and fails, even though the commit is actually green.

Root Cause

The deduplication logic in checksFilters.ts keeps the check with the highest job ID per name+app_id combination. Job IDs are assigned sequentially, so a job from a freshly-cancelled older run can have a higher ID than any job yet created by the newer, still-running replacement run.

Example: workflow run A is preempted by run B midway through. Run A's system_test_hybrid job gets cancelled (job ID 70584197559). Run B hasn't reached that stage yet, so there's no higher-ID job for system_test_hybrid. The deduplication picks the cancelled job as "most recent."

evaluateChecksStatus() treats CANCELLED as a hard failure unconditionally. With fail_fast: true (the default), it returns passed: false immediately — before even inspecting the in-progress checks from the newer run.

Reproduction

1. Have a workflow that triggers on multiple events (e.g. push + pull_request) for the same commit
2. The first triggered run starts long-running jobs
3. A second run is triggered and GitHub cancels the first run's jobs mid-flight
4. allcheckspassed runs while the second run is still in its early stages (before it has created check runs for the long-running jobs)
5. allcheckspassed sees the highest-ID check for those jobs as CANCELLED and fails

Suggested Fix

The deduplication unit should be workflow run, not job ID. When multiple check runs share the same name and app_id, the action should group them by their workflow run ID, identify the most recent run per workflow, and only evaluate checks from that run.

Concretely: use the List check runs for a Git reference endpoint's started_at field (or a separate call to the workflow runs API) to correlate checks back to their run, then pick the most recent run per workflow name before deduplicating by job ID within that run. A CANCELLED check from a superseded run would never surface because a more recent run — even if still in progress — would take precedence.

This correctly handles the case where the replacement run hasn't yet emitted a check for a given job: the in-progress checks from that run signal that evaluation should wait, rather than failing on the cancelled remnants of the preempted run.