Avoid regressions by reviewing features before status advances

[ddbeck]

Summary

Some features advance to Baseline low or high with late-breaking problems, forcing us to make status regression or editorial override decisions with limited deliberation or investigation.

We ought to have a process to identify and review rising features shortly before their headline status advances from non-Baseline to Baseline low and from Baseline low to high. This would help us avoid rushed work and unnecessary status regressions.

Background

Most features' statuses are calculated from mdn/browser-compat-data (BCD). Often, recognizing an advancing feature happens because of the BCD upgrade: the reviewer (i.e., me) sees that the headline status has changed in the dist file.

There have been a few a last-minute change to a feature, typically made at the time we upgrade BCD and recalculate statuses (or just after). This has caused us to make last-minute changes to features; see #3228 for a list of examples.

We have only one case that I know of—anchor positioning (see #3558)—where we worked proactively to control a feature from advancing erroneously.

Related issues

• Baseline status for Anchor positioning #3558
• Status change reconsiderations #3228
• Ping owners on BCD bump PRs which regress a baseline status and add the regression to release notes #2726
• Baseline status regressions should require a note #1971

[ddbeck]

Here's my proposal for handling this:

We ought to have a process where, a weeks before a feature is expected to advance, we file a go/no-go issue for a rising feature with the question “Do you know of any reason, such as an incompatibility, bug, or breaking specification change, that this feature should not advance?”

This would initiate a notice period. Then, a week or two before advancing, affirmatively resolve the go/no-go issue. That issue would also become a priority discussion item for the WebDX meetings.

For example, for a Basline low→high feature:

1. Five weeks before we expect that feature to go Baseline high, file an issue calling for review of that status change.
2. Put out a "call for attention" on that issue by putting it on the WebDX meeting agenda and linking to it in the WebDX Matrix chat.
3. At the WebDX meeting that falls one to two weeks before the anticipated day of advancement, discuss the comments (if any) on the go/no-go issue.
4. Resolve the issue or delegate the work to prevent the advancement, either with an editorial override or upstream contribution.

For non-Baseline→low features, we should anticipate a shorter notice period that starts as soon as we might know when that feature is about to ship in the last browser. In many cases this would be less than a month (i.e., we'd only know it's going to ship in the immediately preceding beta release).

If we're unable to come to a consensus on a go/no-go, then we may buy time with a temporary editorial override, freezing the status quo for a few weeks. If no resolution occurred after some time limit, we could allow the freeze to expire and follow the data. A policy of expiring status freezes would prevent features from staying in a confused state, as is the situation for the features listed in #3228. In those cases, we might prefer to get real-world developer feedback where the decision is too difficult to resolve on our own.

This would require a number of pieces of work to implement

• Process: Commitment from the WebDX chairs to include these items on the discussion agenda
• Code: A script and GitHub Actions workflow to generate issues; a script to freeze/unfreeze feature statuses.
• Documentation: A checklist for go/no issues (e.g., to search for BCD bugs and PRs)