[Stability] Messaging E2E: chromium-msg shard fails on real-time-delivery cross-window propagation

[TortoiseWolfe]

Summary

The chromium-msg 1/1 shard of E2E Tests failed on main commit `00edd23`. Investigation of the merged-blob report (artifact ID 6651581858) shows one hard failure plus three flaky retries, all in the messaging-realtime surface. None of them are caused by the three PRs that just merged (#54 RLS test fix, #55 docs, #56 AdminGate test) — the failure surface is tests/e2e/messaging/, which those PRs do not touch.

This issue tracks it as a real symptom rather than dismissing it as flake. STATUS.md already calls this surface out as the dominant flake hotspot ("9 rounds of mitigation. Cause: stale closures, unstable hook refs, hydration timing"). The retry helpers in real-time-delivery.spec.ts have already been bumped from 3 attempts to 5 attempts (5 × 60s = 5 minute budget per assertion). They are still hitting the wall.

Hard failure

Test: `tests/e2e/messaging/real-time-delivery.spec.ts:342` — "Real-time Message Delivery (T098) > should show delivery status (sent → delivered → read)"

Error: `expect(locator).toBeVisible()` failed after 60s.

• Locator: `getByText(testMessage)` on page2
• Expected: visible
• Received: `<element(s) not found>`

Where it fails: line 85 of the `waitForMessageOnPage2` helper. The helper navigates+reloads up to 5 times waiting for the message to appear on the second window. After 5 attempts (~5 min), it gives up. Both retries (`attempt 0` followed by another `attempt 0` — Playwright's retry on failure) failed at the same line.

This is the real-time subscription on page2 not receiving the message that page1 sent. Either:

1. Supabase Realtime broadcast didn't deliver to the second client's subscription
2. The subscription was unsubscribed between page1's send and page2's render
3. The auth session on page2 lost permission to read the conversation mid-test
4. Hydration timing on page2 missed the message that arrived before subscribe()

Flaky (passed on retry)

File:line	Test	Pattern
`message-delete-placeholder.spec.ts:341`	"should show [Message deleted] placeholder and preserve adjacent messages"	initial fail → passed on retry
`message-editing.spec.ts:337` (fails at line 394)	"T115: should edit message within 15-minute window"	initial fail → passed on retry
`real-time-delivery.spec.ts:310` (fails at line 85)	"should deliver message in <500ms between two windows"	initial fail → passed on retry

The fact that two of the three flaky failures are at line 85 (same helper) and the hard failure is also at line 85 strongly suggests one root cause across all four: cross-window realtime propagation under CI.

What's been ruled out

The recent merges to main do not touch this surface:

• PR fix(rls): payment RLS verified — correct stale audit-log assertion #54 — `tests/rls/service-role.test.ts` only (RLS test contract).
• PR docs(stability): record post-#44 state, add cross-issue tracking plan #55 — `STATUS.md` + `docs/STABILITY-TRACKING.md` (docs only).
• PR test(admin): extract AdminGate, pin safety properties via regression test #56 — `src/app/admin/{layout.tsx,AdminGate.tsx,AdminGate.test.tsx}` (admin path; no messaging code).

The previous main commit (`62f8a40`) had a green E2E run, so the hard failure either appeared recently or is intermittent enough to slip through. Given STATUS.md's flake history, intermittent is more likely than regression-from-merge.

Plan

1. Confirm flake-vs-regression by rerunning the failed shard once the in-flight run completes. If rerun is green, this is in the existing flake budget. If rerun is also red, escalate to root-cause analysis.
2. Pull the actual trace.zip for the specific failure (the merged blob doesn't include per-test traces; need the raw shard artifact). Inspect the page2 timeline: does Realtime ever connect? Does it receive the INSERT event? Does it filter the event out?
3. Cross-reference `useConversationRealtime.ts` and `useTypingIndicator.ts` — STATUS.md / tracking-doc lists this as the resolved-but-watch-for-regression hotspot. Verify the `useMemo(() => createClient(), [])` wrapper is still in place and the subscription teardown isn't leaking between the two test windows.
4. Decide whether the helper's 5-attempt × 60s retry budget is the right answer or whether it's masking a real subscription bug. The comment at line 72-76 implies the bump from 3→5 was already a mitigation, not a fix. We may have hit the wall on what retries can absorb.

Acceptance

Either:

• (a) The rerun is green AND the hard failure is reproducible <30% of the time AND we add a Watch entry in docs/STABILITY-TRACKING.md Family A. Stays open as a watch.
• (b) Root cause identified, fix landed, hard failure reproduces 0/10 reruns. Closes.

Reference

• Failed run: https://github.com/TortoiseWolfe/ScriptHammer/actions/runs/24970006226
• Failing job: `E2E (chromium-msg 1/1)`, job ID 73111466675
• Failing commit: `00edd23` (main)
• STATUS.md row: "Stability hotspot: ConversationView regression" (009 User Messaging)
• Tracking doc: `docs/STABILITY-TRACKING.md` Family A

[TortoiseWolfe]

Update: failure surface is broader than chromium-msg alone

After the full run completed (run `24970006226`), the picture changed. The full failure list:

Job	Hard-failed	Failed tests
`E2E (chromium-msg 1/1)`	1 (3 flaky)	real-time-delivery cross-window propagation
`E2E (firefox-msg 1/1)`	(TBD; failed)	likely same messaging-realtime
`E2E (webkit-msg 1/1)`	(TBD; failed)	likely same messaging-realtime
`E2E (webkit-gen 4/6)`	4	cross-page-navigation: navigate-all-pages, browser-back/forward, deep linking, theme-on-nav
`E2E (webkit-gen 5/6)`	13 (1 flaky)	theme-switching (multiple themes), mobile-touch-targets
`E2E (webkit-gen 6/6)`	12 (7 flaky)	mobile-horizontal-scroll, form-submission, mobile-navigation

The webkit-gen failures are unrelated to messaging. They span theme-switching, mobile viewports, form submission. That's webkit-on-CI fragility, not a messaging regression.

The messaging shards failing across all three browsers (chromium + firefox + webkit) is a separate signal — same test surface failing on three different engines means the issue is server-side (Supabase Realtime broadcast / Cloud rate limits) or test-fixture-side (test users colliding), not browser-specific.

Hypothesis: race-pile-up from concurrent merges

The prior main commit (`62f8a40`, dated 2026-04-26 20:28Z) had 26/26 jobs green. This commit (`00edd23`) has 6/26 failing. Three PRs merged in 30 seconds (PR #54, #55, #56 all merged 23:40-23:41Z). Two of those triggered E2E runs that GitHub then cancelled when main moved (those cancelled runs were never useful for signal). The `00edd23` run is the survivor — and it ran while leftover artifacts and Supabase test-user state from the cancelled runs were still in flight on Supabase Cloud.

That'd explain the multi-surface failure profile: not one regression, but a transient state-collision from too-fast merging. The fix isn't code; it's "don't merge three PRs in 30 seconds against a shared cloud Supabase."

Action

Manually triggered a rerun of the failed shards (`gh run rerun --failed`). Watching for outcome:

• If rerun is green: confirms environmental / state-collision hypothesis. Document in tracking doc as a merge-cadence rule.
• If rerun reproduces the failures: real regression — investigate per-surface (messaging vs webkit).

This issue stays open until the rerun gives a definitive signal. Updating with the conclusion when it lands.

[TortoiseWolfe]

Round 2 evidence: 1-vs-31 swing rules out a code regression, points to shared-state collision

The chromium-msg rerun completed with 31 hard failures, where round 1 had 1 hard failure + 3 flaky. Same commit, same config, same shard. A real code regression would produce the same failure set across reruns — this wild 1-vs-31 swing is the fingerprint of an environmental / shared-state issue, not a code bug.

What round 2 tells us

The first failing test is `complete-user-workflow.spec.ts:206 — Complete messaging workflow: sign-in -> connect -> message -> sign-out`. Failure mode:

```
TimeoutError: locator.waitFor: Timeout 45000ms exceeded.

• waiting for getByRole('tab', { name: /Connections/i }) to be visible
• 7 × waiting for "http://localhost:3000/" navigation to finish...
  • navigated to "http://localhost:3000/"
    ```

The page repeatedly navigates back to `/` after sign-in. This is the auth-loss redirect pattern STATUS.md documented as the "9 rounds of E2E flake mitigation" hotspot — `AuthContext.signOut()` does `window.location.href='/'`, so when something fires a spurious SIGNED_OUT, the navigation chain to `/` is exactly what we see.

When this happens at the first test (round 2), every subsequent test cascades because storageState is gone. When it happens mid-suite (round 1), only the unlucky test fails.

Root-cause hypothesis: cross-shard Supabase test-user collision

`pnpm test:e2e` runs chromium-msg, firefox-msg, and webkit-msg in parallel. All three shards use the same Supabase Cloud test users (`test-user-a@scripthammer.test`, etc.). When one shard's `afterAll` deletes a user while another shard's middle-of-test session is still active:

1. The active session's next API call gets `401`
2. Supabase's auth client fires `SIGNED_OUT`
3. AuthContext does `window.location.href='/'`
4. Test fails on whatever element it was waiting for

This is the same root cause as #50 (RLS fixture cleanup-stale issue), just on the E2E surface instead of the RLS surface. Both are symptoms of "test users on Supabase Cloud are not isolated per shard."

Action

This issue (#57) should be closed as a duplicate of an expanded #50. Filing a follow-up to expand #50's scope to cover both surfaces (RLS and E2E messaging) and add the per-shard-namespace fix. Closing this as supplemental evidence.

The fix isn't "rerun until green" — that's already the current de-facto policy and it bites whoever merges. The fix is:

1. Per-shard test-user namespace — `test-user-a-${shardName}@scripthammer.test` — eliminates cross-shard collisions entirely
2. Or: serialize the messaging shards — run chromium-msg, firefox-msg, webkit-msg in series instead of parallel. Slower but isolated.

Pinging #50 to absorb this scope.

[TortoiseWolfe]

Filed PR #58 to address a separate issue from this one — the cascade ECONNREFUSED storm in firefox/webkit-gen shards on the same CI run was caused by silent wait-on failures (the Start server step proceeding to Playwright when serve hadn't bound). #58 makes that step fail loud + capture diagnostics. Should reduce noise on future runs so the actual #57/#50 cross-shard messaging issue can be diagnosed without 13 unrelated cascade-failures cluttering the signal.

[TortoiseWolfe]

PR #58 merged (commit 5de242a). The PR's own E2E run was 26/26 green — including the firefox/webkit/chromium-msg shards that were red yesterday. So:

• The diagnostic-loud fix is safe (success path unchanged; all 26 jobs pass).
• The 14/26 failure on commit 00edd23 did not reproduce on b43b9cb (= 00edd23 + workflow diagnostics). That's strong evidence the original cascade was environmental (serve-binding flake on the GitHub runner side), not a code regression.

The chromium-msg messaging-realtime issue tracked here (#57) also didn't reproduce on this run. With the noise from the firefox/webkit cascade now eliminated by #58, future failures here will be easier to diagnose.

Leaving #57 open. The 1-fail / 31-fail swing pattern from yesterday's reruns is still real — just dormant on this run. If it recurs, the next CI failure will produce cleaner signal.