|
| 1 | +# Plan: Browser Testing for @pgflow/client |
| 2 | + |
| 3 | +## Context |
| 4 | + |
| 5 | +The setTimeout binding bug (fixed in this PR) was not caught by existing tests because: |
| 6 | + |
| 7 | +1. **Unit tests use fake timers** - Vitest's `vi.useFakeTimers()` mocks setTimeout, hiding context binding issues |
| 8 | +2. **Integration tests run in Node.js** - Node.js is permissive with function context, browsers are strict |
| 9 | +3. **No browser-environment testing** - All tests run in Node.js, never in actual browser environments |
| 10 | + |
| 11 | +The bug only manifests in real browsers when `setTimeout` loses its `window`/`globalThis` context. |
| 12 | + |
| 13 | +## Goal |
| 14 | + |
| 15 | +Add browser-based testing to catch browser-specific issues like: |
| 16 | +- Function context binding (setTimeout, setInterval, requestAnimationFrame) |
| 17 | +- DOM API compatibility |
| 18 | +- Web API behavior (Fetch, WebSocket, etc.) |
| 19 | +- Browser-specific quirks |
| 20 | + |
| 21 | +## Solution: Vitest Browser Mode |
| 22 | + |
| 23 | +Use Vitest's built-in browser mode to run tests in real browsers (Chromium, Firefox, Safari). |
| 24 | + |
| 25 | +### Why Vitest Browser Mode? |
| 26 | + |
| 27 | +- ✅ Runs tests in **actual browsers**, not Node.js |
| 28 | +- ✅ Uses Playwright/WebdriverIO under the hood |
| 29 | +- ✅ Integrates seamlessly with existing Vitest setup |
| 30 | +- ✅ Supports parallel execution with Node.js tests |
| 31 | +- ✅ CI/CD friendly (headless mode) |
| 32 | +- ✅ No separate test framework needed |
| 33 | + |
| 34 | +## Implementation Plan |
| 35 | + |
| 36 | +### Phase 1: Setup Infrastructure |
| 37 | + |
| 38 | +**1. Install dependencies** |
| 39 | +```bash |
| 40 | +pnpm add -D @vitest/browser playwright |
| 41 | +``` |
| 42 | + |
| 43 | +**2. Update vitest.workspace.ts** |
| 44 | + |
| 45 | +Add browser test configuration alongside existing Node.js config: |
| 46 | + |
| 47 | +```typescript |
| 48 | +// vitest.workspace.ts |
| 49 | +import { defineWorkspace } from 'vitest/config' |
| 50 | + |
| 51 | +export default defineWorkspace([ |
| 52 | + './vitest.config.ts', // existing Node.js tests |
| 53 | + { |
| 54 | + test: { |
| 55 | + name: 'browser', |
| 56 | + include: [ |
| 57 | + 'pkgs/**/*.browser.test.ts', |
| 58 | + 'apps/**/*.browser.test.ts', |
| 59 | + ], |
| 60 | + browser: { |
| 61 | + enabled: true, |
| 62 | + provider: 'playwright', |
| 63 | + instances: [ |
| 64 | + { browser: 'chromium' }, |
| 65 | + ], |
| 66 | + headless: true, // CI-friendly |
| 67 | + }, |
| 68 | + }, |
| 69 | + }, |
| 70 | +]) |
| 71 | +``` |
| 72 | + |
| 73 | +**3. Add npm scripts** |
| 74 | + |
| 75 | +```json |
| 76 | +{ |
| 77 | + "scripts": { |
| 78 | + "test:browser": "vitest --workspace --project browser", |
| 79 | + "test:browser:ui": "vitest --workspace --project browser --ui", |
| 80 | + "test:all": "vitest --workspace" |
| 81 | + } |
| 82 | +} |
| 83 | +``` |
| 84 | + |
| 85 | +### Phase 2: Migrate setTimeout Binding Test |
| 86 | + |
| 87 | +**1. Rename test file** |
| 88 | +``` |
| 89 | +pkgs/client/__tests__/SupabaseBroadcastAdapter.setTimeout-binding.test.ts |
| 90 | +→ |
| 91 | +pkgs/client/__tests__/SupabaseBroadcastAdapter.setTimeout-binding.browser.test.ts |
| 92 | +``` |
| 93 | + |
| 94 | +**2. Simplify test** (no mocking needed in browser) |
| 95 | + |
| 96 | +```typescript |
| 97 | +import { describe, it, expect } from 'vitest' |
| 98 | +import { SupabaseBroadcastAdapter } from '../src/lib/SupabaseBroadcastAdapter.js' |
| 99 | +import { createClient } from '@supabase/supabase-js' |
| 100 | + |
| 101 | +describe('SupabaseBroadcastAdapter - Browser Environment', () => { |
| 102 | + const supabaseUrl = 'https://test.supabase.co' |
| 103 | + const supabaseKey = 'test-key' |
| 104 | + |
| 105 | + it('should handle setTimeout without losing context', async () => { |
| 106 | + // In a real browser, if setTimeout binding is broken, this will throw: |
| 107 | + // "TypeError: 'setTimeout' called on an object that does not implement interface Window" |
| 108 | + |
| 109 | + const supabase = createClient(supabaseUrl, supabaseKey) |
| 110 | + const adapter = new SupabaseBroadcastAdapter(supabase, { |
| 111 | + reconnectDelayMs: 100, |
| 112 | + stabilizationDelayMs: 100, |
| 113 | + }) |
| 114 | + |
| 115 | + const mockChannel = { |
| 116 | + on: vi.fn().mockReturnThis(), |
| 117 | + subscribe: vi.fn((callback) => { |
| 118 | + setTimeout(() => callback('SUBSCRIBED'), 10) |
| 119 | + return mockChannel |
| 120 | + }), |
| 121 | + unsubscribe: vi.fn(), |
| 122 | + } |
| 123 | + |
| 124 | + supabase.channel = vi.fn().mockReturnValue(mockChannel) |
| 125 | + |
| 126 | + // This should work without throwing |
| 127 | + await expect( |
| 128 | + adapter.subscribeToRun('test-run-id') |
| 129 | + ).resolves.toBeDefined() |
| 130 | + }) |
| 131 | +}) |
| 132 | +``` |
| 133 | + |
| 134 | +### Phase 3: Identify Other Browser-Critical Tests |
| 135 | + |
| 136 | +Tests that should run in browser mode: |
| 137 | + |
| 138 | +**From `@pgflow/client`:** |
| 139 | +- SupabaseBroadcastAdapter (realtime subscriptions, setTimeout/setInterval usage) |
| 140 | +- PgflowClient (browser-specific Supabase client behavior) |
| 141 | +- Any code using Web APIs (Fetch, WebSocket, localStorage, etc.) |
| 142 | + |
| 143 | +**From `apps/demo`:** |
| 144 | +- Svelte component rendering |
| 145 | +- User interactions (clicks, form submissions) |
| 146 | +- Realtime updates in the UI |
| 147 | + |
| 148 | +### Phase 4: CI/CD Integration |
| 149 | + |
| 150 | +**Update `.github/workflows/ci.yml`:** |
| 151 | + |
| 152 | +```yaml |
| 153 | +- name: Run browser tests |
| 154 | + run: pnpm nx affected -t test:browser --base=$NX_BASE --head=$NX_HEAD |
| 155 | +``` |
| 156 | +
|
| 157 | +**Playwright installation** (for CI): |
| 158 | +```yaml |
| 159 | +- name: Install Playwright browsers |
| 160 | + run: pnpm exec playwright install chromium |
| 161 | +``` |
| 162 | +
|
| 163 | +### Phase 5: Documentation |
| 164 | +
|
| 165 | +**Update `pkgs/client/README.md`:** |
| 166 | + |
| 167 | +```markdown |
| 168 | +## Testing |
| 169 | +
|
| 170 | +### Unit Tests (Node.js) |
| 171 | +```bash |
| 172 | +pnpm nx test client |
| 173 | +``` |
| 174 | + |
| 175 | +### Browser Tests |
| 176 | +```bash |
| 177 | +pnpm nx test:browser client |
| 178 | +``` |
| 179 | + |
| 180 | +Browser tests run in real browsers to catch browser-specific issues |
| 181 | +like function context binding, DOM API compatibility, etc. |
| 182 | +``` |
| 183 | +
|
| 184 | +## Test Coverage Strategy |
| 185 | +
|
| 186 | +### Node.js Tests (Current) |
| 187 | +- Fast execution |
| 188 | +- Mock-heavy |
| 189 | +- Business logic |
| 190 | +- Integration with database |
| 191 | +- Most existing tests stay here |
| 192 | +
|
| 193 | +### Browser Tests (New) |
| 194 | +- Slower execution |
| 195 | +- Minimal mocking |
| 196 | +- Browser-specific behavior |
| 197 | +- Web API compatibility |
| 198 | +- DOM interactions |
| 199 | +- Context binding issues |
| 200 | +
|
| 201 | +## Migration Strategy |
| 202 | +
|
| 203 | +**DO NOT migrate all tests to browser mode.** |
| 204 | +
|
| 205 | +Only migrate/create browser tests for: |
| 206 | +1. Browser-specific bugs (like setTimeout binding) |
| 207 | +2. DOM interactions |
| 208 | +3. Web API usage |
| 209 | +4. Visual/rendering tests (if needed in future) |
| 210 | +
|
| 211 | +Most tests should remain in Node.js for speed. |
| 212 | +
|
| 213 | +## Future Enhancements |
| 214 | +
|
| 215 | +**1. Multi-browser testing** |
| 216 | +```typescript |
| 217 | +instances: [ |
| 218 | + { browser: 'chromium' }, |
| 219 | + { browser: 'firefox' }, |
| 220 | + { browser: 'webkit' }, // Safari |
| 221 | +], |
| 222 | +``` |
| 223 | + |
| 224 | +**2. Visual regression testing** |
| 225 | +```typescript |
| 226 | +import { page } from '@vitest/browser/context' |
| 227 | + |
| 228 | +it('renders correctly', async () => { |
| 229 | + await page.screenshot({ path: 'snapshot.png' }) |
| 230 | +}) |
| 231 | +``` |
| 232 | + |
| 233 | +**3. Performance testing** |
| 234 | +```typescript |
| 235 | +it('loads within 2 seconds', async () => { |
| 236 | + const start = performance.now() |
| 237 | + await adapter.subscribeToRun('test-run-id') |
| 238 | + const duration = performance.now() - start |
| 239 | + expect(duration).toBeLessThan(2000) |
| 240 | +}) |
| 241 | +``` |
| 242 | + |
| 243 | +## Success Criteria |
| 244 | + |
| 245 | +- [ ] Vitest browser mode configured and working |
| 246 | +- [ ] setTimeout binding test runs in real browser |
| 247 | +- [ ] CI/CD runs browser tests in headless mode |
| 248 | +- [ ] Documentation updated |
| 249 | +- [ ] Team knows when to write browser vs Node.js tests |
| 250 | + |
| 251 | +## Risks & Mitigations |
| 252 | + |
| 253 | +**Risk:** Browser tests are slower than Node.js tests |
| 254 | +**Mitigation:** Only use browser mode for browser-specific tests, keep most tests in Node.js |
| 255 | + |
| 256 | +**Risk:** CI/CD needs Playwright browsers installed |
| 257 | +**Mitigation:** Add `playwright install` step to CI workflow |
| 258 | + |
| 259 | +**Risk:** Flaky tests in browser environments |
| 260 | +**Mitigation:** Use Vitest's built-in retry mechanism, proper waits/expectations |
| 261 | + |
| 262 | +## References |
| 263 | + |
| 264 | +- [Vitest Browser Mode Guide](https://vitest.dev/guide/browser/) |
| 265 | +- [Vitest Browser Mode Discussion](https://github.com/vitest-dev/vitest/discussions/5828) |
| 266 | +- [Playwright Provider](https://github.com/vitest-dev/vitest/tree/main/packages/browser) |
| 267 | + |
| 268 | +## Timeline |
| 269 | + |
| 270 | +**Estimated effort:** 4-8 hours |
| 271 | + |
| 272 | +- Phase 1 (Setup): 1-2 hours |
| 273 | +- Phase 2 (Migrate test): 1 hour |
| 274 | +- Phase 3 (Identify tests): 1-2 hours |
| 275 | +- Phase 4 (CI/CD): 1-2 hours |
| 276 | +- Phase 5 (Docs): 1 hour |
| 277 | + |
| 278 | +## Open Questions |
| 279 | + |
| 280 | +1. Should we run browser tests on every PR or only on main? |
| 281 | +2. Which browsers should we support in CI? (Chromium only vs all three) |
| 282 | +3. Do we need visual regression testing for the demo app? |
| 283 | +4. Should browser tests be required to pass for PRs to merge? |
0 commit comments