chore(playwright): migrate E2E tests from cypress to playwright#2258
Open
wreality wants to merge 14 commits intochore/cypress-reorgfrom
Open
chore(playwright): migrate E2E tests from cypress to playwright#2258wreality wants to merge 14 commits intochore/cypress-reorgfrom
wreality wants to merge 14 commits intochore/cypress-reorgfrom
Conversation
Move cypress tests from client/test/cypress to a top-level cypress/ directory with its own package.json so cypress can be installed and run independently of the Quasar client build. - New cypress/ package with its own eslint config and cypress.config.cjs - Remove cypress deps and eslint overrides from client/ - Drop test/cypress from client/tsconfig.json exclude - Update root package.json, .lando.extras.yml, e2e workflow and docs to reference the new cypress/ path
- Bump cypress to v15 in cypress/package.json and lockfile - Fix browser launch config in cypress.config.cjs for v15 - Resolve SSL and node engine errors in the cypress container - Update do-test-e2e workflow for the new cypress version
Add a lint-cypress job that runs yarn lint in the new cypress/ package on PRs touching cypress files, and wire a cypress path filter into the changes job.
Introduces a per-worker test isolation mechanism so multiple parallel test workers can mutate the database concurrently without interfering with each other. Each worker calls setupTestToken with a unique token, which creates shadow copies of every seeded table prefixed with that token. A request middleware reads an X-Test-Token header and routes all queries through the shadow tables for the duration of the request. Components: - TableSnapshot — drops + recreates shadow tables for a token using CREATE TABLE LIKE + INSERT INTO SELECT, behind a per-token MySQL advisory lock. - IntegrationTestingMiddleware — reads X-Test-Token, calls DB::setTablePrefix() for the request lifetime, and resets the prefix in finally so it never leaks across requests. - GraphQL mutations: setupTestToken, teardownTestToken, seedSubmission, and a renamed-in-place ResetDatabase resolver that now also cleans up stale shadow tables before migrate:fresh --seed. - ResetDatabaseMutationTest + TestIsolationTest cover the new flows. The existing resetDatabase mutation name is preserved so the legacy Cypress suite continues to work; only the resolver class moves into the IntegrationTesting namespace. The client GraphQL schema is regenerated to expose the new mutations.
The Playwright suite needs a few more deterministic fixtures than the
current Cypress suite uses:
- A dedicated "settings test user" account whose profile/settings can
be mutated freely without affecting tests that use regularUser.
- Submission 113 ("Details Test Submission") for the details page
assignment-management and invitation flows.
- Submission 114 ("Inline Comment Test Submission") which mirrors
submission 100's overall + inline comment seed so the inline-comment
modification tests can mutate comments without disturbing the read-
only review tests on submission 100.
- A third publication ("Pilcrow Test Publication Setup") with the
publication admin and editor pre-attached, used by the
publication-setup admin tests.
- Style criteria seeded for publication 3 in addition to publication 1.
OverallCommentSeeder is also generalised to take a submission id rather
than hard-coding 100, since it now seeds both submissions 100 and 114.
The notification popup uses a Quasar q-popup-proxy which renders as a QMenu above its 450px breakpoint and as a QDialog below it. There is a known Firefox-specific bug in QMenu mode where the popup never renders visually after the model is set to true (quasarframework/quasar#16085); the button reports aria-expanded="true" but no content appears. Force the popup-proxy into QDialog mode on Firefox by setting the breakpoint to a value above any reasonable viewport. Other browsers keep the normal popup layout.
Bootstrap the /playwright workspace alongside the existing /cypress suite while we migrate. The two suites coexist until the migration is finished. This commit just sets up tooling — no fixtures, helpers, or specs yet. - package.json with @playwright/test, eslint + eslint-plugin-playwright, typescript + vue-tsc, and dev scripts: test, test:chromium, test:firefox, test:webkit, test:headed, test:ui, test:report, lint, lint:fix, typecheck, validate. - playwright.config.ts: three projects (chromium, firefox, webkit) at a shared 1280x720 viewport, fullyParallel false, 5 workers, retain-on- failure traces, generous per-project timeout for webkit. - tsconfig.json with @fixtures and @helpers path aliases. - eslint.config.mjs using typescript-eslint + the flat playwright plugin config, with project-specific rules. - .gitignore for node_modules, test-results, blob-report, screenshots. - patches/playwright-core+1.58.2.patch: removes the windowPosition passed to the recorder/inspector launch. The reposition-after-mapping race corrupts WSLg's display surface (see python/cpython#132430 for the same race pattern). Applied via patch-package + a postinstall hook in package.json.
- tests/global-setup.ts: runs once before all workers and calls
the resetDatabase mutation to do migrate:fresh --seed and clean up
any leftover shadow tables from previous runs.
- tests/fixtures/base.ts: extends the playwright test with three
fixtures:
* loginAs(email, goto, options) — logs in via the GraphQL API,
captures cookies from the API request context, injects them into
the browser context, navigates to the goto path, and verifies the
header username matches. Per-email session cache speeds up
subsequent calls; { noCache: true } opts out for tests that log
out or otherwise destroy the session.
* resetDatabase() — calls setupTestToken to recreate the worker's
shadow tables before the test.
* seedSubmission(options) — calls the seedSubmission mutation to
create an isolated test submission for the worker.
Also wires the page fixture to add an X-Test-Token header to every
request to the backend so the IntegrationTestingMiddleware routes
queries to the worker's shadow tables, and injects a CSS init
script that disables animations + transitions.
- tests/helpers/a11y.ts: thin wrapper around @axe-core/playwright
that fails the test on violations and excludes a few rules that
Quasar trips on (button-name, landmark-complementary).
- tests/helpers/graphql.ts: waitForGQLOperation, userSearch, and a
seedSubmission helper used directly from the fixture.
- tests/helpers/quasar.ts: qSelectOpen, qSelectItems, expectNotification.
- tests/helpers/users.ts: SEEDED_USERS map (email -> {password, username})
with the canonical seeded credentials.
- tests/fixtures/test.txt: a small file used by the draft upload spec.
Port the Cypress E2E suite to Playwright. Tests are organised by area and consolidated where the previous Cypress structure had many small tests covering related flows. - a11y.spec.ts — guest home + dashboard accessibility - account.spec.ts — profile + settings (uses settingsUser) - admin/publication-setup.spec.ts — access control, user assignment, style criteria management, basic settings, content blocks, publication creation/validation (uses publication 3) - admin/users.spec.ts — admin user management access control + details - auth.spec.ts — login form, registration, header user menu, logout, dirty guard - notifications/comments.spec.ts — overall comment, overall comment reply, and inline comment reply notification recipients - notifications/popup.spec.ts — notification dropdown read/dismiss - reviews-page.spec.ts — coordinator + reviewer reviews page - submissions/create.spec.ts — submission creation form validation - submissions/details.spec.ts — submission details, assignment management, status change comments in activity (uses submission 113) - submissions/draft.spec.ts — draft content, file upload, preview/submit - submissions/index.spec.ts — submissions table, draft visibility, export and status actions - submissions/review.spec.ts — overall comment interactions, comment ownership, status management, footnote scrolling, scroll-to-comments - submissions/review-inline.spec.ts — inline comment interactions and modification (uses submission 114)
Wires the new Playwright suite into the existing reusable workflow pattern alongside the cypress job. The e2e-playwright job runs after build-images and consumes the version output, mirroring how the cypress job is invoked. The playwright path filter is added to the changes job so the workflow can be skipped on PRs that don't touch /playwright (or any of the dependencies the e2e tests exercise).
Add a Playwright subsection to the Integration Tests (E2E) section in docs/developers/testing.md, alongside the existing Cypress documentation, plus a Playwright (TypeScript) subsection under Code Style & Linting that documents the lint, lint:fix, typecheck, and validate yarn scripts.
wreality
force-pushed
the
chore/playwright
branch
from
1b2e3ae to
f7f3678
Compare
Add a 'playwright' entry to .lando.extras.yml so developers can opt into a fully containerized Playwright setup alongside the existing Cypress extra. The service uses the official mcr.microsoft.com/playwright:v1.58.2-noble image which ships with chromium, firefox, webkit, and all required system libraries preinstalled — no host install-deps dance needed. Tooling commands exposed (cwd-independent): - lando playwright # raw CLI passthrough - lando playwright-test # full suite, all browsers - lando playwright-test-chromium # chromium only - lando playwright-test-firefox # firefox only - lando playwright-test-webkit # webkit only - lando playwright-lint # ESLint - lando playwright-typecheck # tsc --noEmit - lando playwright-validate # lint + typecheck docs/developers/testing.md: rewrite the Playwright subsection with two parallel flows (Lando recommended, host alternative), call out the headed/UI mode limitations inside containers, and note the host node_modules pitfall when switching from host to Lando.
wreality
force-pushed
the
chore/playwright
branch
5 times, most recently
from
c179e0e to
fe6a350
Compare
Three related CI workflow tweaks for the playwright e2e job: 1. The workflow installs only chromium (`--with-deps chromium`) but the playwright config defines three projects (chromium, firefox, webkit). Running `npx playwright test` with no filter would try to launch all three and fail on the missing firefox/webkit binaries. Pin the job to `--project=chromium`. Firefox and webkit coverage in CI can be added in a followup. 2. Add an explicit "Rebuild Lighthouse schema cache for testing env" step that calls `lighthouse:clear-cache` + `lighthouse:cache` inside the phpfpm container before the tests run. This guarantees the @show-gated integration-testing mutations (resetDatabase / setupTestToken / teardownTestToken / seedSubmission) are present in the cached schema, regardless of what environment the cache might have been written in previously. 3. ci.env had leading whitespace on every value (`APP_ENV= testing`) which docker-compose tolerates but is inconsistent with normal env-file syntax. Strip the leading spaces for clarity.
…local caches
Two related fixes to how the backend image is built:
1. The upstream php:8.4-fpm base image ships with the default
`clear_env = yes` in the www pool config, which causes php-fpm to
strip all environment variables before invoking PHP scripts. Inside
Laravel this means `getenv('APP_ENV')` returns empty and the env()
helper falls back to the 'production' default in config/app.php,
regardless of what the container is actually launched with.
That made the @show directive on our integration-testing mutations
(@show(env: ["local", "testing"])) always hide those fields in
container deployments, even when the container was started with
APP_ENV=testing. The legacy Cypress suite didn't trip it because
it doesn't use any @show-gated mutations.
Add `clear_env = no` to zz-docker.conf so env vars flow through.
Lando's base php image already overrides this, which is why the
bug only surfaced under the CI stack image.
2. backend/.dockerignore did not exclude bootstrap/cache/, so any
Laravel caches sitting in a developer's working copy
(lighthouse-schema.php in particular) would be COPY'd into the
built image. Those caches are frozen against the dev-time APP_ENV
and would override the schema resolution at runtime. Ignore them,
except for .gitignore itself.
wreality
force-pushed
the
chore/playwright
branch
from
fe6a350 to
418e16f
Compare
wreality
force-pushed
the
chore/cypress-reorg
branch
from
7467695 to
4ac72b7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
the database without interfering with each other.
the transition; the `resetDatabase` mutation name is preserved.
Based on `chore/cypress-reorg`.
Backend
every seeded table using `CREATE TABLE LIKE` + `INSERT INTO SELECT`,
behind a MySQL advisory lock.
and calls `DB::setTablePrefix()` for the request lifetime, resetting
the prefix in `finally` so it never leaks.
`seedSubmission`. The existing `resetDatabase` mutation is preserved
(its resolver class moves into the `IntegrationTesting` namespace and
now also cleans up leftover shadow tables).
and a third publication with style criteria for the admin tests.
`tests/Api/ResetDatabaseMutationTest`.
Client
QDialog mode to work around
quasarframework/quasar#16085
where the popup renders invisibly in QMenu mode on Firefox.
Playwright
`tsconfig.json`, `eslint.config.mjs`, `.gitignore`, plus a
`patch-package` patch that drops `windowPosition` from the
Playwright Inspector launch (reposition-after-mapping race corrupts
WSLg's display surface, same pattern as
python/cpython#132430).
1280x720 viewport. Per-project webkit timeout bump to 60s.
caching per email (Laravel file sessions are shared across workers).
`{ noCache: true }` opts out for tests that log out.
worker's shadow tables.
backend requests get `X-Test-Token`.
helpers, seeded user map.
previously-missing scenarios (comment notification recipients, scroll
to overall-comments buttons, reviewer access before acceptance).
CI
`testing.yml` alongside the existing Cypress job. A `playwright`
path filter is added so the job can be skipped on unrelated PRs.
Docs
the existing Cypress docs, plus a TypeScript lint subsection under
Code Style & Linting documenting the new yarn scripts.
Test plan