Conversation
- Wait for dialog service command registration in waitForAppReady so Help menu and PAPI platform.about run after handlers exist (dock can paint first). - Assert About via dock tab text; use theme-toggle testid and longer theme waits. - Add aria-label and data-testid on toolbar theme button for stable selection. Made-with: Cursor
Playwright types env as string values only; destructure the variable out of process.env instead of setting it to undefined. Made-with: Cursor
- Declare paranextDialogServiceCommandsReady on globalThis; assign in dialog service - waitForAppReady uses Reflect.get (no type assertions); omit ELECTRON_RUN_AS_NODE with eslint note - Regenerate papi.d.ts after global type change Made-with: Cursor
…oltip Made-with: Cursor
tjcouch-sil
left a comment
There was a problem hiding this comment.
Thanks for looking into this! Just one piece of feedback
@tjcouch-sil reviewed 7 files and all commit messages, and made 2 comments.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on jolierabideau).
src/renderer/services/dialog.service-host.ts line 415 at r1 (raw file):
// The dock layout can paint before this async service finishes; menu items that invoke // `platform.about` and similar commands must not run until registration completes (E2E reads this). globalThis.paranextDialogServiceCommandsReady = true;
I don't really like the idea of adding globalThis stuff like this just for E2E testing; feels jank. What if, in waitForAppReady, instead you called the command rpc.discover to check if the dialog's request handlers were registered? I dunno if you are already connected to the WebSocket at that point in the E2E process; I'm not familiar enough with them. Just an idea.
Another option we could consider is running a network event when all the services are done loading in index.tsx (and I guess making a request handler that tells you whether the renderer is done loading). That would also require being able to connect to the WebSocket in the E2E process before waitForAppReady. I would naturally think we would want to do it in the app service. I guess you could run the event from the renderer and add an event handler in the app service.
…This Poll PAPI rpc.discover from Node until command:platform.about is registered. Removes paranextDialogServiceCommandsReady from renderer and global typings. Made-with: Cursor
Made-with: Cursor
jolierabideau
left a comment
There was a problem hiding this comment.
@jolierabideau made 1 comment.
Reviewable status: 3 of 7 files reviewed, 1 unresolved discussion (waiting on tjcouch-sil).
src/renderer/services/dialog.service-host.ts line 415 at r1 (raw file):
Previously, tjcouch-sil (TJ Couch) wrote…
I don't really like the idea of adding
globalThisstuff like this just for E2E testing; feels jank. What if, inwaitForAppReady, instead you called the commandrpc.discoverto check if the dialog's request handlers were registered? I dunno if you are already connected to the WebSocket at that point in the E2E process; I'm not familiar enough with them. Just an idea.Another option we could consider is running a network event when all the services are done loading in
index.tsx(and I guess making a request handler that tells you whether the renderer is done loading). That would also require being able to connect to the WebSocket in the E2E process beforewaitForAppReady. I would naturally think we would want to do it in the app service. I guess you could run the event from the renderer and add an event handler in the app service.
Done. I went with the first option!
tjcouch-sil
left a comment
There was a problem hiding this comment.
Nice! I have some suggestions, but it looks good overall!
@tjcouch-sil reviewed 4 files and all commit messages, made 5 comments, and resolved 1 discussion.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on jolierabideau).
src/renderer/services/dialog.service-host.ts line 415 at r1 (raw file):
Previously, jolierabideau wrote…
Done. I went with the first option!
Thanks!
e2e-tests/fixtures/helpers.ts line 10 at r2 (raw file):
/** Keep in sync with `GET_METHODS` in `src/shared/data/rpc.model.ts`. */ const GET_METHODS = 'rpc.discover';
Did you try importing like import { GET_METHODS } from '../../src/shared/data/rpc.model';?
e2e-tests/fixtures/helpers.ts line 268 at r2 (raw file):
} type OpenRpcDiscoverResult = {
Can you import this from here?
e2e-tests/fixtures/helpers.ts line 276 at r2 (raw file):
* a connection, sends one request, waits for the matching response id, then closes. */ async function sendPapiRequestOnce<T>(
Can this be refactored to have less duplication with sendPapiCommand?
Extract sendPapiJsonRpcOnce for shared open/send/response handling; wire sendPapiCommand and sendPapiRequestOnce through it. Keep sendPapiCommand deprecated per E2E policy. Align GET_METHODS and OpenRpc with shared sources; add e2e tsconfig paths for @shared imports. Made-with: Cursor
jolierabideau
left a comment
There was a problem hiding this comment.
@jolierabideau resolved 3 discussions.
Reviewable status:complete! all files reviewed, all discussions resolved.
Share elapsed time between dock-layout wait and PAPI registration so sequential steps cannot sum to 2× the caller timeout. Made-with: Cursor
Reorder optional timeoutErrorMessage before defaulted params so ESLint passes. Update sendPapiCommand and sendPapiRequestOnce call sites. Made-with: Cursor
tjcouch-sil
left a comment
There was a problem hiding this comment.
Nice improvement! Sadly there is another thing to look at
@tjcouch-sil reviewed 3 files and all commit messages, and made 2 comments.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on jolierabideau).
e2e-tests/tsconfig.json line 13 at r4 (raw file):
"noEmit": true, "resolveJsonModule": true, "baseUrl": "..",
Are these baseUrl and paths required anymore? Unfortunate timing; I just put up a PR to remove baseUrl because it is apparently going away.
Fix flaky / failing smoke E2E tests (Windows CI and local)
Summary
Stabilizes the Playwright smoke project (
npm run test:e2e:smoke) by aligning tests with real DOM and startup ordering, waiting for real PAPI registration viarpc.discover(no rendererglobalThisflag), hardening the theme toggle test, fixing Electron launchenvtyping for Playwright, tightening theme-toggle assertions so slow updates fail fast, and typing browser globals used insidepage.evaluate.Problem
ui-interaction.spec.tsfailed on “About via PAPI” (wrong locator) and “toggle theme” (theme id unchanged / timing). Help-menu About could fail on first run and pass on retry.command:platform.aboutnot found, while the dock was already visible.Root causes
Dock vs. dialog service timing — The React tree (including the dock layout) can paint before
startDialogService()finishes registering commands such asplatform.about.waitForAppReadyonly waited for the dock, so UI tests could open the Help menu before the handler existed.About PAPI assertion —
getByLabel('%mainMenu_about%')does not match how tabs expose accessibility:PlatformTabTitleusesaria-labelfrom%tab_aria_tab%(“Tab”), not the about menu key. The reliable signal is the same as the Help-menu test: a.dock-tabwhose text matches/About/i.Theme toggle — Icon-only CSS selectors (
lucide-sun/lucide-moon) and asserting before the theme stylesheet node existed were brittle on CI. The stylesheet node must exist before asserting ondata-theme-id; stable selection usesdata-testid.Electron
env—ELECTRON_RUN_AS_NODE: undefinedis invalid for Playwright’senvtype (Record<string, string>). The variable must be omitted (destructure it out ofprocess.env), not set toundefined.E2E TypeScript —
e2e-testsusedlib: ["ES2022"]only, sodocumentinsidepage.evaluatefailed type-checking; addingDOMfixes that without hacks in each test.Changes
E2E readiness —
rpc.discoverinstead of a renderer globallaunchElectronAppalready waits until the PAPI WebSocket accepts connections on port 8876, so the test process can open a short-lived client from Node anytime after that.waitForAppReady(ine2e-tests/fixtures/helpers.ts):rpc.discover(GET_METHODS/ OpenRPC) untilresult.methodsincludescommand:platform.about, with the same overall timeout as the dock wait.That uses the same method registry as the live app (renderer registrations included) and avoids test-only
globalThishooks or extra typings inpapi.d.ts.Supporting helpers:
sendPapiRequestOnce(JSON-RPC wheremethodis a PAPI request type),waitForPapiMethodRegistered(exported for reuse).Renderer / typings — removed E2E-only global
dialog.service-host.ts— No longer setsglobalThis.paranextDialogServiceCommandsReadyafter registration.src/shared/global-this.model.ts/lib/papi-dts/papi.d.ts— RemovedparanextDialogServiceCommandsReadydeclarations.E2E helpers (
e2e-tests/fixtures/helpers.ts)waitForAppReady— Dock wait +rpc.discoverpolling forcommand:platform.about(see above).launchElectronApp— Buildenvby destructuringELECTRON_RUN_AS_NODEout ofprocess.env, then setNODE_ENV: 'development', so the variable is removed without passingundefined.E2E TypeScript (
e2e-tests/tsconfig.json)DOMtocompilerOptions.libsodocumentand other browser APIs inpage.evaluatecallbacks type-check.Smoke UI tests (
e2e-tests/tests/smoke/ui-interaction.spec.ts).dock-tab+/About/i) instead ofgetByLabel('%mainMenu_about%').#theme-stylesto exist; locate the button withgetByTestId('theme-toggle');scrollIntoViewIfNeededbefore click; use 5stoPasstimeouts for toggle and restore so slow theme updates surface as failures instead of being masked by a long window.Toolbar (
platform-bible-toolbar.tsx)Button:aria-label={themeButtonTooltip}(matches existing tooltip copy) anddata-testid="theme-toggle"for stable E2E selection and clearer accessibility.%tooltip_theme_loading_error%→%toolbar_theme_loading_error%.Files touched
src/renderer/services/dialog.service-host.tsglobalThisreadiness flagsrc/shared/global-this.model.tsparanextDialogServiceCommandsReadylib/papi-dts/papi.d.tse2e-tests/fixtures/helpers.tswaitForAppReadyviarpc.discover+ Electronenvfixe2e-tests/tsconfig.jsonDOMlib forpage.evaluate/documente2e-tests/tests/smoke/ui-interaction.spec.tssrc/renderer/components/platform-bible-toolbar.tsxaria-label,data-testid, localization fixHow to verify
Confirm smoke passes on first run without retries (especially Help → About and theme toggle).
Notes
EBUSY/ temp user-data cleanup warnings during teardown; those are separate from the assertion failures addressed here.rpc.discoverpolling uses small delays between attempts so we do not hammer the WebSocket while the renderer is still registering handlers.toPassstays at 5s on purpose: if the theme data provider regresses and updates take many seconds, the test should fail loudly.This change is