feat: add flow compilation verification at worker startup

[jumski]

Flow Compilation at Worker Startup

This PR adds automatic flow compilation and verification during worker startup. When a flow worker starts, it now ensures the flow is properly compiled in the database before processing any tasks.

Key features:

• Adds ensureFlowCompiled method to the Queries class to check flow compilation status
• Implements flow compilation during worker startup in the FlowWorkerLifecycle class
• Adds environment detection to determine compilation mode (development vs production)
• Introduces FlowShapeMismatchError for handling flow structure mismatches
• In development mode (local Supabase), mismatched flows are automatically recompiled
• In production mode, mismatches throw errors to prevent unexpected behavior

The PR includes comprehensive integration tests covering:

• Compiling new flows on worker startup
• Verifying existing flows with matching structure
• Handling mismatches in production mode (throws error)
• Recompiling mismatched flows in development mode

This ensures flows are always properly compiled before execution, preventing runtime errors from mismatched flow definitions.

[changeset-bot]

⚠️ No Changeset found

Latest commit: c5ef186

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[jumski]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• feat: add advisory locks to prevent concurrent flow compilation race conditions #477
• refactor: remove ensure-compiled endpoint and related code from control plane #476
• refactor: simplify environment detection for flow compilation mode #475
• feat: add flow compilation verification at worker startup #474 👈 (View in Graphite)
• feat: add secure local development detection for flow compilation #473
• refactor: use Supabase service role key for worker authentication #472
• feat: include options in FlowShape for proper flow creation #471
• feat: add ensure-compiled endpoint to control plane server #467
• feat: add ensure_flow_compiled function with dev/prod mode support #466
• feat: add flow compilation utilities for creating flows from JSON shapes #465
• feat: add SQL functions to compare and retrieve flow shapes #464
• Setup Worker Startup Compilation Phase 1 #463
• main

---

How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

• merge:queue - adds this PR to the back of the merge queue
• hotfix:queue - for urgent hot fixes, skip the queue and merge this PR next

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

[nx-cloud]

View your CI Pipeline Execution ↗ for commit c5ef186

Command	Status	Duration	Result
nx affected -t verify-exports --base=origin/mai...	✅ Succeeded	3s	View ↗
nx affected -t build --configuration=production...	✅ Succeeded	24s	View ↗
nx affected -t lint typecheck test --parallel -...	✅ Succeeded	3m 51s	View ↗
nx run edge-worker:e2e	✅ Succeeded	2m 39s	View ↗
nx run cli:e2e	✅ Succeeded	3s	View ↗

---

☁️ Nx Cloud last updated this comment at 2025-12-01 09:44:10 UTC

[github-actions]

🔍 Preview Deployment: Website

✅ Deployment successful!

🔗 Preview URL: https://pr-474.pgflow.pages.dev

📝 Details:

• Branch: 12-01-add_flow_compilation_at_worker_startup_with_sql_direct_call
• Commit: f630938839fd8e9608874aea1ef70b6f1260050c
• View Logs

_Last updated: _