feat: per-flow-type concurrency + granular autoProvision + env-aware pathwayMode#75
Merged
feat: per-flow-type concurrency + granular autoProvision + env-aware pathwayMode#75
Conversation
Apply deno fmt to three files that accumulated formatting drift — unrelated to the pathways config work that follows in subsequent commits. Keeps deno fmt --check green in CI. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Pump concurrency was hardcoded to 1 event at a time per flow type, blocking
real-world throughput. Introduce PumpConcurrencyConfig accepting either a
shared numeric default or per-flow-type overrides keyed by flow type name.
Usage:
startPump({ concurrency: 4 }) // shared
startPump({ concurrency: { default: 2, byFlowType: { ... } } })
Omitted flow types fall back to the shared default (or 1 when unset), so the
feature is additive and backward compatible. Breaking-change marker only
reflects the option-surface type widening.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Auto-provisioning was all-or-nothing — each of dataCore, flowType, and eventType stages ran together, making it impossible to manage resources with a mix of pre-provisioned and managed items. Add optional skip flags that short-circuit the create/update path while still resolving resource ids for downstream stages: - skipDataCore: fetches the data core id but never writes it; rethrows the existing missing-data-core error if the core doesn't exist. - skipFlowTypes: reads the flow-type list so event types can still resolve their parent id; errors loudly if a named flow type is missing. - skipEventTypes: short-circuits the event type loop entirely (and the flow-type list fetch too, when both are skipped). Ownership semantics from fragment 79a08d39-e7b5-44ec-a3c0-262f292a1b22 still hold — the description-missing errors from the non-skip path still fire when a skip flag is set but the resource is required. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two correlated changes that together make Pathways usable across Next.js /
Vercel and traditional cluster-mode deployments without footguns:
1. AutoProvisionConfig replaces the all-or-nothing defaultAutoProvision
boolean. Toggle each of dataCore/flowType/eventType/pathway
independently; most deployments want resources on + pathway off, which
is the new default. startPump({ autoProvision: ... }) accepts the same
object for per-call override.
2. pathwayMode now defaults to "managed" in production and "virtual" in
development/test. Virtual cluster mode breaks serverless runtimes
(port 9090 collisions, instrumentation hook not firing, non-leader pod
timeouts — see solutions 27e83406-8d57-4e95-a464-7df9bf0b9d06 and
8780b9c4-ec26-4921-8a6f-23ca7e8ee0a3). Managed routes event delivery
through the control plane and is safe everywhere.
defaultAutoProvision is kept as a deprecated alias that maps to the new
shape; existing apps keep working. The "!" marks the behavioral default
change — prod deployments that relied on pathwayMode being "virtual"
must now set it explicitly.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…urrency Runtime Defaults section now shows the env-aware pathwayMode table (production → managed, development/test → virtual), explains why managed is the default in production, and demonstrates granular AutoProvisionConfig with a deprecated-mapping note for defaultAutoProvision. Adds a new Pump Concurrency section with shared-default and per-flow-type override examples, and notes the per-call autoProvision override at startPump() call time. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
jbiskur
force-pushed
the
feat/command-poller
branch
from
36d95e0 to
de04e56
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
startPump()via newPumpConcurrencyConfig(was hardcoded to 1).autoProvisionon builder +startPump()via newAutoProvisionConfig(dataCore/flowType/eventType/pathway). LegacydefaultAutoProvision: booleankept as deprecated alias.pathwayModedefault — production →managed, development/test →virtual. Managed is the safe default for serverless runtimes (Next.js/Vercel/Cloudflare Workers).skipDataCore/skipFlowTypes/skipEventTypesto support id-resolution-only passes.Breaking changes
pathwayModedefault flips fromvirtual→managed. ExplicitpathwayMode: "virtual"still wins.startPump()concurrency may now exceed 1 per flow type — handlers should be idempotent.Test plan
deno fmt --checkgreendeno lintgreendeno check src/mod.tsgreendeno test -A --ignore=tests/postgres-pathway-state.test.ts— all green