Skip to content

[ee] fix: ignore arg values in default debounce key#8845

Closed
rubenfiszel wants to merge 1 commit intomainfrom
kafka-trigger-debounce-fix
Closed

[ee] fix: ignore arg values in default debounce key#8845
rubenfiszel wants to merge 1 commit intomainfrom
kafka-trigger-debounce-fix

Conversation

@rubenfiszel
Copy link
Copy Markdown
Contributor

@rubenfiszel rubenfiszel commented Apr 15, 2026

Summary

The bug: Jobs that should debounce into a single run were firing individually whenever the runnable's args carried any per-call variation outside debounce_args_to_accumulate. This most commonly showed up on Kafka triggers into flows with a preprocessor step + debouncing, where the preprocessor returned anything that differed per message (timestamp, kafka offset, msg id, etc.) — every message got its own job even though the user clearly intended them to collapse.

Root cause: resolve_debounce_key built the default key as

{workspace}/{flow|script}/{path}#args:<sorted, colon-joined values of every arg except the first accumulated one>

Only the first entry of debounce_args_to_accumulate was excluded. Any other arg — including fields the user had no reason to think about, like a Date.now() they returned for logging — made the key unique per call and silently broke debouncing.

Fix: The default key is now just the runnable's fully-qualified path:

{workspace}/{flow|script}/{path}

All calls to the same runnable now land in the same batch regardless of their args. The surviving job still keeps its own (latest) non-accumulated args, and debounce_args_to_accumulate is still merged across the batch at pull time. Users who want per-arg debouncing can still set an explicit debounce_key template (e.g. "pp_$args[region]").

This matches how users expect debouncing to behave — batch everything on this runnable within the delay window, then run once with the accumulated payload.

Changes

  • backend/windmill-queue/src/jobs.rs: resolve_debounce_key drops the args-based suffix in its default branch; the unused args_to_ignore_if_default parameter is removed. The explicit debounce_key template path (with $args[...] interpolation) is unchanged.
  • windmill-queue/src/jobs_ee.rs (companion EE PR): updates the two call sites in maybe_debounce and maybe_debounce_post_preprocessing to drop the removed parameter, and skips the now-unused debounce_args_to_accumulate field in their DebouncingSettings destructuring.
  • backend/windmill-queue/tests/debounce_test.rs: flips test_post_preprocessing_args_to_accumulate_different_non_accumulated (renamed to ..._non_accumulated_ignored) to assert that jobs with the same runnable but differing non-accumulated args now collapse. All other debounce tests continue to pass unchanged.

Semantic change

This is a breaking behavior change for anyone whose flow/script relied on the old "differentiate by non-accumulated args" default. They must now set an explicit debounce_key template to keep that behavior. The existing test_post_preprocessing_debounce_args_differentiation test already demonstrates the template pattern: debounce_key: Some("pp_$args[region]").

Test plan

  • cargo test -p windmill-queue --test debounce_test --features private,enterprise -- --skip stress → 50 passed
  • Live reproduction end-to-end: kafka trigger (auto_commit=false) → flow with preprocessor returning {items:[value], fixed:'const', leaked: Date.now()} → debounce_delay_s=5, debounce_args_to_accumulate=["items"]. Before the fix: 5 kafka messages → 5 separate jobs. After the fix: 5 kafka messages → 1 surviving job with items:[v1,v2,v3,v4,v5] and 4 jobs marked "Debounced by ...".
  • Same repro via webhook (5 sequential POSTs) shows identical collapsing behavior.
  • test_post_preprocessing_debounce_args_differentiation (which uses an explicit debounce_key template) still passes, confirming the custom-template path is untouched.

Generated with Claude Code


Summary by cubic

Fixes debouncing so calls to the same runnable collapse even when non-accumulated args differ. This resolves Kafka-trigger flows where preprocessor output (timestamps, offsets, IDs) made every job unique.

  • Bug Fixes

    • Default debounce key is now the runnable path only ({workspace}/{flow|script}/{path}); arg values are ignored.
    • Removed args_to_ignore_if_default from resolve_debounce_key; explicit debounce_key templates with $args[...] remain supported.
    • Updated tests to confirm non-accumulated args no longer split debounced jobs.
  • Migration

    • If you relied on the old per-arg default, set an explicit debounce_key (e.g. "pp_$args[region]").

Written for commit daab7e4. Summary will update on new commits.

The default debounce key baked all non-accumulated arg values into a
colon-joined suffix, so any varying non-accumulated field (e.g. a
timestamp, a kafka offset, or a msg id present in a preprocessor's
output) silently broke debouncing — every call got a unique key and no
jobs ever collapsed into a single debounced run.

The default key is now just the runnable's fully-qualified path. The
surviving job keeps its own (latest) non-accumulated args, and
debounce_args_to_accumulate is still merged across the batch at pull
time. Users who want per-arg debouncing can still set a custom
debounce_key template (e.g. "pp_$args[region]").

Drops the now-unused args_to_ignore_if_default parameter from
resolve_debounce_key.

Companion: windmill-labs/windmill-ee-private#<pending>

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@cloudflare-workers-and-pages
Copy link
Copy Markdown

Deploying windmill with  Cloudflare Pages  Cloudflare Pages

Latest commit: daab7e4
Status: ✅  Deploy successful!
Preview URL: https://9701764f.windmill.pages.dev
Branch Preview URL: https://kafka-trigger-debounce-fix.windmill.pages.dev

View logs

@rubenfiszel
Copy link
Copy Markdown
Contributor Author

Closing for now — deferring the default-key semantics change. The docs PR (windmill-labs/windmilldocs#1353) documenting current behavior stays open.

@github-actions github-actions bot locked and limited conversation to collaborators Apr 15, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant