Skip to content

Make DataDrip job queue name configurable#12

Merged
arnauvp-factorial merged 5 commits intomainfrom
parameterize_queue_name
Feb 12, 2026
Merged

Make DataDrip job queue name configurable#12
arnauvp-factorial merged 5 commits intomainfrom
parameterize_queue_name

Conversation

@heyrun
Copy link
Copy Markdown
Contributor

@heyrun heyrun commented Feb 11, 2026

🚪 Why?

The Dripper and DripperChild jobs previously used a hardcoded queue name. This PR makes the queue configurable via DataDrip.queue_name (mattr_accessor) or the DATA_DRIP_QUEUE environment variable.

We would need to set DATA_DRIP_QUEUE=within_24_hours in the backend ConfigMap.

Copy link
Copy Markdown

@arnauvp-factorial arnauvp-factorial left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Makes the job queue name configurable via DataDrip.queue_name or DATA_DRIP_QUEUE, and uses a block form of queue_as so the value is resolved at enqueue time. Tests cover configuration and dynamic resolution for both Dripper and DripperChild. Specs pass.


Suggestions

1. Breaking change: default queue

Previously:

  • Dripper used :data_drip
  • DripperChild used :data_drip_child

With this PR, when DATA_DRIP_QUEUE is not set, both use :default. So:

  • Existing apps that rely on queue names for routing/monitoring will see jobs move to default unless they set DATA_DRIP_QUEUE.
  • There is no way to keep the old "two queues" behavior (parent vs child) with a single config.

Options:

  • Document this in the CHANGELOG/PR and recommend setting DATA_DRIP_QUEUE=data_drip (or the desired name) when upgrading, or
  • If you want to preserve previous behavior when not configured, you could default queue_name to ENV["DATA_DRIP_QUEUE"].presence || "data_drip" so unconfigured apps keep using :data_drip (with both jobs on that one queue).

2. Single queue for Dripper and DripperChild

Both jobs now use the same DataDrip.queue_name. If the goal is "one configurable queue for all DataDrip jobs," that’s consistent. If anyone needed parent and child on different queues, that’s no longer possible without extra code (e.g. a second config or overrides). Worth confirming this is intentional.

3. Test: default value

The example defaults to :default when DATA_DRIP_QUEUE is not set only asserts be_a(Symbol), not the actual value. To make it robust and explicit (and avoid surprises if DATA_DRIP_QUEUE is set in CI), you could do:

it "defaults to :default when DATA_DRIP_QUEUE is not set" do
  with_modified_env("DATA_DRIP_QUEUE" => nil) do
    expect(DataDrip.queue_name).to eq(:default)
  end
end

(using a helper that temporarily clears/restores the env, or around that unsets DATA_DRIP_QUEUE). That would both document and enforce the default.


Aside from the above, the implementation and use of queue_as { ... } look good.

Copy link
Copy Markdown

@arnauvp-factorial arnauvp-factorial left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see comments

@arnauvp-factorial arnauvp-factorial merged commit 35508fc into main Feb 12, 2026
16 checks passed
@arnauvp-factorial arnauvp-factorial deleted the parameterize_queue_name branch February 12, 2026 12:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants