Make DataDrip job queue name configurable#12
Conversation
arnauvp-factorial
left a comment
There was a problem hiding this comment.
Summary
Makes the job queue name configurable via DataDrip.queue_name or DATA_DRIP_QUEUE, and uses a block form of queue_as so the value is resolved at enqueue time. Tests cover configuration and dynamic resolution for both Dripper and DripperChild. Specs pass.
Suggestions
1. Breaking change: default queue
Previously:
Dripperused:data_dripDripperChildused:data_drip_child
With this PR, when DATA_DRIP_QUEUE is not set, both use :default. So:
- Existing apps that rely on queue names for routing/monitoring will see jobs move to
defaultunless they setDATA_DRIP_QUEUE. - There is no way to keep the old "two queues" behavior (parent vs child) with a single config.
Options:
- Document this in the CHANGELOG/PR and recommend setting
DATA_DRIP_QUEUE=data_drip(or the desired name) when upgrading, or - If you want to preserve previous behavior when not configured, you could default
queue_nametoENV["DATA_DRIP_QUEUE"].presence || "data_drip"so unconfigured apps keep using:data_drip(with both jobs on that one queue).
2. Single queue for Dripper and DripperChild
Both jobs now use the same DataDrip.queue_name. If the goal is "one configurable queue for all DataDrip jobs," that’s consistent. If anyone needed parent and child on different queues, that’s no longer possible without extra code (e.g. a second config or overrides). Worth confirming this is intentional.
3. Test: default value
The example defaults to :default when DATA_DRIP_QUEUE is not set only asserts be_a(Symbol), not the actual value. To make it robust and explicit (and avoid surprises if DATA_DRIP_QUEUE is set in CI), you could do:
it "defaults to :default when DATA_DRIP_QUEUE is not set" do
with_modified_env("DATA_DRIP_QUEUE" => nil) do
expect(DataDrip.queue_name).to eq(:default)
end
end(using a helper that temporarily clears/restores the env, or around that unsets DATA_DRIP_QUEUE). That would both document and enforce the default.
Aside from the above, the implementation and use of queue_as { ... } look good.
🚪 Why?
The Dripper and DripperChild jobs previously used a hardcoded queue name. This PR makes the queue configurable via DataDrip.queue_name (mattr_accessor) or the DATA_DRIP_QUEUE environment variable.
We would need to set DATA_DRIP_QUEUE=within_24_hours in the backend ConfigMap.