draft: Skip poll based on buffer fullness

[tomgi]

Hide whitespace changes for much cleaner diff

---

Draft to get feedback before looking to raise a PR at https://github.com/que-rb/que

---

Diff on top of our existing changes: wrap_job_in_rails_executor_and_poll-interval-variance...seek-pass-oss:que:wrap_job_in_rails_executor_and_poll-interval-variance_and_skip_poll_based_on_buffer_fullness

---

Context

During high-throughput spike periods, Que's job polling mechanism can lead to very high database load with continuous polling, even when the job buffer is already nearly full and workers are actively processing jobs.

What is possible to be happening currently:

• locker accepts enough jobs via listen to fill the buffer
• the jobs are super fast, so in the few milliseconds in between wait and next poll we have already worked a few
• poller polls for a very few jobs to fill the buffer fully
• locker accept more jobs via listen immediately (and potentially displaces any polled low-importance jobs)
• workers work a few jobs again
• the previous poll was satisfied?, so poller polls again immediately (potentially for the same low-importance jobs again)

Changes

Add optional skip-poll-when-buffer-above-threshold CLI parameter defaulting to 1.0.

When the buffer already has at least skip-poll-when-buffer-above-threshold fraction of its capacity filled, the polling is skipped for that iteration.

Backward-compatibility

Currently, the polling is skipped only when the buffer is completely full.

When the new skip-poll-when-buffer-above-threshold parameter is not provided, it defaults to 1.0 so there's no change compared to the current behaviour.

[lee-treehouse]

is 0..11 12 or 11? I think it is 12 right?

[tomgi]

.. vs ...

> (0..100).to_a[0..11]
=> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]

> (0..100).to_a[0...11]
=> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

This spec is hugely copy-pasted inspired by the one above

que/spec/que/locker_spec.rb

Lines 422 to 423 in a58d350

	# Should have locked first 11 only.
	assert_equal ids[0...11], locked_ids

[tomgi]

No idea what it does, but 🙈 since it hangs without it and 5 other tests also have it 🤷‍♂️

I'm going to follow the existing pattern and resist shaving this yak 🙂

Probably something to do with this mutex

que/lib/que/job_buffer.rb

Line 215 in 1d66a70

@cv.wait(mutex)

that push signals

que/lib/que/job_buffer.rb

Lines 248 to 252 in 1d66a70

	def _push(item)
	Que.assert(waiting_count > 0)
	@items << item
	@cv.signal
	end

[lee-treehouse]

🎖️ I think this is a great idea

[tomgi]

Opened a draft in que-rb#441