Skip to content

Conversation

dt
Copy link
Member

@dt dt commented Aug 20, 2025

Backport 1/1 commits from #151950.

/cc @cockroachdb/release


Previously this could queue up a large number of small changes to flush, particularly if flushing was slower than updates, potentially causing BACKUP to hang for hours as it drains this queue (of only marginally useful information, in the case of a otherwise completed job that is just writing out the debug info).

Now all the updates that arrive over a 15s window are rolled up before being saved. If the channel becomes full while saving, additional messages may be dropped.

Release note: none.
Epic: none.
Release justification: fixes severe bug that could stall backups.

Previously this could queue up a large number of small changes to flush,
particularly if flushing was slower than updates. Now all the updates
that arrive over a 15s window are rolled up before being saved. If the channel
becomes full while saving, additional messages may be dropped.

Release note: none.
Epic: none.
@dt dt requested a review from a team as a code owner August 20, 2025 22:47
@dt dt requested review from jeffswenson and removed request for a team August 20, 2025 22:47
Copy link

blathers-crl bot commented Aug 20, 2025

Thanks for opening a backport.

Before merging, please confirm that it falls into one of the following categories (select one):

  • Non-production code changes. Includes test-only changes, build system changes, etc.
  • Fixes for serious issues. Defined in the policy as correctness, stability, or security issues, data corruption/loss, significant performance regressions, breaking working and widely used functionality, or an inability to detect and debug production issues.
  • Other approved changes. These changes must be gated behind a disabled-by-default feature flag unless there is a strong justification not to.

Add a brief release justification to the PR description explaining your selection.

Also, confirm that the change does not break backward compatibility and complies with all aspects of the backport policy.

All backports must be reviewed by the TL and EM for the owning area.

@blathers-crl blathers-crl bot added the backport Label PR's that are backports to older release branches label Aug 20, 2025
Copy link

blathers-crl bot commented Aug 20, 2025

It looks like your PR touches production code but doesn't add or edit any test code. Did you consider adding tests to your PR?

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

Copy link

blathers-crl bot commented Aug 20, 2025

✅ PR #152210 is compliant with backport policy

Confidence: high
Critical bug criteria met: [Significant performance regressions]
Backward compatible: true
Explanation: The PR contains a change to a production file (pkg/backup/backup_job.go) but qualifies for an exemption from the standard backport policy requirements because it includes a 'Release justification: fixes severe bug that could stall backups' section. This justification aligns with the CockroachDB Backport Policy under the 'critical bug criteria,' specifically addressing a significant performance issue potentially leading to system stalling. The PR details and change indicate this fix is critical to system performance during backup operations, and thus its inclusion in a backport is justified.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@cockroach-teamcity
Copy link
Member

This change is Reviewable

@msbutler msbutler merged commit 63c5a01 into cockroachdb:release-25.2 Aug 21, 2025
16 checks passed
@dt dt deleted the backport25.2-151950 branch August 21, 2025 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport Label PR's that are backports to older release branches target-release-25.2.6
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants