[v26.1.x] ct/l0/gc: collect stranded L0 objects after all cloud topics deleted#30283
Open
vbotbuildovich wants to merge 1 commit intoredpanda-data:v26.1.xfrom
Open
[v26.1.x] ct/l0/gc: collect stranded L0 objects after all cloud topics deleted#30283vbotbuildovich wants to merge 1 commit intoredpanda-data:v26.1.xfrom
vbotbuildovich wants to merge 1 commit intoredpanda-data:v26.1.xfrom
Conversation
max_gc_eligible_epoch returned std::nullopt when the partition snapshot was empty, causing try_to_collect to report no_collectible_epoch after listing. L0 objects left over from previously deleted topics stayed in the bucket forever. Fall through with snap_revision (controller last applied offset) as the watermark: this is the zero-iteration case of the existing join, which already starts at snap_revision and walks it down per partition. Safety rests on invariants from the existing algorithm: - get_partitions ensures that the topic table snapshot is in sync with the controller stm (i.e. snap revision) - L0 objects never have an epoch less than any constinuent topic's initial revision ID. So with an empty topic table at snapshot revision N, any _new_ data must have an epoch > N. Therefore any L0 objects w/ epoch <= N is safe to GC. Includes a ducktape regression test: - pause GC - accumulate some L0 objects - delete the only cloud topic - resume GC - assert that orphaned objects drain. Without the fixt the test times out w/ stuck objects. Signed-off-by: Oren Leiman <oren.leiman@redpanda.com> (cherry picked from commit 0eee6a8)
Member
|
@dotnwat I think you didn't get a chance to sign off on the PR into dev. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport of PR #30200