[fix][broker] Key_Shared subscription doesn't always deliver messages from the replay queue after a consumer disconnects and leaves a backlog #5
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ead letter topic producer (apache#24020) Co-authored-by: Lari Hotari <lhotari@apache.org> (cherry picked from commit 302c1d5)
…topic is partitioned (apache#24154)" This reverts commit b619f9c.
This reverts commit ed64bf9.
…st (apache#24492) (cherry picked from commit 13e0a7b)
…yTracker with StampedLock optimistic reads (apache#24542)" This reverts commit 455a052.
…ties (apache#24532) Co-authored-by: Claude <noreply@anthropic.com>
This reverts commit 36cd2c2.
…kupAutoClusterFailover (apache#24662) (cherry picked from commit 95dcb58)
…pache#24661) (cherry picked from commit dc2ab26)
… skipDocGeneration=true (apache#24668) (cherry picked from commit a239240)
…1.0 version specific path (apache#24669) (cherry picked from commit 3b7bef1)
…avoid infinite loop when schema validation fails on an incoming message (apache#24663) Co-authored-by: Lari Hotari <lhotari@users.noreply.github.com> (cherry picked from commit 829df71)
…ntial topic policy updates (apache#24427) (cherry picked from commit e5e7981)
…ification to be ignored (apache#23977) Co-authored-by: fanjianye <fanjianye@bigo.sg> (cherry picked from commit a532798)
… lost data (apache#24665) (cherry picked from commit 90a70db)
…leted because topic GC (apache#24651) (cherry picked from commit 7c57683)
…ng messages for a topic, even though there are many subscriptions (apache#24622) (cherry picked from commit 84205eb)
…titions (apache#24679) (cherry picked from commit f795bf8)
…tions have not been created (apache#24680)
(cherry picked from commit fcd5f92)
…aramsLookupAutoClusterFailover (apache#23336) (cherry picked from commit 8fd5bf5)
(cherry picked from commit e9a9e04)
…her than FastThreadLocalThread (apache#24719)
…stCreateMissedPartitions (apache#24683)
…e of failed delete data from transaction buffer topic (apache#24648)
…ers and printed noisy error logs (apache#24634)
Signed-off-by: Zixuan Liu <nodeces@gmail.com>
…pache#24696) (cherry picked from commit 96bc370)
…tead of a record to compatible with lower version JDK
…deliver messages from the replay queue after a consumer disconnects and leaves a backlog unless new messages are produced
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
15 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes apache#23845
Motivation
There is a not covered scenario for draining hashes and key shared subscriptions at
PersistentStickyKeyDispatcherMultipleConsumers. The detailed scenario described at apache#23845 (comment)As a result draining hashes could contain entries with consumer which was stopped. This leads consuming to get stuck.
Modifications
Decrease draining hash entry
refCountin case its hash range returned to the initial owner which holds the entry in pending acks.Verifying this change
This change added tests and can be verified as follows:
org.apache.pulsar.client.api.KeySharedSubscriptionTest#testMessageDeliveredFromDrainingHashesto verify the scenarioorg.apache.pulsar.broker.service.ConsistentHashingStickyKeyConsumerSelectorTest#testShouldNotSwapExistingConsumers,org.apache.pulsar.broker.service.ConsumerHashAssignmentsSnapshotTest#testResolveConsumerRemovedHashRanges_NoChanges,org.apache.pulsar.broker.service.ConsumerHashAssignmentsSnapshotTest#testResolveConsumerUpdatedHashRanges_RangeAdded,org.apache.pulsar.broker.service.ConsumerHashAssignmentsSnapshotTest#testResolveConsumerRemovedHashRanges_RangeUpdated,org.apache.pulsar.broker.service.ConsumerHashAssignmentsSnapshotTest#testResolveConsumerUpdatedHashRanges_OverlappingRangesDoes this pull request potentially affect one of the following parts:
If the box was checked, please highlight the changes
Documentation
docdoc-requireddoc-not-neededdoc-completeMatching PR in forked repository
PR in forked repository: #4