Skip to content

[improve][broker]add active status into cursor stats#17884

Merged
Jason918 merged 1 commit intoapache:masterfrom
HQebupt:addActive2CurosrStat
Oct 1, 2022
Merged

[improve][broker]add active status into cursor stats#17884
Jason918 merged 1 commit intoapache:masterfrom
HQebupt:addActive2CurosrStat

Conversation

@HQebupt
Copy link
Contributor

@HQebupt HQebupt commented Sep 29, 2022

Motivation

The active state indicates whether a cursor is active. And it affects the cache hit rate, because inactive cursors will be evicted from the entry cache. This active state helps to troubleshoot issues with low cache hit rate.
Also it is meaningful for configuring a suitable values for managedLedgerCursorBackloggedThreshold in ServiceConfiguration

# Configure the threshold (in number of entries) from where a cursor should be considered 'backlogged'
# and thus should be set as inactive.
managedLedgerCursorBackloggedThreshold=1000

Modifications

Add active status into cursor stats.
It will get the following sample data when getting the internalStats of a partitioned topic.

{
    "entriesAddedCounter": 6,
    "numberOfEntries": 10302152,
    "totalSize": 63786554161,
    "currentLedgerEntries": 6,
    "currentLedgerSize": 31182,
    "lastLedgerCreatedTimestamp": "2022-09-29T14:48:52.825+08:00",
    "waitingCursorsCount": 0,
    "pendingAddEntriesCount": 0,
    "lastConfirmedEntry": "1359228:5",
    "lastIndex": 1165055506,
    "state": "LedgerOpened",
    "ledgers": [
        {
            "ledgerId": 1356881,
            "entries": 50000,
            "size": 309705120,
            "offloaded": false,
            "underReplicated": false
        }
    ],
    "cursors": {
        "cg_test_map_cache1": {
            "markDeletePosition": "1356881:3399",
            "readPosition": "1356881:3400",
            "waitingReadOp": false,
            "pendingReadOps": 0,
            "messagesConsumedCounter": -10298746,
            "cursorLedger": -1,
            "cursorLedgerLastEntry": -1,
            "individuallyDeletedMessages": "[]",
            "lastLedgerSwitchTimestamp": "2022-09-29T14:48:52.859+08:00",
            "state": "NoLedger",
            "numberOfEntriesSinceFirstNotAckedMessage": 1,
            "totalNonContiguousDeletedMessagesRange": 0,
            "subscriptionHavePendingRead": false,
            "subscriptionHavePendingReplayRead": false,
            "active": false,
            "properties": {
                "index": 846056487
            }
        },
        "cg_test_map_cache2": {
            "markDeletePosition": "1359208:79",
            "readPosition": "1359228:6",
            "waitingReadOp": false,
            "pendingReadOps": 0,
            "messagesConsumedCounter": -8,
            "cursorLedger": 1359234,
            "cursorLedgerLastEntry": 1,
            "individuallyDeletedMessages": "[]",
            "lastLedgerSwitchTimestamp": "2022-09-29T14:48:52.859+08:00",
            "state": "Open",
            "numberOfEntriesSinceFirstNotAckedMessage": 15,
            "totalNonContiguousDeletedMessagesRange": 0,
            "subscriptionHavePendingRead": false,
            "subscriptionHavePendingReplayRead": false,
            "active": true,
            "properties": {
                "index": 1165055008
            }
        }
    },
    "schemaLedgers": [],
    "compactedLedger": {
        "ledgerId": -1,
        "entries": -1,
        "size": -1,
        "offloaded": false,
        "underReplicated": false
    }
}

Verifying this change

  • Make sure that the change passes the CI checks.

This change is a trivial rework / code cleanup without any test coverage.

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API: (no)
  • The schema: (no)
  • The default values of configurations: (no)
  • The wire protocol: (no)
  • The rest endpoints: (no)
  • The admin cli options: (no)
  • Anything that affects deployment: (no)

Documentation

Check the box below or label this PR directly.

Need to update docs?

  • doc-required
  • doc-not-needed
  • doc
  • doc-complete

Matching PR in forked repository

PR in forked repository: HQebupt#1

@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Sep 29, 2022
@HQebupt
Copy link
Contributor Author

HQebupt commented Sep 29, 2022

@AnonHxy @codelipenghui @Jason918 @Technoboy- @mattisonchao PTAL thanks. The matching PR pass the tests.

@AnonHxy
Copy link
Contributor

AnonHxy commented Sep 30, 2022

LGTM

@HQebupt
Copy link
Contributor Author

HQebupt commented Oct 1, 2022

/pulsarbot ready-to-test

@HQebupt
Copy link
Contributor Author

HQebupt commented Oct 1, 2022

/pulsarbot run-failure-checks

1 similar comment
@HQebupt
Copy link
Contributor Author

HQebupt commented Oct 1, 2022

/pulsarbot run-failure-checks

@Jason918 Jason918 added this to the 2.12.0 milestone Oct 1, 2022
@Jason918 Jason918 merged commit 8c1152c into apache:master Oct 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-not-needed Your PR changes do not impact docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants