-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Implement Request and Response Policy Based Routing in Cluster Mode #3422
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Implement Request and Response Policy Based Routing in Cluster Mode #3422
Conversation
* Add search module builders and tests * Add tests
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com>
6e3b627 to
1b2eaa6
Compare
ndyakov
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Submitting partial review for the aggregators.
ndyakov
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Submitting another partial review.
ndyakov
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Final part of initial review
Overview:
- Let's use atomics when possible.
- Left questions related to the node selection and setting of values.
Overall the design of the solution looks good, would have to do an additional pass over the test files once this review is addressed.
Thank you both @ofekshenawa and @htemelski-redis!
727a799 to
14bd6e1
Compare
ndyakov
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
left some comments related to aggregators
ndyakov
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The aggregators look good, there are some prints left in the code as bunch of unanswered questions. Let's resolve them before merging this. cc @ofekshenawa , @htemelski-redis
a4ac8df to
7181bcc
Compare
* feat: load the policy table in cluster client * Remove comments
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds support for explicit routing policies in Redis Cluster operations by introducing a command policy resolver system. The changes enable commands to be routed to specific shards based on their requirements (all shards, multiple shards, single shard) and aggregate responses accordingly.
- Implements a comprehensive routing and aggregation framework for cluster commands
- Adds Clone() methods to various command types to support concurrent execution
- Introduces configurable shard picker strategies (round-robin, random, static)
- Adds extensive test coverage for routing policies and command aggregation
Reviewed Changes
Copilot reviewed 30 out of 46 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| osscluster_router.go | New file implementing command routing logic and response aggregation |
| osscluster.go | Integration of command policy resolver and routing infrastructure |
| command_policy_resolver.go | New resolver for determining command routing policies |
| internal/routing/*.go | Core routing policy types and aggregation implementations |
| internal/util/atomic_*.go | Atomic min/max utilities for response aggregation |
| *_commands.go | Added cmdType fields and Clone() methods to command structs |
| osscluster_test.go | Extensive test coverage for routing policies and aggregation |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
htemelski-oss
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good
…ch-commands-to-shards
…regation for different step
…ch-commands-to-shards
ndyakov
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I do think we can merge this as experimental feature and release it in a bet next week. will have to sync with one other PR (#3560)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Copilot reviewed 30 out of 46 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
ndyakov
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks good, missing a README about this change, would you mind adding and marking it as Experimental in the options comment? Other than this, left one comment regarding the new dependency that we are going to have. I do think we can implement the atomic.Float64 functionality without the need to pull the whole package.
@ofekshenawa let me know what you think and if you would like to merge it as it is, we can release the beta as it is and resolve the dependency before the GA release.
| require go.uber.org/atomic v1.11.0 | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i personally do not like that we are pulling the whole package just for the Float64 atomic type, since its implementation is ~150 lines (that we actually don't need and ca implement here with Uint64 and some math.
This PR introduces support for Redis COMMAND-based request_policy and response_policy routing for Redis commands when used in OSS Cluster client.
Key Additions:
Command Policy Loader: Parses and caches COMMAND metadata with routing/aggregation tips on first use.
Routing Engine Enhancements:
Implements support for all request policies: default(keyless), default(hashslot), all_shards, all_nodes, multi_shard, and special.
Response Aggregator: Combines multi-shard replies based on response_policy:
all_succeeded, one_succeeded, agg_sum, special, etc.
Includes custom handling for special commands like FT.CURSOR.
Raw Command Support: Policies are enforced on Client.Do(ctx, args...).