Remove the manager and implement decentralized cluster architecture#367
Merged
Remove the manager and implement decentralized cluster architecture#367
Conversation
There was a problem hiding this comment.
Pull request overview
This PR removes the manager component and implements a decentralized cluster architecture where edge and cloud nodes are responsible for keeping the database schema consistent using the remote object store as the single source of truth. This is a significant architectural change that simplifies the cluster deployment model.
Key Changes:
- Removed the
modelardb_managercrate and all manager-related code - Implemented
Clusterstruct in the server to handle cluster operations between peer nodes - Enhanced SQL parser to support
TRUNCATE CLUSTERandVACUUM CLUSTERsyntax - Moved cluster metadata management from manager to the
DataFoldermodule - Updated Docker Compose configuration to run multiple edge and cloud nodes without a manager
Reviewed changes
Copilot reviewed 35 out of 36 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| crates/modelardb_server/src/cluster.rs | New file implementing cluster operations for peer-to-peer communication |
| crates/modelardb_storage/src/data_folder/cluster.rs | Moved cluster metadata management from manager to DataFolder |
| crates/modelardb_storage/src/parser.rs | Added support for CLUSTER keyword in TRUNCATE and VACUUM statements |
| crates/modelardb_server/src/remote.rs | Added get_flight_info() and cluster-aware table operations |
| crates/modelardb_server/src/data_folders.rs | Updated to create Cluster instead of connecting to manager |
| crates/modelardb_embedded/src/operations/client.rs | Simplified client to connect directly to any node URL |
| docs/user/README.md | Updated documentation to reflect decentralized architecture |
| docker-compose-cluster.yml | Modified to run 2 edge and 2 cloud nodes without manager |
| crates/modelardb_manager/* | Removed entire manager crate |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
chrthomsen
approved these changes
Nov 30, 2025
skejserjensen
approved these changes
Dec 11, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #363 by removing the manager and instead implementing a decentralized cluster architecture where we only have edge and cloud nodes. These nodes are responsible for keeping the database schema consistent themselves and use the remote object store as the single source of truth.
It should be noted that this PR includes the main changes that allow the cluster architecture to work and removes the manager crate. Some optimizations and features have not been implemented yet but are mentioned in #366 as future work.
The following is a non-exhaustive list of the changes that have been made in this PR to make it easier to get an overview:
Clusterstruct in the server that is responsible for all cluster related functionality such as communicating with peer nodes and handling database initialization.get_flight_info()from manager remote to server remote.TRUNCATE CLUSTERandVACUUM CLUSTERsyntax and supporting truncating and vacuuming the entire cluster.OperationsAPI.