feat: deploy

Makefile

@@ -1,6 +1,10 @@
+VERSION ?= latest
+IMAGE_NAME = mokv
+IMAGE_TAG = $(IMAGE_NAME):$(VERSION)
+
 .PHONY: compile
 compile:
-	protoc internal/api/*.proto \
+	protoc ./api/*.proto \
 		--go_out=. \
 		--go-grpc_out=. \
 		--go_opt=paths=source_relative \
@@ -9,11 +13,7 @@ compile:
 
 .PHONY: test
 test:
-	go test -cover -race ./...
-
-.PHONY: start
-start:
-	go run .
+	go test -cover -race -v ./...
 
 .PHONY: build
 build:
@@ -25,4 +25,54 @@ perf:
 
 .PHONY: perf-long
 perf-long:
-	-go test -bench=. -benchtime=60s ./mokv -benchmem -run=^#
+	-go test -bench=. -benchtime=60s ./mokv -benchmem -run=^#
+
+.PHONY: docker-build
+docker-build:
+	docker build -t $(IMAGE_TAG) .
+
+.PHONY: kind-load
+kind-load:
+	kind load docker-image $(IMAGE_TAG)
+
+.PHONY: deploy
+deploy:
+	helm install mokv deploy/mokv
+
+.PHONY: upgrade
+upgrade:
+	helm upgrade mokv deploy/mokv
+
+.PHONY: redeploy
+redeploy: docker-build kind-load
+	kubectl rollout restart statefulset mokv
+
+.PHONY: clean
+clean:
+	helm uninstall mokv && \
+	kubectl delete pvc datadir-mokv-0 datadir-mokv-1 datadir-mokv-2
+
+.PHONY: logs
+logs:
+	kubectl logs -f mokv-0
+
+.PHONY: status
+status:
+	kubectl get pods -l app.kubernetes.io/name=mokv
+
+.PHONY: start
+start:
+	@echo "Starting up mokv..."
+	@kind get clusters | grep -q kind || (echo " Creating kind cluster..." && kind create cluster)
+	@echo "Building Docker image..."
+	@$(MAKE) docker-build
+	@echo "Loading image into kind..."
+	@$(MAKE) kind-load
+	@helm list | grep -q mokv && (echo "♻ Upgrading existing deployment..." && $(MAKE) upgrade) || (echo "Deploying mokv..." && $(MAKE) deploy)
+	@echo "Waiting for pods to be ready..."
+	@kubectl wait --for=condition=ready pod -l app.kubernetes.io/name=mokv --timeout=120s
+	@echo "mokv is running!"
+	@kubectl get pods -l app.kubernetes.io/name=mokv
+	@echo ""
+	@echo "Run 'kubectl port-forward pod/mokv-0 9400:8400' to access the cluster"
+	@echo "Then test with: go run cmd/test_kv.go -addr localhost:9400"

README.md

@@ -1,127 +1,116 @@
 # mökv
 
-`mökv` is a distributed, in-memory key-value store. It utilizes [`Raft`](https://github.com/hashicorp/raft) for consensus, [`serf`](https://github.com/hashicorp/serf) for discvoery, and [`gRPC`](https://github.com/grpc/grpc-go) for communication.
+`mökv` is a distributed, in-memory key-value store built with [`Raft`](https://github.com/hashicorp/raft) for consensus, [`Serf`](https://github.com/hashicorp/serf) for discovery, and [`gRPC`](https://github.com/grpc/grpc-go) for communication.
 
 ## Features
 
-- Distributed Architecture: Data is replicated across multiple nodes for fault tolerance.
-- In-Memory Storage: Provides fast read and write operations.
-- `Raft` Consensus: Ensures data consistency across the cluster.
-- `gRPC` Interface: Offers a well-defined `API` for interacting with the store.
-- Metrics: Exposes `Prometheus` metrics for monitoring cluster health and performance.
-- Service Discovery: Uses `serf` for automatic node discovery and membership management.
-- Load Balancing: Implements `gRPC` client-side load balancing, directing write operations to the leader and read operations to followers.
+- **Distributed & Fault-Tolerant**: Data replicated across multiple nodes using Raft consensus
+- **In-Memory Storage**: Fast read/write operations with persistent snapshots
+- **Service Discovery**: Automatic node discovery and membership via Serf
+- **Smart Load Balancing**: Client-side routing directs writes to leader, reads to followers
 
-## Getting Started
-
-To run `mökv`:
+## Quick Start
 
 ### Prerequisites
 
-- [`Go`](https://go.dev/dl/)
-- [`ghz`](https://ghz.sh/) (for performance testing. Optional)
+- [`Docker`](https://www.docker.com/)
+- [`kind`](https://kind.sigs.k8s.io/) (for local Kubernetes)
+- [`kubectl`](https://kubernetes.io/docs/tasks/tools/)
+- [`Helm`](https://helm.sh/)
 
-### Installation
+### Run it locally
 
-1. Clone the repository:
+```bash
+make dev
+```
 
-   ```bash
-   git clone git@github.com:dynamic-calm/mokv.git
-   cd mokv
-   ```
+This will:
 
-2. Compile the code:
+1. Create a kind cluster (if needed)
+2. Build the Docker image
+3. Load it into kind
+4. Deploy with Helm
+5. Wait for all pods to be ready
 
-   ```bash
-   make build
-   ```
+Then test it:
 
-   This will create an executable binary `mokv` in the `bin/` directory.
+```bash
+kubectl port-forward pod/mokv-0 9400:8400
+# In another terminal:
+go run cmd/get_servers.go -addr localhost:9400
+```
 
 ### Configuration
 
-Configuration is done through command-line flags or a configuration file. A sample configuration file (`example/config.yaml`) is provided.
+Customize via `deploy/mokv/values.yaml` or override during installation:
 
-Here's an example `config.yaml`:
-
-```yaml
-data-dir: /tmp/mokv-data
-node-name: node1
-bind-addr: 127.0.0.1:8401
-rpc-port: 8400
-start-join-addrs: []
-bootstrap: true
-metrics-port: 4000
+```bash
+helm install mokv deploy/mokv --set replicas=5 --set storage=2Gi
 ```
 
-### Running mökv
-
-1. Start the first node:
-
-   ```bash
-   bin/mokv --config-file example/config.yaml
-   ```
-
-2. Start additional nodes:
-
-   Modify the `example/config.yaml` file with the appropriate `node-name`, `bind-addr`, and `rpc-port`. Crucially, set `start-join-addrs` to the address of the first node (e.g., `127.0.0.1:8401`). Also set `bootstrap: false` for the additional nodes. Then, run the command again:
+Default values:
 
-   ```bash
-   bin/mokv --config-file example/config2.yaml  # Example config for the second node
-   ```
-
-   Refer to [`example/start_nodes.sh`](example/start_nodes.sh) for a convenient script to start a cluster.
+```yaml
+replicas: 3 # Number of nodes
+storage: 1Gi # Persistent volume size per node
+rpcPort: 8400 # gRPC port
+serfPort: 8401 # Serf discovery port
+```
 
-## Usage
+## API
 
-`mökv` exposes a `gRPC` `API` defined in `internal/api/kv.proto`. You can use a `gRPC` client to interact with the store.
+`mökv` exposes a gRPC API defined in `api/kv.proto`:
 
 ```proto
 service KV {
-    rpc Get(GetRequest) returns (GetResponse) {}
-    rpc Set(SetRequest) returns (SetResponse) {}
-    rpc Delete(DeleteRequest) returns (DeleteResponse) {}
-    rpc List(google.protobuf.Empty) returns (stream GetResponse) {}
-    rpc GetServers(google.protobuf.Empty) returns (GetServersResponse){}
+    rpc Get(GetRequest) returns (GetResponse);
+    rpc Set(SetRequest) returns (SetResponse);
+    rpc Delete(DeleteRequest) returns (DeleteResponse);
+    rpc List(google.protobuf.Empty) returns (stream GetResponse);
+    rpc GetServers(google.protobuf.Empty) returns (GetServersResponse);
 }
 ```
 
-## How it Works: Core Components and Data Flow
-
-`mökv` combines `Serf` for node discovery and `Raft` for consistent data replication. Here's how the key components interact:
+## Architecture
 
-- `Serf`: Dynamic Membership: `Serf` uses `UDP` to monitor cluster membership. When a node joins, the `serf.EventMemberJoin` event triggers the `Join` function ([`internal/kv/kv.go`](/kv/kv.go)), adding the node as a `Raft` voter. This ensures the `Raft` cluster reflects the current active nodes.
+**Raft Consensus**: Ensures strong consistency with leader-based replication. Writes go through the leader and are replicated to followers.
 
-- `Raft`: Consensus and the FSM: `Raft` guarantees data consistency. One node is `Leader`, handling all write operations. Write operations become `Raft` log entries, replicated to `Followers`. The _Finite State Machine (`FSM`)_ is the core of `Raft's` operation:
+**Serf Discovery**: Nodes automatically discover each other via gossip protocol. When a node joins via Serf, it's added as a Raft voter.
 
-  - Applying Log Entries: When a log entry is committed (acknowledged by a quorum), the `Apply` method of the `FSM` (in `internal/kv/kv.go`) is invoked. The `Apply` method handles different request types:
+**Client-Side Load Balancing**: Custom gRPC resolver and picker route:
 
-    - Set Request: Updates the in-memory key-value store (`kv.store`) with the new key-value pair.
-    - Delete Request: Removes the specified key from the in-memory store.
+- **Writes** (`Set`, `Delete`) → Leader
+- **Reads** (`Get`, `List`) → Followers (load balanced)
 
-  - Data Flow for Writes: `gRPC` -> `Raft Leader` -> `Log Entry` -> `Replication to Followers` -> `FSM Apply` -> `kv.store`.
+**Kubernetes Components**:
 
-- Persistence (`raft-boltdb`): `mökv` uses `raft-boltdb` to persist Raft's log, stable state, and periodic snapshots to disk. This enables recovery after node failures.
+- **StatefulSet**: Stable network identities (mokv-0, mokv-1, mokv-2)
+- **Headless Service**: Direct pod-to-pod communication via FQDNs
+- **PersistentVolumeClaims**: Durable storage for Raft logs and snapshots
+- **Init Container**: Auto-configures each pod (bootstrap vs join)
 
-  - Snapshotting: The `FSM's` `Snapshot` method creates a snapshot of the current in-memory state.
-  - Restoring State: After a crash, the `FSM's` `Restore` method loads the latest snapshot and replays any subsequent log entries, reconstructing the in-memory `kv.store` to a consistent state. This entire process happens automatically when `setupRaft` is called during startup.
+## Management
 
-## gRPC
+**Update deployment**:
 
-`mökv` uses `gRPC` for efficient communication between clients and the cluster.
-
-- API Definition: The core `gRPC` service, `KV`, is defined in [`internal/api/kv.proto`](internal/api/kv.proto), exposing methods like `Get`, `Set`, `Delete`, `List`, and `GetServers`.
+```bash
+make docker-build
+make kind-load
+kubectl rollout restart statefulset mokv
+```
 
-- `gRPC` Server: The server implementation resides in [`internal/server/server.go`](internal/server/server.go), handling `gRPC` requests.
+**Scale the cluster**:
 
-- Interceptors: `gRPC` Interceptors are used to handle:
+```bash
+helm upgrade mokv deploy/mokv --set replicas=5
+```
 
-  - Logging: Each incoming request is logged for monitoring.
+**Uninstall**:
 
-- Client-Side Load Balancing (Name Resolution and Picker): `mökv` uses client-side load balancing.
+```bash
+make clean
+```
 
-  - Name Resolver ([`internal/discovery/resolver.go`](internal/discovery/resolver.go)): The name resolver periodically calls `GetServers` to discover available `mökv` nodes and their roles (Leader/Follower). It updates the list of available servers with the `is_leader` attribute.
-  - Picker ([`internal/discovery/picker.go`](internal/discovery/picker.go)): The Picker directs requests based on the operation type and the leader status of available connections:
+## Local Development
 
-    - Writes (`Set`, `Delete`): These are routed to the _Leader_ node to ensure consistency.
-    - Reads (`Get`, `List`): These are balanced among available _Follower_ nodes for improved read performance.
+For local testing without Kubernetes, see [`example/start_nodes.sh`](example/start_nodes.sh).