Real-Time Leaderboard System

A production-grade, scalable real-time leaderboard system built with Go, gRPC, Redis, and Kafka. This project demonstrates best practices in distributed systems design, including clean architecture, JWT authentication, event-driven architecture, and comprehensive testing.

🏗️ Architecture

I have designed the system to be modular and event-driven. Here is the high-level flow:

1. Clients connect via high-performance gRPC.
2. Auth Middleware secures endpoints using JWT tokens.
3. The Service Layer orchestrates business logic.
4. Redis powers the leaderboards for fast reads and writes.
5. Kafka streams score updates asynchronously to other services (like Monitor service).

System Diagram

┌─────────────┐
│   Client    │
└──────┬──────┘
       │ gRPC
       ▼
┌─────────────────────────────────────┐
│        gRPC Server                  │
│  ┌───────────────────────────────┐  │
│  │  Auth Middleware (JWT)        │  │
│  └───────────┬───────────────────┘  │
│              ▼                      |                                 
│  ┌───────────────────────────────┐  │
│  │  Service Layer                │  │
│  └─────┬────────────────┬────────┘  │
│        │                │           │
│        │ (Fast Write)   │ (Event)   │
│        ▼                ▼           │
│    ┌────────┐      ┌──────────┐     │
│    │ Redis  │      │  Kafka   │     │
│    └────────┘      └─────┬────┘     │
└──────────────────────────┼──────────┘
                           │
                           ▼
                    ┌──────────────┐
                    │   Monitor    │
                    │   Service    │
                    └──────────────┘

Technology Stack

• Go 1.23+: High-performance backend language
• gRPC: Efficient RPC framework for API communication
• Redis: In-memory data store for leaderboards and user data
• Kafka: Distributed event streaming for real-time updates
• JWT: Secure token-based authentication
• Docker: Containerization and orchestration
• Protocol Buffers: Efficient data serialization

---

🚀 Getting Started

Ready to dive in? You can have the full system running locally in just a few minutes.

Prerequisites

• Go 1.23 or higher
• Docker & Docker Compose
• protoc (Protocol Buffers compiler)

Installation

The easiest way to run the system is using the provided Makefile commands which handle Docker orchestration for you.

1. Clone the repo:

   git clone https://github.com/DevanshTomar/real-time-scoreboard.git
   cd real-time-scoreboard

2. Set up dependencies:

   make setup

3. Start all services with Docker

   make docker-up

4. Run the demo client

   make run-client

Manual Setup

If you prefer running services individually (great for debugging):

1. Start Infrastructure: docker-compose up -d redis kafka zookeeper
2. Build & Run Server: make run-server
3. Run Monitor: make run-monitor (in a new terminal)
4. Run Client: make run-client (in a new terminal)

---

🎯 Key Features

I didn't just build a scoreboard; I built a system ready for the real world.

• ✅ Secure by Default: JWT authentication and bcrypt password hashing ensure user data is safe.
• ✅ Real-Time Performance: Redis Sorted Sets allow it to update rankings instantly, even with millions of users.
• ✅ Event-Driven: Kafka integration means the leaderboard service updates can trigger achievements, push notifications, or analytics pipelines without slowing down the main API.
• ✅ Clean Code: The project follows strict Clean Architecture principles, making it testable and maintainable.
• ✅ Observability: A dedicated Monitor service logs events as they happen, giving you visibility into the system.

---

📡 API Reference

The system exposes a gRPC API. The main interactions are:

Authentication

• Register: Create a new account.
• Login: Exchange credentials for a JWT.

Gameplay & Scoring

• SubmitScore: Post a new score. The system automatically updates your high score and global rank.
• GetGlobalLeaderboard: See who's on top across all games.
• GetGameLeaderboard: Filter rankings by specific game.
• GetUserRank: Check your own standing.

(See proto/leaderboard.proto for the exact service definitions.)

---

🧪 Testing & Quality

• Run All Tests: make test
• Integration Tests: make test-integration (Requires Docker containers to be running)
• Check Coverage: go test -cover ./...

---

🔧 Configuration

Everything is configurable via environment variables. Check out .env.example.

Common settings:

• GRPC_PORT: Port for the main server (Default: 50051)
• REDIS_ADDR: Connection string for Redis.
• KAFKA_BROKERS: Kafka broker list.

Variable	Description	Default
GRPC_PORT	gRPC server port	50051
REDIS_ADDR	Redis address	localhost:6379
REDIS_POOL_SIZE	Connection pool size	10
KAFKA_BROKERS	Kafka broker addresses	localhost:9092
KAFKA_TOPIC	Event topic name	leaderboard-updates
JWT_SECRET	JWT signing secret	(must be set)
JWT_EXPIRATION_HOURS	Token expiration	24

🚦 Monitoring

The monitor service consumes Kafka events and logs them in real-time. View logs with:

docker-compose logs -f monitor

Example output:

{
  "time": "2024-01-15T10:30:45Z",
  "level": "INFO",
  "msg": "score update received",
  "user_id": "abc123",
  "username": "alice",
  "game_id": "game1",
  "score": 100,
  "timestamp": "2024-01-15T10:30:45Z"
}

---

📊 Load Testing

To test the system with concurrent users, run:

make run-loadtest

This will stress test the system with 500 concurrent users submitting scores.

---

🐛 Troubleshooting

• "Server won't start": Check if Redis and Kafka are running (docker-compose ps). Ensure port 50051 isn't taken.
• "Tests are failing": Integration tests need real infrastructure. Make sure you ran docker-compose up before testing.
• "gRPC Connection Refused": Verify the server is up and your firewall isn't blocking the connection.