A scalable backend system built using Node.js, PostgreSQL, and Redis to process asynchronous tasks using a distributed worker architecture.
- REST API for task submission
- Redis-based queue for fast task dispatch
- PostgreSQL for persistent storage
- Multiple workers for parallel processing
- Retry logic with exponential backoff
- Priority-based task scheduling (HIGH, MEDIUM, LOW)
- Node.js (Express)
- PostgreSQL
- Redis
-
Install dependencies: npm install
-
Start PostgreSQL and Redis
-
Run the server: node server.js
-
Run workers (in separate terminals): node worker.js node worker.js
Create a task:
curl -X POST http://localhost:3000/tasks
-H "Content-Type: application/json"
-d '{"type":"EMAIL","payload":{"message":"hello"},"priority":"HIGH"}'
Client → API → PostgreSQL → Redis Queue → Workers → Status Updates
- Distributed systems
- Asynchronous processing
- Queue-based architecture
- Fault tolerance and retries
- Horizontal scaling with multiple workers
Mohammed Masroor Ahmed