This project demonstrates a real-time vote streaming application using Docker, PostgreSQL, Python, Apache Kafka, Apache Spark for transformations, and Streamlit for visualizations.
The Real-Time Vote Streaming Application is designed to showcase a scalable and efficient system for processing and visualizing voting data in real-time. It utilizes various technologies to achieve this, including Docker for containerization, PostgreSQL for data storage, Apache Kafka for message streaming, Apache Spark for data transformations, and Streamlit for interactive visualizations.
- Real-time vote streaming and processing.
- Scalable architecture using Docker containers.
- PostgreSQL database for persistent storage of vote data.
- Apache Kafka for efficient message streaming.
- Apache Spark for data transformations and analytics.
- Interactive visualizations powered by Streamlit.
Before you begin, ensure you have the following installed:
- Docker
- Docker Compose
- Python 3.9 and up
- Apache Kafka
- Apache Spark
- PostgreSQL
-
Clone the repository:
git clone https://github.com/yourusername/real-time-vote-streaming.git
-
Navigate to the project directory:
cd Real-Time-Vote-Application -
Build and start the Docker containers:
docker-compose up --build docker.yml
-
Install Python dependencies:
pip install -r requirements.txt
- Start Apache Kafka broker and Zookeeper services.
- Run the Spark streaming application to process incoming vote data from an API and save it in postgres and also produces those voters in producers to store it in topics.
- Start the Streamlit server for visualizations.
- Interact with the Streamlit dashboard to view real-time voting statistics.
Contributions are welcome! Please fork the repository and submit a pull request.
This project is licensed under the Apache License - see the LICENSE file for details.