Skip to content

Minho #22

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dominhok wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Minho #22

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
de25627
Final Product
dominhok Apr 13, 2025
5c9f935
update README
dominhok Apr 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
226 changes: 97 additions & 129 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,51 +1,117 @@
# CodeWave - Disaster Safety Alert System
# CodeWave - Rapid & Personalized Disaster Safety Alert System

## Overview
## Project Goal

**To deliver timely and personalized safety guidance during disasters via SMS and voice calls, adapting to user location, language, and specific needs.**

CodeWave leverages AI to provide crucial, context-aware instructions and intelligently interprets voice responses to connect users needing help with emergency services automatically.

This project is a FastAPI-based application designed to provide targeted safety alerts during disaster scenarios. It leverages various AI and communication technologies to deliver personalized notifications via SMS and voice calls, understand user voice responses, and provide relevant safety information including interactive maps.
## Overview

The system aims to cater alerts based on user vulnerability types and location, utilizing Retrieval-Augmented Generation (RAG) for contextually relevant safety instructions and Large Language Models (LLMs) for interpreting user needs expressed via voice.
CodeWave is an intelligent communication platform designed for disaster scenarios. It integrates AI and communication technologies to deliver tailored safety alerts. Utilizing RAG (Retrieval-Augmented Generation) and sophisticated LLMs, the system generates contextually relevant safety information and understands user voice input to identify and assist those in distress.

## Key Features

* **User Registration:** Allows users to register with their vulnerability type, address (geocoded to lat/lon), phone number, guardian details, and notification preferences (SMS/Call, Language).
* **Disaster Simulation:** An endpoint (`/api/simulate_disaster/`) triggers alert notifications to users within a specified radius of a simulated disaster event.
* **Targeted Notifications:** Filters users based on proximity to the disaster location.
* **RAG-Powered Alerts:** Utilizes Upstage AI LLM and embeddings with a FAISS vector store (built from local `.txt` manuals) to generate context-aware SMS and voice alert messages tailored to the disaster type and potentially user vulnerability.
* **Twilio Integration:**
* Sends SMS alerts, including a link to an interactive map.
* Initiates outbound voice calls with safety information using Text-to-Speech (TTS).
* Handles incoming voice calls, gathers user speech input.
* **Voice Response Interpretation:** Uses Anthropic's Claude model to analyze user voice responses during alert calls to determine if urgent assistance or reporting is required.
* **Emergency Contact Notification:** If Claude interprets a user response as requiring assistance, it automatically initiates a voice call to a pre-configured emergency phone number, relaying the user's details and voice message.
* **External API Integrations:**
* Kakao Geocoding API: Converts user addresses to latitude/longitude coordinates.
* Safety Data Portal (Korea): Fetches nearby shelter information via a proxy endpoint.
* **Interactive Map:** Serves an HTML map page (`/map/disaster_map/{simulation_id}`) using Naver Maps API, displaying the user's location (requires browser permission), the disaster epicenter and radius, and the nearest identified shelter with a directions link.
* **Dashboard Summary (Basic):** An endpoint (`/api/dashboard/summary`) uses an LLM to summarize user voice reports clustered by region.
* **Multi-language Support (Basic):** Handles basic translation (KO -> EN using Upstage) for messages based on user preference.
### 1. Adaptive Personalized Alerting
* **Contextual User Profiles:** Securely manages user information including precise location (via geocoding), vulnerability considerations (e.g., visual/hearing impairment, mobility issues), language preference, and contact details.
* **Precision Targeting:** Disseminates alerts exclusively to users within the defined impact radius of a disaster event.
* **AI-Generated Guidance:**
* Employs RAG with curated safety manuals (`how-to-s/`) to generate **actionable core guidance** (SMS/Voice) specific to the disaster type.
* Dynamically adapts message content and tone based on user vulnerability profiles, leveraging LLM capabilities.

### 2. Interactive Communication & Emergency Support
* **Multi-Channel Delivery:** Utilizes Twilio API for robust delivery of alerts via both SMS and voice (TTS).
* **Intelligent Voice Interaction:** Captures and interprets user voice responses using advanced STT.
* **AI-Powered Response Assessment (Claude 3.5 Sonnet):** Analyzes voice responses to accurately determine if a user requires urgent assistance or is reporting a critical situation.
* **Automated Emergency Assistance Forwarding:** When assistance is needed, the system automatically initiates a call to the designated emergency contact number (`EMERGENCY_PHONE_NUMBER`).
* Provides the operator with essential details: **user's phone number, reported location,** and the **transcribed voice message**.

### 3. Enhanced Situational Awareness & Accessibility
* **Real-time Safety Resources:** Integrates with external APIs like the Safety Data Portal (Korea) to provide locations of nearby shelters.
* **Geospatial Intelligence:** Uses Kakao Geocoding for accurate address-to-coordinate conversion.
* **Interactive Incident Map:**
* Delivers a unique map link via SMS (`/map/disaster_map/{simulation_id}`).
* Visually presents the disaster epicenter and radius, user's current location, the nearest available shelter, and directions (Naver Maps API).
* **Regional Insights Dashboard:** Offers an aggregated summary of user voice reports by region, generated by an LLM (`/api/dashboard/summary`).

## Use Cases & Personalization Examples

CodeWave ensures effective communication across diverse user needs:

* **Scenario: Large Fire in Gangnam-gu**
* **Elderly Resident (Korean, SMS Preferred):** Receives a concise Korean SMS focusing on fire safety relevant to their profile, possibly advising shelter-in-place guidance if evacuation is challenging, plus the map link.
* **Visually Impaired User (English, Call Preferred):** Receives a clear English voice call (TTS) detailing the fire, with audible instructions. They can respond verbally; STT hints enhance recognition accuracy.
* **Hearing Impaired User (Korean, SMS Preferred):** Receives a detailed Korean SMS with guidance and the map link, enabling visual assessment of shelters and the affected zone.
* **Foreign Resident (English, SMS Preferred):** Receives a translated English SMS regarding the nearby incident, advising caution and providing the map link.

* **System Adaptability:**
* **Language:** The system automatically translates core messages based on the user's `preferred_language` setting.
* **Modality:** Notification channel (SMS/Voice) is selected based on user preference (`wants_info_call`).
* **Content:** User `vulnerability_type` informs the AI to tailor the focus and phrasing of generated advice.

This adaptive approach ensures critical safety information is delivered rapidly and effectively, accommodating language barriers, location specifics, and diverse accessibility requirements.

## Technology Stack

* **Backend:** Python, FastAPI
* **Communication:** Twilio (SMS, Voice)
* **AI/LLM:**
* Upstage AI (Embeddings, LLM for RAG and Translation)
* Anthropic (Claude LLM for Voice Interpretation)
* Langchain (Orchestration, Vector Store, Prompts)
* **Database:** SQLAlchemy (with SQLite as default)
* **Vector Store:** FAISS
* **Geocoding:** Kakao REST API
* **Mapping:** Naver Maps API (via JavaScript)
* **Other:** Uvicorn (ASGI Server), Pydantic (Data Validation), python-dotenv (Environment Variables), httpx (HTTP Client)
* Langchain (Orchestration)
* Anthropic Claude 3.5 Sonnet (Voice Interpretation, RAG Summarization)
* Upstage (Embeddings, Translation KO->EN)
* FAISS (Vector Store)
* **Communication:** Twilio (SMS, Voice, TTS, STT)
* **Database:** SQLAlchemy (Default: SQLite)
* **APIs & Mapping:** Kakao Geocoding, Safety Data Portal (Korea), Naver Maps API
* **Infra & Tools:** Uvicorn, Pydantic, python-dotenv, httpx, Ngrok (for Development Webhooks)

## Quick Start

### Prerequisites

* Python 3.10+
* `pip`
* API Keys/Accounts: Twilio, Anthropic, Upstage, Kakao (REST & JS), Safety Data Portal (Korea)
* Ngrok (Optional, for local webhook testing)

### Installation & Setup

1. **Clone:** `git clone <repo-url> && cd <repo-dir>`
2. **Virtual Env:** `python -m venv venv && source venv/bin/activate` (or `.\venv\Scripts\activate` on Windows)
3. **Install Deps:** `pip install -r requirements.txt`
4. **Environment Variables:**
* `cp .env.example .env`
* Edit `.env` and **fill in all required API keys and phone numbers**. Ensure `EMERGENCY_PHONE_NUMBER` is correctly set.
* Configure your public URL (e.g., Ngrok for development) by setting `BASE_URL` in the `.env` file. This URL is used for generating links in notifications (like the map link).
5. **RAG Data:** Place relevant safety manuals (`.txt` files) in the `how-to-s/` directory. The vector store is built automatically on the first run.

### Running

1. **Start Server:**
```bash
uvicorn flask_twilio_demo.app:app --reload --host 0.0.0.0 --port 30000
```
* (`--reload` is for development; remove for production).
2. **Ngrok (if needed for webhooks):**
* `ngrok http 30000 --domain <your-reserved-ngrok-domain.ngrok.app>` (Use your reserved domain)
* Update Twilio webhook URLs in the Twilio console to point to your ngrok HTTPS URL (e.g., `https://<your-domain>.ngrok.app/twilio/sms`). The application reads the base URL from the `BASE_URL` environment variable for generating callback URLs and map links.
3. **Access:**
* **API Docs:** `http://localhost:30000/docs`
* **Map (after simulation):** Link sent via SMS (uses `BASE_URL`).

## Key Endpoints

* `/api/users/` (POST): Register a new user.
* `/api/simulate_disaster/` (POST): Trigger a disaster simulation and send alerts.
* `/map/disaster_map/{simulation_id}` (GET): View the interactive map.
* `/twilio/sms` & `/twilio/voice`: Webhook endpoints for Twilio.

## Project Structure (Key Files/Dirs)

```
.
├── flask_twilio_demo/
│ ├── __init__.py
│ ├── app.py # Main FastAPI application logic, API endpoints
│ ├── app.py # Main application logic, API endpoints
│ ├── models.py # SQLAlchemy database models
│ ├── static/ # Static files (CSS, JS, Frontend Build Output)
│ │ ├── static/ # (Potentially nested static dir from frontend build)
Expand All @@ -56,102 +122,4 @@ The system aims to cater alerts based on user vulnerability types and location,
├── .env.example # Example environment variable file
├── README.md # This file
└── requirements.txt # Python dependencies
```

## Setup and Installation

### Prerequisites

* Python 3.10+ (Developed with 3.12)
* `pip` (Python package installer)
* An `ngrok` account (or similar tunneling service) for local development if testing Twilio webhooks. A paid ngrok plan is recommended to avoid interstitial pages that can interfere with API calls from certain browsers (like iOS Safari).
* Accounts and API Keys for:
* Twilio
* Upstage AI
* Anthropic
* Kakao (REST API Key for Geocoding, JavaScript App Key for Naver Maps via Kakao)
* Safety Data Portal (Korea)

### Steps

1. **Clone the Repository:**
```bash
git clone <repository-url>
cd <repository-directory>
```

2. **Create and Activate a Virtual Environment:**
```bash
# Linux/macOS
python3 -m venv venv
source venv/bin/activate

# Windows
python -m venv venv
.\venv\Scripts\activate
```

3. **Install Dependencies:**
```bash
pip install -r requirements.txt
```
*(Note: If `requirements.txt` is missing, you might need to install packages listed in `app.py`'s imports manually or generate it using `pip freeze > requirements.txt` after manual installation.)*

4. **Set Up Environment Variables:**
* Copy the example environment file:
```bash
cp .env.example .env
```
* Edit the `.env` file and fill in your actual API keys and configuration values obtained from the respective services (Twilio, Upstage, Anthropic, Kakao, Safety Data Portal). Pay special attention to `TWILIO_PHONE_NUMBER` (must be a Twilio number you own) and `EMERGENCY_PHONE_NUMBER` (the number to call when a user reports an emergency).
* Set the `DATABASE_URL` if you want to use a different database than the default SQLite file (`sqlite:///./flask_twilio_demo/users.db`).

5. **Prepare RAG Data:**
* Ensure the `how-to-s/` directory exists in the project root.
* Place relevant safety manuals or instruction documents as `.txt` files inside the `how-to-s/` directory. These will be used to build the FAISS vector store on the first run (or if the store doesn't exist).

6. **Database Setup:**
* The application uses SQLAlchemy and Alembic (if configured) or `Base.metadata.create_all()` to manage the database schema.
* When the application starts for the first time, it should automatically create the necessary tables in the database specified by `DATABASE_URL` (default: `users.db` in `flask_twilio_demo`).

7. **(Optional) Frontend Setup:**
* The `app.py` includes a catch-all route to serve a frontend application (likely React, based on typical setups) from the `flask_twilio_demo/static/` directory.
* If a separate frontend build process is required, follow its specific instructions (e.g., `npm install` and `npm run build`) to generate the static assets in the correct directory.

## Running the Application

1. **Start the FastAPI Server:**
```bash
uvicorn flask_twilio_demo.app:app --reload --host 0.0.0.0 --port 30000
```
* `--reload`: Enables auto-reloading during development (remove for production).
* `--host 0.0.0.0`: Makes the server accessible on your local network.
* `--port 30000`: Specifies the port number (adjust if needed).

2. **Set up a Tunnel (for Webhooks):**
* If testing Twilio webhooks locally (e.g., incoming calls/SMS, voice responses), you need a tunneling service like `ngrok`.
* Start ngrok for the port your FastAPI app is running on:
```bash
ngrok http 30000
```
* Ngrok will provide a public HTTPS URL (e.g., `https://<unique-id>.ngrok.io`).
* **Crucially:** Update the `hardcoded_base_url` variable within the `simulate_disaster` function in `app.py` to match this ngrok URL. (Ideally, move this to the `.env` file as `BASE_URL`).
* Configure your Twilio phone number's webhook URLs (for incoming SMS and Voice) in the Twilio console to point to your ngrok URL + the respective webhook paths (e.g., `https://<unique-id>.ngrok.io/twilio/sms`, `https://<unique-id>.ngrok.io/twilio/voice`).

3. **Access the Application:**
* **API Documentation (Swagger UI):** Open your browser to `http://localhost:30000/docs` (or your ngrok URL + `/docs`).
* **Map Interface (Example):** After running a simulation, access the map link sent via SMS, which will look like `https://<your-ngrok-url>/map/disaster_map/{simulation_id}`.
* **Frontend:** Access `http://localhost:30000` (or your ngrok URL).

## Usage Examples

1. **Register a User:** Use the Swagger UI (`/docs`) to send a POST request to `/api/users/` with the required user details in the request body (using the specified aliases like `personType`, `phone`, etc.).
2. **Simulate a Disaster:** Send a POST request to `/api/simulate_disaster/` with `DisasterAlertData` in the request body. This will trigger SMS/voice alerts to registered users near the disaster location.

## Important Considerations

* **In-Memory Simulation Data:** Active simulation data (used for map links) is stored in an in-memory dictionary (`app.state.active_simulations`). This data will be lost if the server restarts. For persistence, consider using a database or cache (like Redis).
* **ngrok and Webhooks:** A tunneling service like ngrok is essential for Twilio webhooks to reach your local development server. Remember to update the `hardcoded_base_url` in `app.py` or configure it via `.env`. Using a paid ngrok plan is recommended to avoid issues with interstitial pages.
* **API Keys:** Keep all your API keys secure in the `.env` file and **do not** commit the `.env` file to version control. Use `.env.example` as a template.
* **Error Handling:** While some error handling is present, further robustness could be added, especially around external API calls and LLM interactions.
* **Scalability:** The current setup (in-memory storage, default SQLite) is suitable for development but may need adjustments (e.g., PostgreSQL, Redis, Celery for background tasks) for production scaling.
* **RAG Data Quality:** The effectiveness of the generated alert messages heavily depends on the quality and relevance of the `.txt` files provided in the `how-to-s` directory.
```
Loading