🏦 SahaYaa

AI-Powered Multilingual Voice Banking Assistant

🚀 Live Demo • 📖 Documentation • 🎯 Features

---

Breaking language barriers in digital banking with voice-first AI

---

📋 Table of Contents

• 🌟 Project Overview
• 💡 Why SahaYaa?
• ✨ Features
• 🛠️ Tech Stack
• 🚀 Getting Started
• 💻 Usage
• 🎥 Demo Video
• 🏗️ Architecture
• 🗺️ Roadmap
• 📄 License
• 🙏 Acknowledgements

---

🌟 Project Overview

SahaYaa (सहाया - meaning "help" in Hindi) is an intelligent voice-first banking assistant designed to democratize digital banking across India's linguistic diversity. Built with state-of-the-art NLP and speech technologies, SahaYaa enables seamless banking operations through natural voice conversations in 8 languages.

🎯 What Problem Does It Solve?

• Digital Divide: 🇮🇳 Only 22% of Indians speak English, yet most banking apps require it
• Accessibility: 👨‍🦯 Voice-first design helps visually impaired and elderly users
• Complexity: 🔢 Eliminates complex UIs - just speak naturally
• Trust: 🔐 Multi-layer authentication builds user confidence

👥 Who It's For

• 🏘️ Rural Banking Customers - Limited digital literacy
• 👴 Senior Citizens - Prefer voice over typing
• ♿ Differently-abled Users - Screen-reader friendly
• 🚀 Tech-savvy Users - Faster than traditional banking apps

---

💡 Why SahaYaa?

🎨 What Sets Us Apart

Feature	Traditional Apps	SahaYaa
Language Support	2-3 languages	8 languages (7 Indian + English)
Interaction	Touch/Type	Voice-first
Learning Curve	Complex UI	Zero learning curve
Accessibility	Limited	Fully accessible
Security	OTP only	Biometric + OTP
Offline Support	Requires internet	Hybrid mode (planned)

🔥 Unique Features

1. 🎤 Indic ASR Excellence - Powered by AI4Bharat's IndicConformer (600M parameters)
2. 🧠 Context-Aware NLU - Understands code-mixed speech (Hinglish, Tanglish, etc.)
3. 🔐 Smart Security - Conditional OTP for high-value transactions (>₹5000)
4. 🎭 Natural Conversations - Handles follow-ups, corrections, and clarifications
5. 📊 Real-time Feedback - Voice + visual confirmation for critical operations

---

✨ Features

🏦 Banking Operations ✅ Balance Inquiry - Check account balance ✅ Money Transfer - UPI-style transfers ✅ Bill Payments - Electricity, mobile, gas ✅ Transaction History - Last N transactions ✅ Loan Inquiries - Check eligibility & rates ✅ Credit Limit - View available credit	🔒 Security Features 🔐 Multi-Factor Auth - Biometric + OTP 👤 Liveness Detection - Anti-spoofing 🚨 Risk-based OTP - Smart threshold (₹5000) 🔑 Session Management - 60-min timeout 📝 Audit Logging - Complete transaction trail 🛡️ CORS Protection - Secure API endpoints
🌐 Multilingual Support 🇮🇳 Hindi (हिंदी) 🇮🇳 Bengali (বাংলা) 🇮🇳 Marathi (मराठी) 🇮🇳 Odia (ଓଡ଼ିଆ) 🇮🇳 Tamil (தமிழ்) 🇮🇳 Telugu (తెలుగు) 🇬🇧 English	🤖 AI/ML Features 🎙️ Automatic Speech Recognition 🔊 Text-to-Speech (Multilingual) 🧠 Intent Classification (13+ intents) 🏷️ Entity Extraction (Amounts, accounts) 📈 Context Tracking (Multi-turn dialogues) 🔄 Fallback Handling (Graceful degradation)

---

🛠️ Tech Stack

Core Technologies

Category	Technology	Version	Purpose
🤖 NLU Engine		3.6.13	Intent classification & dialogue
🎙️ ASR		600M	Speech-to-text (8 languages)
🔊 TTS		2.4.0	Text-to-speech synthesis
🌐 API Framework		0.104	Voice API gateway
🧠 ML Framework		2.1.0	ASR model inference
💾 ML Ops		4.35.0	Model loading & inference
🎨 Frontend		-	Web interface
🔧 Audio		Latest	Audio processing

Development Stack

Language: Python 3.10
NLU Pipeline: RegexFeaturizer → DIETClassifier → TEDPolicy
ASR Model: ai4bharat/indic-conformer-600m-multilingual
Dialogue: Rasa 3.6 (Rule + ML policies)
Server: Uvicorn (ASGI)
Audio Format: WebM → WAV (16kHz mono)

---

🚀 Getting Started

📋 Prerequisites

• Python 3.10 or higher
• 8GB+ RAM
• ffmpeg installed (Guide)
• Git for version control

⚡ Quick Installation

git clone https://github.com/yourusername/sahayaa.git
cd sahayaa
python -m venv venv310
source venv310/bin/activate # Windows: venv310\Scripts\activate
pip install -r requirements.txt
rasa train  # First time only

🎬 Running SahaYaa

Open 3 terminals for:

Terminal 1: Action Server

 rasa run actions -p 5055

Terminal 2: Rasa Server

rasa run --enable-api --cors "*" -p 5005

Terminal 3: Voice API

uvicorn app.main:app --reload --port 8001

uvicorn voice_api:app --reload --port 8002

Terminal 4

python -m http.server 3000

🌐 Access the App

http://127.0.0.1:3000/#chat

---

💻 Usage

📱 Example Conversations

🔍 Check Balance (Hindi)

🗣️ You: "मेरे अकाउंट का बैलेंस क्या है?" 🤖 Bot: "आपके खाते acct_savings_1 में वर्तमान बैलेंस 48500.5 रुपये है।"

💸 Money Transfer - Small Amount (No OTP)

🗣️ You: "रिया को तीन सौ रुपये भेजो" 🤖 Bot: "300 रुपये acct_savings_1 से acct_friend_riya में सफलतापूर्वक भेजे गए।" "ट्रांजेक्शन आईडी TX00003।"

🔐 Money Transfer - Large Amount (OTP Required)

🗣️ You: "छः हज़ार रुपये भेजो" 🤖 Bot: "सुरक्षा के लिए, हमने आपके पंजीकृत मोबाइल पर 6 अंकों का OTP भेजा है।" "कृपया OTP बोलें।" 🗣️ You: "एक दो तीन चार पांच छह" 🤖 Bot: "OTP सत्यापित। 6000 रुपये सफलतापूर्वक भेजे गए।"

🎥 Demo Video

🎬 Watch SahaYaa in Action https://drive.google.com/file/d/1EGpoo4h7GJ8zrYXxYTdVKlnnoEUjA7nU/view?usp=sharing

---

🏗️ Architecture

USER INTERFACE (Web UI)
         |
    WebM Audio
         ▼
VOICE API (FastAPI)
    | ASR | Hindi→Digits | Audio Processing |
         ▼
RASA NLU + Dialogue
    | Intent | Entity | Dialogue |
         ▼
ACTION SERVER (Banking/OTP/TTS)
         ▼
SECURE BANKING API (Balance, Transfer)

---

🗺️ Roadmap

✅ Multilingual ASR
✅ Voice-first web interface
✅ OTP-based security for high-value transactions
✅ Real-time TTS responses
🌍 Support for 7 Indian languages

---

📄 License

Distributed under the MIT License.
See LICENSE for more info.

---

🙏 Acknowledgements

🌟 Open Source Projects

• AI4Bharat
• Rasa
• FastAPI
• PyTorch
• Hugging Face