Biotact Production - AI-Powered RAG System

Intelligent knowledge base assistant for Biotact pharmaceutical company

🚀 Live Demo: https://core.biotact.uz 📧 Support: ruslan.biotact@gmail.com, x.temrjan@gmail.com

---

🎯 Overview

Production-ready RAG (Retrieval-Augmented Generation) system that provides intelligent responses based on Biotact's knowledge base using OpenAI embeddings and GPT-4o.

Key Features

• ✅ Smart RAG Pipeline: Semantic search with GPT-4o responses
• ✅ Multi-Department Support: Marketing, Sales, Medical, Support
• ✅ File Upload: Web interface for uploading .md documents
• ✅ Incremental Indexing: Only indexes new/changed documents
• ✅ User Management: Admin approval workflow
• ✅ Modern UI: React + TypeScript + Tailwind CSS
• ✅ Production Ready: Deployed on Ubuntu with Nginx + SSL

---

🏗️ Current Architecture

Nginx (SSL/TLS)
  ├── Frontend (React SPA) → /frontend/dist
  └── API (FastAPI) → localhost:8000
        ├── Qdrant (Vector DB) → localhost:6333
        ├── OpenAI Embeddings (text-embedding-3-large)
        └── OpenAI GPT-4o (LLM)

Note: Simple single-server deployment. No Docker, no Kubernetes, no microservices overhead.

---

🔧 Technology Stack

Backend

• Framework: FastAPI 0.115.0
• Language: Python 3.11+
• Vector DB: Qdrant 1.12
• AI/ML:
  • OpenAI text-embedding-3-large (3072 dimensions)
  • OpenAI GPT-4o (LLM)
• RAG Framework: LlamaIndex 0.11.20

Frontend

• Framework: React 18 + TypeScript
• Build Tool: Vite 6
• UI Components: shadcn/ui + Tailwind CSS
• State: React Context API
• Icons: Lucide React

Infrastructure

• Web Server: Nginx
• SSL: Let's Encrypt
• OS: Ubuntu 22.04 LTS
• Deployment: systemd services

---

📦 Quick Start

Prerequisites

# Required
- Ubuntu 20.04+ / Debian 11+
- Python 3.11+
- Node.js 18+
- 8GB+ RAM (for Qdrant)
- OpenAI API key

# Optional
- Nginx (for production)
- SSL certificate

Installation (Local Development)

1. Clone repository

git clone https://github.com/temrjan/biotact_core.git biotact-production
cd biotact-production

2. Setup environment

# Copy example .env
cp .env.example .env

# Edit .env and add your API keys
nano .env

Required environment variables:

# OpenAI
OPENAI_API_KEY=sk-proj-...
OPENAI_EMBEDDING_MODEL=text-embedding-3-large
OPENAI_EMBEDDING_DIMENSIONS=3072

# Qdrant
QDRANT_HOST=localhost
QDRANT_PORT=6333
QDRANT_COLLECTION_NAME=biotact_knowledge_v3

# URLs (for production)
API_URL=https://core.biotact.uz

3. Run setup script

chmod +x setup_local_simple.sh
./setup_local_simple.sh

This will:

• Create Python virtual environment
• Install backend dependencies
• Install Qdrant
• Setup directories

4. Install frontend

cd frontend
npm install
npm run dev  # Development mode
# OR
npm run build  # Production build

5. Prepare documents

# Copy your .md files to:
cp your-docs/*.md data/documents/

6. Index documents

# Activate venv
source venv/bin/activate

# Run incremental indexing
python scripts/incremental_index.py

7. Start backend

# Development
venv/bin/uvicorn backend.main:app --reload --host 0.0.0.0 --port 8000

# Production (with logs)
venv/bin/uvicorn backend.main:app --host 0.0.0.0 --port 8000 >> logs/backend.log 2>&1 &

8. Access the application

# Frontend (dev): http://localhost:5173
# Backend API: http://localhost:8000
# API Docs: http://localhost:8000/docs

---

📚 Usage

Admin Login

Email: <admin-email>
Password: <your-secure-password>

Upload Documents

1. Login as admin
2. Go to right sidebar
3. Drag & drop .md files or click to select
4. Wait for upload to complete
5. Click "Запустить индексацию" button
6. Wait 30-60 seconds for indexing

Query the System

1. Select department (Marketing, Sales, Medical, Support)
2. Type your question
3. System will:
   • Search relevant documents (vector similarity)
   • Generate response using GPT-4o
   • Show sources used

---

🔐 API Endpoints

Public

• GET / - Root endpoint
• GET /health - System health check
• GET /departments - List available departments
• POST /auth/login - User authentication
• POST /auth/register - User registration

Authenticated

• POST /chat - RAG chat with AI
• POST /query - RAG query
• POST /search - Vector search only
• GET /stats - System statistics
• POST /upload - Upload .md files (admin only)
• POST /index - Run incremental indexing (admin only)

Admin Only

• GET /auth/admin/users/pending - Pending users
• POST /auth/admin/users/{id}/approve - Approve user
• POST /auth/admin/users/{id}/reject - Reject user

API Documentation: http://localhost:8000/docs

---

🗂️ Project Structure

biotact-production/
├── backend/
│   ├── main.py              # FastAPI app + RAG system
│   ├── config/
│   │   └── departments.py   # Department configurations
│   └── prompts/
│       ├── marketing.txt    # Marketing prompt
│       ├── sales.txt        # Sales prompt
│       ├── medical.txt      # Medical prompt
│       └── support.txt      # Support prompt
├── frontend/
│   ├── src/
│   │   ├── components/      # React components
│   │   ├── contexts/        # React contexts
│   │   ├── api/            # API client
│   │   └── types/          # TypeScript types
│   ├── dist/               # Production build
│   └── package.json
├── data/
│   ├── documents/          # 📁 Upload .md files here
│   ├── .index_state.json   # Indexing state tracker
│   └── qdrant/            # Qdrant data (ignored by git)
├── scripts/
│   └── incremental_index.py # Incremental indexing script
├── docs/
│   └── INDEXING_GUIDE.md   # Indexing documentation
├── logs/                   # Application logs
├── venv/                   # Python virtual environment
├── .env                    # Environment variables (not in git)
├── .env.example           # Example environment variables
└── requirements.txt        # Python dependencies

---

📖 Documentation

• Indexing Guide - Detailed indexing documentation
• Indexing Quick Start - Quick reference
• Deployment Status - Current deployment state
• API Docs - Interactive API documentation

---

🔄 Incremental Indexing

The system uses hash-based incremental indexing to avoid reprocessing unchanged files.

How it works:

1. Calculates MD5 hash of each file
2. Compares with previous state (.index_state.json)
3. Only indexes new/modified files
4. Removes vectors for deleted files

Run indexing:

# Manual
venv/bin/python scripts/incremental_index.py

# Via API (admin only)
curl -X POST https://core.biotact.uz/index

See: INDEXING_GUIDE.md

---

🚀 Production Deployment

1. Install Nginx

sudo apt update
sudo apt install nginx

2. Configure Nginx

# Copy nginx config
sudo cp infrastructure/nginx/core.biotact.uz.conf /etc/nginx/sites-available/core.biotact.uz
sudo ln -s /etc/nginx/sites-available/core.biotact.uz /etc/nginx/sites-enabled/

# Test config
sudo nginx -t

# Reload
sudo systemctl reload nginx

3. Setup SSL (Let's Encrypt)

sudo apt install certbot python3-certbot-nginx
sudo certbot --nginx -d core.biotact.uz

4. Setup systemd service

# Create service file
sudo nano /etc/systemd/system/biotact-backend.service

[Unit]
Description=Biotact Backend API
After=network.target

[Service]
Type=simple
User=temrjan
WorkingDirectory=/home/temrjan/project/biotact-production
Environment="PATH=/home/temrjan/project/biotact-production/venv/bin"
ExecStart=/home/temrjan/project/biotact-production/venv/bin/uvicorn backend.main:app --host 0.0.0.0 --port 8000
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

# Enable and start
sudo systemctl daemon-reload
sudo systemctl enable biotact-backend
sudo systemctl start biotact-backend

---

🧪 Testing

Health Check

curl http://localhost:8000/health
# {"status":"healthy","postgres":true,"qdrant":true,"embeddings":true,"llm":true}

Test RAG Query

curl -X POST http://localhost:8000/chat \
  -H "Content-Type: application/json" \
  -d '{
    "message": "Что такое биотакт?",
    "department": "marketing"
  }'

Check Qdrant

curl http://localhost:6333/collections/biotact_knowledge_v3

---

📊 System Requirements

Minimum (Development)

• CPU: 2 cores
• RAM: 4GB
• Disk: 20GB
• Bandwidth: 10 Mbps

Recommended (Production)

• CPU: 4 cores
• RAM: 8GB
• Disk: 50GB SSD
• Bandwidth: 100 Mbps

Current Deployment

• Server: Ubuntu 22.04 LTS
• CPU: 2 vCPU
• RAM: 8GB
• Disk: 80GB
• Vectors in DB: 45
• Documents: 5 .md files

---

🛠️ Troubleshooting

Backend won't start

# Check logs
tail -50 logs/backend.log

# Check port
lsof -i :8000

# Check Qdrant
curl http://localhost:6333/health

Frontend build fails

cd frontend
rm -rf node_modules package-lock.json
npm install
npm run build

Indexing errors

# Check documents directory
ls -la data/documents/

# Check Qdrant connection
curl http://localhost:6333/collections

# Re-run indexing with verbose output
venv/bin/python scripts/incremental_index.py

Qdrant connection errors

# Check Qdrant process
ps aux | grep qdrant

# Restart Qdrant
pkill qdrant
# Start Qdrant (see setup script)

---

🔒 Security Notes

⚠️ Current Implementation (Development):

• User passwords in plain text (in-memory dictionary)
• Simple token authentication (not JWT)
• No rate limiting
• No input validation

✅ TODO for Production:

• Implement PostgreSQL user storage
• Hash passwords with bcrypt/argon2
• Use JWT tokens with expiration
• Add rate limiting
• Add input sanitization
• Enable CORS restrictions
• Add request logging
• Setup firewall rules

---

📈 Performance

Current Metrics:

• Response time: ~3-5 seconds (GPT-4o generation)
• Concurrent users: ~10-20
• Vector search: <100ms
• Embedding generation: ~500ms
• Total throughput: ~5-10 requests/minute

Optimization Tips:

• Use GPT-4o-mini for faster/cheaper responses (17x cheaper)
• Increase similarity_top_k for better context
• Add Redis caching for frequent queries
• Use streaming responses for better UX

---

🤝 Contributing

This is a private production system. For authorized team members:

1. Create feature branch: git checkout -b feature/your-feature
2. Make changes and test locally
3. Commit: git commit -m "feat: your feature"
4. Push: git push origin feature/your-feature
5. Create Pull Request

---

📄 License

Proprietary - Biotact © 2024

Unauthorized copying, modification, or distribution is prohibited.

---

👥 Team

• Project Lead: Temrjan (@temrjan)
• Admin Contacts:
  • ruslan.biotact@gmail.com
  • x.temrjan@gmail.com

---

📞 Support

Issues: Create issue on GitHub Email: support@biotact.com Telegram: @biotact_support

---

🎯 Roadmap

✅ Completed (v1.0)

• Basic RAG pipeline
• Multi-department support
• File upload interface
• Incremental indexing
• Admin user management
• Production deployment
• Migration to GPT-4o

🚧 In Progress (v1.1)

• PostgreSQL user storage
• Better error handling
• Response caching
• Analytics dashboard

📋 Planned (v2.0)

• Chat history
• Document versioning
• Multi-language support (Uzbek)
• Voice input/output
• Mobile app
• Advanced analytics

---

Version: 1.0.1 Last Updated: November 20, 2025 Status: ✅ Production Ready