Skip to content

README.md

Latest commit

 

History

History
427 lines (329 loc) · 11.9 KB

README.md

File metadata and controls

427 lines (329 loc) · 11.9 KB

🔍 DeepSearch AI

DeepSearch AI Node.js React Gemini AI

An intelligent search application powered by Google Gemini AI that transforms user queries into comprehensive, well-researched responses - similar to Perplexity or Grok AI.

🚀 Quick Start • 📖 Features • 🏗️ Architecture • 🔧 API Reference • 🤝 Contributing

🌟 Features

🤖 AI-Powered Search (Primary Feature)

  • 🧠 Query Decomposition: Intelligently breaks down complex questions into sub-questions
  • 🔗 Chain-of-Thought Reasoning: Connects related concepts and builds logical responses
  • 🔍 Multi-Source Analysis: Executes targeted searches based on AI analysis
  • ⚡ Intelligent Synthesis: Uses Gemini AI to analyze and synthesize information from multiple sources
  • ❓ Follow-up Questions: Generates relevant follow-up questions to explore topics deeper
  • 📊 Real-time Progress: Step-by-step processing updates in the UI

🔍 Traditional Search (Legacy Support)

  • 📝 Text Search: Comprehensive web search using DuckDuckGo
  • 📰 News Search: Latest news and articles
  • 🖼️ Image Search: Visual content discovery
  • ⚡ Parallel Processing: High-volume query handling
  • 🧠 Cognitive Bias Analysis: Identifies and categorizes cognitive biases

⚡ Performance & Reliability

  • 💾 Universal Caching: In-memory, file-based, or Redis caching options
  • 🛡️ Rate Limiting: Conservative API usage with retry logic
  • 🔒 Security: Helmet, CORS, and input validation
  • 📱 Responsive UI: Modern React interface with Tailwind CSS
  • 🔄 Error Handling: Graceful degradation and comprehensive error management

🏗️ Architecture

graph TB
    A[User Interface<br/>React + Tailwind] --> B[API Gateway<br/>Express.js]
    B --> C[AI Service<br/>Gemini Pro]
    B --> D[Search Service<br/>DuckDuckGo]
    B --> E[Cache Service<br/>Universal Cache]
    E --> F[Memory Cache]
    E --> G[File Cache]
    E --> H[Redis Cache]

    style A fill:#e1f5fe
    style B fill:#f3e5f5
    style C fill:#fff3e0
    style D fill:#e8f5e8
    style E fill:#fce4ec
Loading

🔧 Technology Stack

Layer Technology Purpose
Frontend React 18 + Vite + Tailwind CSS Modern, responsive user interface
Backend Node.js + Express RESTful API server
AI Engine Google Gemini Pro API Query analysis and response synthesis
Search DuckDuckGo (duck-duck-scrape) Web search functionality
Caching Universal Cache System Performance optimization
Security Helmet + CORS + Validation Comprehensive security layer

🚀 Quick Start

📋 Prerequisites

  • Node.js (v16 or higher)
  • npm or yarn
  • Google Gemini API Key (Get one here)
  • Redis (optional, for advanced caching)

⚡ Installation

1️⃣ Clone the Repository

git clone https://github.com/yourusername/deepsearch-ai.git
cd deepsearch-ai

2️⃣ Backend Setup

cd backend
npm install

# Configure environment variables
cp .env.example .env
# Edit .env and add your GEMINI_API_KEY

npm start

3️⃣ Frontend Setup

cd ../frontend
npm install

4️⃣ Environment Configuration

Create .env file in the backend directory:

# 🤖 AI Configuration (Required)
GEMINI_API_KEY=your_gemini_api_key_here

# 🌐 Server Configuration
PORT=3000
NODE_ENV=development
CORS_ORIGIN=http://localhost:5174

# 💾 Caching Configuration
CACHE_TYPE=memory
REDIS_URL=redis://localhost:6379
REDIS_ENABLED=false

🔑 Important: Get your Gemini API key from Google AI Studio

5️⃣ Start the Application

# Terminal 1: Start Backend
cd backend
npm start

# Terminal 2: Start Frontend
cd frontend
npm run dev

6️⃣ Access the Application

🔧 API Reference

🤖 AI Search Endpoints

Main AI Search

POST /api/ai
Content-Type: application/json

{
  "query": "How does quantum computing work?"
}

Response Structure:

{
  "query": "How does quantum computing work?",
  "timestamp": "2024-01-01T00:00:00.000Z",
  "decomposition": {
    "coreQuestion": "How does quantum computing work?",
    "subQuestions": ["What are qubits?", "How do quantum gates work?"],
    "chainOfThought": "To understand quantum computing..."
  },
  "response": "# How Quantum Computing Works\n\n...",
  "sources": [...],
  "followUpQuestions": [...],
  "processingSteps": {...}
}

AI Health Check

GET /api/ai/health

Simple Search (No AI)

GET /api/ai/simple?q=machine%20learning

🔍 Traditional Search Endpoints

General Search

GET /api/search?q={query}&type={text|news|images}

Cognitive Bias Analysis

GET /api/search/bias?q={query}

Parallel Search

POST /api/search/parallel
Content-Type: application/json

{
  "queries": ["AI trends", "Machine learning"],
  "type": "text"
}

⚙️ Configuration Options

💾 Caching Systems

Cache Type Speed Persistence Use Case
memory ⚡⚡⚡ ❌ Development (Default)
file ⚡ ✅ Simple persistence
redis ⚡⚡⚡ ✅ Production scaling
disabled ⚡⚡ N/A Testing

Switch cache types by changing CACHE_TYPE in .env:

CACHE_TYPE=memory  # or file, redis, disabled

🔒 Security Features

  • Helmet: Security headers and XSS protection
  • CORS: Cross-origin request management
  • Rate Limiting: DDoS protection with exponential backoff
  • Input Validation: Query sanitization and validation
  • Error Handling: Secure error responses

📊 Performance Tuning

  • Rate Limiting: 5-second delays between DuckDuckGo requests
  • Retry Logic: Exponential backoff for failed requests
  • Cache TTL: 1-hour default for search results
  • Request Limits: Conservative API usage patterns

🧪 Testing & Development

🔬 Test Commands

Test AI Search

# Test main AI functionality
curl -X POST "http://localhost:3000/api/ai" \
  -H "Content-Type: application/json" \
  -d '{"query": "How does machine learning work?"}'

# Check AI service health
curl "http://localhost:3000/api/ai/health"

Test Cache System

# Test all cache alternatives
cd backend
node test-cache.js

Test Traditional Search

# Test general search
curl "http://localhost:3000/api/search?q=artificial%20intelligence&type=text"

# Test cognitive bias analysis
curl "http://localhost:3000/api/search/bias?q=confirmation%20bias"

📁 Project Structure

deepsearch-ai/
├── 📂 backend/
│   ├── 📂 src/
│   │   ├── 📂 api/
│   │   │   ├── 🔧 aiSearch.js      # Main AI search endpoint
│   │   │   └── 🔍 search.js        # Traditional search endpoints
│   │   ├── 📂 services/
│   │   │   ├── 🤖 geminiService.js # Google Gemini AI integration
│   │   │   └── 💾 cacheService.js  # Universal caching system
│   │   ├── 📂 utils/
│   │   │   └── 🦆 duckduckgo.js    # DuckDuckGo search wrapper
│   │   └── 🚀 server.js            # Express server setup
│   ├── 📂 cache/                   # File-based cache storage
│   ├── ⚙️ .env                     # Environment configuration
│   └── 📦 package.json            # Dependencies and scripts
├── 📂 frontend/
│   ├── 📂 src/
│   │   ├── 📂 components/
│   │   │   └── 🧠 AISearchApp.jsx  # Main React component
│   │   ├── 🎯 main.jsx             # React entry point
│   │   └── 🎨 index.css            # Tailwind CSS styles
│   ├── 📄 index.html               # HTML template
│   ├── ⚡ vite.config.js           # Vite configuration
│   └── 📦 package.json            # Dependencies and scripts
└── 📖 README.md                   # This file

🚨 Known Issues & Limitations

⚠️ Current Limitations

  1. DuckDuckGo Rate Limiting

    • Conservative 5-second delays between requests
    • Limited to 1-2 searches per AI query to avoid blocking
    • Retry logic with exponential backoff

  2. Search Engine Constraints

    • duck-duck-scrape@2.2.7 lacks proper news method
    • News searches approximated via filtered general searches
    • Image search may have inconsistent results

  3. AI Processing Time

    • Complex queries can take 20-30 seconds
    • Multiple API calls to Gemini for decomposition and synthesis
    • Network latency affects response times

🔧 Workarounds Implemented

  • Universal caching for improved performance
  • Graceful degradation when services are unavailable
  • Conservative rate limiting to prevent API blocks
  • Error handling with fallback responses

🚀 Future Enhancements

🎯 Planned Features

  • Enhanced Search Integration

    • Multiple search engine support (Bing, Google Custom Search)
    • Improved news and image search capabilities
    • Real-time search result streaming

  • Advanced AI Features

    • Multi-language support
    • Voice input/output capabilities
    • Conversation memory and context

  • Performance Improvements

    • Search result caching optimization
    • Parallel AI processing
    • CDN integration for static assets

  • User Experience

    • Search history and bookmarks
    • Customizable UI themes
    • Mobile app development

🔬 Technical Improvements

  • Testing Suite

    • Unit tests for all components
    • Integration tests for API endpoints
    • End-to-end testing with Playwright

  • Monitoring & Analytics

    • Application performance monitoring
    • User analytics and insights
    • Error tracking and alerting

🤝 Contributing

We welcome contributions! Here's how you can help:

🛠️ Development Setup

  1. Fork the repository
  2. Create a feature branch: git checkout -b feature/amazing-feature
  3. Make your changes and test thoroughly
  4. Commit your changes: git commit -m 'Add amazing feature'
  5. Push to the branch: git push origin feature/amazing-feature
  6. Open a Pull Request

📋 Contribution Guidelines

  • Follow the existing code style and conventions
  • Add tests for new features
  • Update documentation as needed
  • Ensure all tests pass before submitting

🐛 Bug Reports

  • Use the GitHub issue tracker
  • Include detailed reproduction steps
  • Provide system information and logs
  • Add screenshots if applicable

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

  • Google Gemini AI for powerful language processing
  • DuckDuckGo for privacy-focused search capabilities
  • React & Tailwind CSS for modern UI development
  • Node.js & Express for robust backend infrastructure

📞 Support

⭐ Star this repository if you find it helpful!

Made with by Solomon Matthews