Skip to content

abpatramsft/deep_research_decision_framework

BranchesTags

Repository files navigation

AI Council

A multi-model LLM orchestration system that uses multiple decision-making frameworks to generate high-quality responses through collaborative deliberation among multiple AI models. This repository contains a FastAPI backend and a Vite/React frontend with additional "Super Chat" flows and optional web-search grounded agent variants.

Overview

AI Council supports multiple decision-making modes:

Council Mode (3-Stage Deliberation)

A collaborative approach where multiple LLM models work together to provide comprehensive answers:

  1. Stage 1: Each council member provides an individual response
  2. Stage 2: Each model ranks all responses (anonymized) to identify the best answers
  3. Stage 3: A chairman model synthesizes the top-ranked responses into a final answer

DxO Mode (Decision by Experts - 4-Stage Framework)

A specialized decision-making framework with dedicated expert agents:

  1. Stage 1: Lead Research Agent performs breadth-first research
  2. Stage 2: Critic Agent analyzes and critiques the research findings
  3. Stage 3: Domain Expert Agent provides specialized domain expertise
  4. Stage 4: Aggregator Agent synthesizes all inputs into a final comprehensive answer

Both approaches leverage the collective intelligence of multiple models to produce more reliable and well-rounded responses.

Features

  • 🤖 Multi-Model Collaboration: Uses multiple Groq models working in parallel
  • 📊 Multiple Decision Frameworks:
    • Council Mode: 3-stage deliberation with ranking system
    • DxO Mode: 4-stage expert-based decision framework
  • 🎯 Ranking System: Models evaluate and rank each other's responses (Council mode)
  • 🧠 Expert Agents: Specialized agents for research, critique, domain expertise, and synthesis (DxO mode)
  • ⚙️ Customizable Instructions: Provide optional user instructions for each agent in DxO mode
  • 💬 Conversation Management: Persistent conversation history with mode-based organization
  • 🗂️ Mode-Based Organization: Conversations organized by mode (Council, Super Chat, DxO, etc.)
  • 🗑️ Delete Conversations: Remove conversations you no longer need
  • 📥 Export Conversations: Download conversations as formatted text files
  • 🔍 Mode Filtering: Filter conversation history by specific mode
  • 🚀 Real-time Streaming: Server-Sent Events for live response updates
  • 🐳 Docker Support: Containerized deployment option
  • ⚡ Fast Development: Hot reload for both frontend and backend

Architecture

Council Process (3-Stage)

User Query
    ↓
┌─────────────────────────────────────┐
│  Stage 1: Individual Responses      │
│  - Model A → Response A             │
│  - Model B → Response B             │
│  - Model C → Response C             │
└─────────────────────────────────────┘
    ↓
┌─────────────────────────────────────┐
│  Stage 2: Ranking & Evaluation     │
│  - Each model ranks all responses   │
│  - Aggregate rankings calculated    │
└─────────────────────────────────────┘
    ↓
┌─────────────────────────────────────┐
│  Stage 3: Chairman Synthesis        │
│  - Top responses identified          │
│  - Final synthesized answer         │
└─────────────────────────────────────┘
    ↓
Final Response

DxO Process (4-Stage Decision by Experts)

User Query + Optional User Instructions
    ↓
┌─────────────────────────────────────┐
│  Stage 1: Lead Research Agent       │
│  - Breadth-first research            │
│  - Multiple angles & perspectives   │
│  - Comprehensive information        │
└─────────────────────────────────────┘
    ↓
┌─────────────────────────────────────┐
│  Stage 2: Critic Agent              │
│  - Critical analysis                │
│  - Identifies gaps & weaknesses    │
│  - Evaluates quality & validity    │
└─────────────────────────────────────┘
    ↓
┌─────────────────────────────────────┐
│  Stage 3: Domain Expert Agent       │
│  - Specialized domain knowledge     │
│  - Expert recommendations           │
│  - Addresses gaps from critique    │
└─────────────────────────────────────┘
    ↓
┌─────────────────────────────────────┐
│  Stage 4: Aggregator Agent          │
│  - Synthesizes all inputs            │
│  - Resolves contradictions           │
│  - Final comprehensive answer       │
└─────────────────────────────────────┘
    ↓
Final Response

Prerequisites

  • Python 3.11+ (for backend)
  • Node.js 20+ and npm (for frontend)
  • Azure / Groq credentials: you can run with Groq or Azure model deployments. See backend/config.py for environment variables.
  • Docker & Docker Compose (optional, for containerized deployment)

Installation

Authentication

  1. Create a Azure project with proper billing set up

  2. Make sure to sign in using the CLI command to authenticate before running your backend and front end.

az login
az login --use-device-code

Backend Setup

  1. Navigate to the backend directory:
cd backend
  1. Create a virtual environment and install dependencies:
python -m venv venv
# macOS / Linux
source venv/bin/activate
# Windows PowerShell
venv\Scripts\Activate.ps1
pip install -r requirements.txt

Frontend Setup

  1. Navigate to the frontend directory and install dependencies:
cd frontend
npm install

If npm run dev fails, ensure you are using Node 20+ and that frontend/package.json dependencies are installed.

Configuration

Configuration and API keys are managed in backend/config.py and via environment variables (supporting a .env file via python-dotenv). Key variables include:

  • GROQ_API_KEY — legacy / Groq key (optional)
  • AZURE_ENDPOINT, AZURE_API_KEY — Azure OpenAI / Azure AI endpoints and keys
  • LEAD_RESEARCH_AGENT, CRITIC_AGENT, DOMAIN_EXPERT_AGENT, AGGREGATOR_AGENT — Azure agent identifiers for web-search grounded DxO flows
  • AZURE_AI_PROJECT_ENDPOINT — endpoint for Azure AI Projects / Agents
  • DATA_DIR — path to conversation storage (default: data/conversations)

Edit backend/config.py or set environment variables in your shell or .env file before running the backend.

Model identifiers and agent names can be adjusted in backend/config.py to match your provider and deployments.

Running the Application

Whitelist your local network IP to use azure blob storage

  1. Get your current public IP
$MY_IP = (Invoke-WebRequest -Uri "https://api.ipify.org").Content
echo "Your IP: $MY_IP"
  1. Enable selected networks mode and add your IP
az storage account update `
  --name $STORAGE_ACCOUNT_NAME `
  --resource-group $RESOURCE_GROUP `
  --public-network-access Enabled `
  --default-action Deny
  1. Add your IP to the firewall
az storage account network-rule add `
  --account-name $STORAGE_ACCOUNT_NAME `
  --resource-group $RESOURCE_GROUP `
  --ip-address $MY_IP

There are multiple ways to run the backend and frontend during development.

Quick local start (recommended during development)

From the repository root you can run the provided helper script:

./startup.sh

This will start the backend and frontend with hot reload.

Backend (manual)

You can run the backend either with Uvicorn or directly as a script (the project supports both):

# Activate venv first (see Setup)
# Option A: uvicorn - run from root directory outside backend folder
uvicorn backend.main:app --host 0.0.0.0 --port 8000 --reload

# Option B: run directly (adds parent path and runs as script)
python main.py

The backend listens on http://localhost:8000 by default.

Frontend

cd frontend
npm run dev

If the frontend fails to start, check Node version and frontend/package.json dependency compatibility.

Docker

./docker-startup.sh
# or
docker-compose up --build

Stop with docker-compose down.

API Documentation

Base URL

  • Local: http://localhost:8000

Endpoints (high level)

Health Check

GET /

Conversations

  • GET /api/conversations?mode={mode} — list conversations (optional mode filter)
  • POST /api/conversations — create a conversation. Body: { "mode": "Council" } (default mode is Council)
  • GET /api/conversations/{conversation_id} — get full conversation
  • DELETE /api/conversations/{conversation_id} — delete conversation
  • GET /api/conversations/{conversation_id}/export — export as formatted text

Send Message (non-streaming)

POST /api/conversations/{conversation_id}/message
Body: {
  "content": "Your question",
  "user_instructions": { ... },        # optional
  "execution_mode": "sequential",    # Super Chat only: "sequential" or "parallel"
  "dxo_variant": "standard"          # Super Chat only: "standard" or "web_search"
}

Returns the complete response when finished. For mode values:

  • Council — 3-stage council process
  • DxO — 4-stage DxO process
  • Super Chat — combined Council + DxO flows (see below)
  • DxO-web_search — DxO run against web-search/agent-backed flows (Azure Agents)

Send Message (streaming)

POST /api/conversations/{conversation_id}/message/stream
Body: same as non-streaming endpoint

This returns Server-Sent Events (SSE) streaming the progress of each stage. SSE event payloads include a type field such as:

  • council_stage1_start / council_stage1_complete
  • council_stage2_start / council_stage2_complete
  • council_stage3_start / council_stage3_complete
  • dxo_stage1_start / dxo_stage1_complete
  • dxo_stage2_start / dxo_stage2_complete
  • dxo_stage3_start / dxo_stage3_complete
  • dxo_stage4_start / dxo_stage4_complete
  • aggregation_start / aggregation_complete (for parallel Super Chat)
  • title_complete (conversation title generation finished)
  • complete (flow finished)
  • error (contains message)

Client UIs (the frontend) should listen to the SSE stream and update the UI as events arrive.

Project Structure

backend/
├── main.py                     # FastAPI application & script entrypoint
├── council.py                  # 3-stage council logic
├── DxO.py                      # 4-stage DxO framework logic
├── DxO_web_search.py           # DxO variant using web-search / agent calls
├── superchat_seq.py            # Super Chat (sequential) orchestration
├── superchat_parallel.py       # Super Chat (parallel) orchestration and aggregator
├── superchat_web_search_seq.py # Super Chat sequential with web-search DxO
├── superchat_web_search_parallel.py # Super Chat parallel with web-search DxO
├── llm_client.py               # LLM client wrappers
├── llm_web_search_client.py    # Web-search / agent client wrappers
├── groq_client.py              # Groq API client (legacy)
├── storage.py                  # Conversation storage (JSON files)
├── config.py                   # Configuration and environment variables
├── requirements.txt            # Python dependencies
└── Dockerfile                  # Backend container config

frontend/
├── src/
│   ├── components/             # React components (modes: Council, DxO, Super Chat)
│   └── services/api.js         # frontend → backend API client
├── package.json                # Node dependencies
└── Dockerfile                  # Frontend container config

data/
└── conversations/              # Conversation JSON storage (shared / mounted)

docker-compose.yml
startup.sh
docker-startup.sh
README.md

Data Storage

Conversations are stored as JSON files in the data/conversations/ directory. Each conversation file contains:

  • id: Unique conversation identifier
  • created_at: ISO timestamp
  • title: Conversation title (auto-generated from first message)
  • mode: Conversation mode/type (e.g., "Council", "Super Chat", "DxO", "Ensemble", "Shoppr")
  • messages: Array of user and assistant messages
    • User messages: { "role": "user", "content": "..." }
    • Assistant messages (Council mode): { "role": "assistant", "stage1": [...], "stage2": [...], "stage3": {...}, "aggregate_rankings": [...], "label_to_model": {...} }
    • Assistant messages (DxO mode): { "role": "assistant", "stage1": {...}, "stage2": {...}, "stage3": {...}, "stage4": {...} }
      • Each stage contains: { "model": "...", "response": "..." }

The data directory is:

  • Local development: ./data/conversations/
  • Docker: Mounted as volume at ./data:/app/data

Conversation Management

Mode-Based Organization

Conversations are organized by mode, allowing you to separate different types of interactions:

  • Council: Multi-model deliberation with 3-stage ranking system (default)
  • DxO: Decision by Experts - 4-stage expert-based framework with specialized agents
  • Super Chat: Single-model conversations
  • Ensemble: Ensemble analysis mode
  • Shoppr: Shopping/research mode

DxO Mode Features

DxO (Decision by Experts) mode provides a specialized decision-making framework:

  • 4 Specialized Agents: Each agent has a specific role in the decision process
  • User Instructions: Optionally provide custom instructions for each agent directly in the UI
  • Integrated Workflow: Research → Critique → Domain Expertise → Final Synthesis
  • Agent-Specific Models: Each agent uses a model optimized for its role

To use DxO mode:

  1. Navigate to the "DxO" tab in the application
  2. Optionally add instructions for each agent in their respective cards
  3. Submit your query to start the 4-stage process
  4. View results from each stage as they complete

Conversation History

Access conversation history from the navigation bar:

  • View all conversations or filter by mode
  • Conversations are grouped by mode with visual indicators
  • Each conversation shows:
    • Title and creation date
    • Message count
    • Mode badge
    • Export and delete actions

Exporting Conversations

Export any conversation as a formatted text file:

  • Click the export icon (download) on any conversation
  • File includes complete conversation with all stages
  • Useful for archiving, sharing, or offline review

Deleting Conversations

Remove conversations you no longer need:

  • Click the delete icon (trash) on any conversation
  • Confirmation dialog prevents accidental deletion
  • Deletion is permanent and cannot be undone

Development

Hot Reload

Both development methods support hot reload:

  • Startup script: Automatic reload on code changes
  • Docker: Code mounted as volumes for live updates

Creating Conversations with Different Modes

When creating conversations programmatically, specify the mode:

// Frontend example
const conversation = await createConversation('Council');
const superChat = await createConversation('Super Chat');
const dxo = await createConversation('DxO');

// Sending message with user instructions (DxO mode)
await sendMessageStream(conversationId, query, onEvent, {
  lead_research: "Focus on recent developments",
  critic: "Pay special attention to methodology",
  domain_expert: "Consider industry best practices",
  aggregator: "Emphasize practical applications"
});
# Backend example
conversation = storage.create_conversation(conversation_id, mode="Council")
dxo_conversation = storage.create_conversation(conversation_id, mode="DxO")

# Sending message with user instructions (DxO mode)
user_instructions = {
    "lead_research": "Focus on recent developments",
    "critic": "Pay special attention to methodology",
    "domain_expert": "Consider industry best practices",
    "aggregator": "Emphasize practical applications"
}

Frontend Environment Variables

The frontend can be configured via environment variables:

  • VITE_API_BASE_URL: Backend API URL (default: http://localhost:8000)

Set in frontend/.env or pass to Docker container.

Backend Environment Variables

  • GROQ_API_KEY: Groq API key (optional, can be in config.py)
  • DATA_DIR: Data directory path (default: data/conversations)

Troubleshooting

Port Already in Use

If ports 8000 or 5173 are already in use:

  • Backend: Change port in startup.sh or docker-compose.yml
  • Frontend: Change port in frontend/vite.config.js

Docker Issues

  • Build fails: Ensure Docker is running and has sufficient resources
  • Data not persisting: Check volume mounts in docker-compose.yml
  • Services not connecting: Verify network configuration

API Key Issues

  • Ensure your Groq API key is valid
  • Check backend/config.py for correct key
  • Verify API key has sufficient quota

License

This project is open source and available for use and modification.

Contributing

Contributions are welcome! Please feel free to submit issues or pull requests.

Support

For issues or questions, please open an issue on the project repository.

Production Deployment

  1. The idea was to deploy this application to azure container apps - the container were pushed to ACR
  2. Azure container app environment was set up
  3. Backend and fronted was created
  4. nginx is updated to ensure it is able to accept http requests as opposed to https and backend allows unsecure transmission of requests
az containerapp ingress update --name backend --resource-group $RG --allow-insecure

To undo this insecure request acceptance change:

az containerapp ingress update --name backend --resource-group $RG --allow-insecure false
  1. Every service has separate docker files for local and prod deployment - to ensure a bit of segregation
  2. To run on local always refer the docker-compose.yaml; and docker-compose.prod.yaml is for prod deployment to azure container apps
  3. TODO: fix and update to a mroe secure way of communication between backend and frontend
  4. TODO: come up with a detailed production deployment guide

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

1 fork
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages