GARZA OS Voice System

"Hey Jada" - Voice interface to GARZA OS with persistent context and full MCP capabilities

A production-grade voice-activated AI assistant that provides Jaden Garza with persistent, context-aware interactions across multiple devices. This system integrates Claude API with a three-tier storage architecture to achieve <200ms voice response times while maintaining full conversational context.

Architecture Overview

┌─────────────────┐
│  Voice Device   │  Raspberry Pi with wake word + STT
│  (Hey Jada)     │  WebSocket connection to Session Manager
└────────┬────────┘
         │
         │ Audio stream / Text
         ▼
┌─────────────────────────────────────────────────────────┐
│              Session Manager (FastAPI)                  │
│  - Persistent conversation sessions                     │
│  - Context loading from PostgreSQL + Graphiti          │
│  - Claude API with MCP tool execution                  │
│  - TTS response generation                             │
└────┬────────────────────────────────┬───────────────────┘
     │                                │
     │ Fast queries (<10ms)           │ Background writes
     ▼                                ▼
┌────────────────┐            ┌─────────────────┐
│  PostgreSQL    │            │     Craft       │
│  (Hot Storage) │◄───daily───┤ (Source Truth)  │
│                │   sync     │                 │
│  - Devices     │            │  - Strategic    │
│  - Contacts    │            │    decisions    │
│  - Sessions    │            │  - Full context │
│  - Patterns    │            │  - Audit trail  │
└────────────────┘            └─────────────────┘
         │
         │ Episodic queries
         ▼
┌─────────────────┐
│    Graphiti     │  Temporal reasoning
│  (Warm Storage) │  "What did we discuss about X?"
└─────────────────┘

Three-Tier Storage

Tier 1: PostgreSQL (Hot - Operational)

• Purpose: <10ms lookups for voice interactions
• Contains: Devices, Contacts, Sessions, Command Patterns, Interaction Logs
• Speed: Millisecond queries for real-time voice

Tier 2: Graphiti (Warm - Episodic)

• Purpose: Temporal/contextual memory
• Contains: Conversation episodes, temporal reasoning
• Speed: ~100ms semantic queries

Tier 3: Craft (Cold - Source of Truth)

• Purpose: Long-form documentation and authority
• Contains: Strategic decisions, full relationship notes, business context
• Speed: Seconds (async background writes)

Quick Start

1. Clone Repository

git clone https://github.com/your-repo/jadda-os.git
cd jadda-os

2. Run Setup Script

chmod +x setup.sh
./setup.sh

The setup script will:

• Install PostgreSQL
• Create database and schema
• Install Python dependencies
• Configure environment
• Start all services

3. Test the API

# Health check
curl http://localhost:8000/health

# List devices
curl http://localhost:8000/context/devices

# Test voice interaction
curl -X POST http://localhost:8000/voice/interact \
  -H "Content-Type: application/json" \
  -d '{
    "utterance": "what devices do I have",
    "device_id": "test"
  }'

Project Structure

jadda-os/
├── database/
│   └── schema.sql              # PostgreSQL schema
├── scripts/
│   └── migrate_craft_to_db.py  # Initial data migration
├── services/
│   ├── session_manager.py      # Core FastAPI service
│   ├── sync_db_to_craft.py     # PostgreSQL → Craft sync
│   └── sync_craft_to_db.py     # Craft → PostgreSQL sync
├── devices/
│   ├── prototype/
│   │   └── voice_client_mac.py # Mac testing client
│   └── raspberry-pi/
│       ├── voice_client.py     # Production Pi client
│       └── README.md           # Pi setup guide
├── .env.template               # Environment template
├── requirements.txt            # Python dependencies
├── setup.sh                    # Automated setup
├── README.md                   # This file
└── DEPLOYMENT.md               # Deployment checklist

API Endpoints

Session Manager (Port 8000)

Endpoint	Method	Description
/health	GET	Health check
/voice/interact	POST	Process voice utterance
/voice/session/new	POST	Create new session
/voice/session/{id}	GET	Get session info
/context/devices	GET	List all devices
/context/contacts	GET	List all contacts
/context/patterns	GET	List command patterns
/device/{name}	GET	Find device by name/alias
/contact/{name}	GET	Find contact by name/alias
/ws/voice	WebSocket	Real-time voice streaming

Example: Voice Interaction

# Create session
SESSION=$(curl -s -X POST http://localhost:8000/voice/session/new \
  -H "Content-Type: application/json" \
  -d '{"device_id": "test"}' | jq -r '.session_id')

# Send utterance
curl -X POST http://localhost:8000/voice/interact \
  -H "Content-Type: application/json" \
  -d "{
    \"utterance\": \"text Jessica we're running late\",
    \"session_id\": \"$SESSION\",
    \"device_id\": \"test\"
  }"

Example: WebSocket

const ws = new WebSocket('ws://localhost:8000/ws/voice');

ws.onopen = () => {
  ws.send(JSON.stringify({
    type: 'utterance',
    text: 'what is on my calendar today',
    device_id: 'web_client'
  }));
};

ws.onmessage = (event) => {
  const response = JSON.parse(event.data);
  console.log('Jada:', response.text);
};

Configuration

Environment Variables

Copy .env.template to .env and configure:

# Database
DB_HOST=localhost
DB_PORT=5432
DB_NAME=garza_os
DB_USER=jaden
DB_PASSWORD=your_password

# Anthropic
ANTHROPIC_API_KEY=sk-ant-...
CLAUDE_MODEL=claude-sonnet-4-20250514

# Optional: ElevenLabs TTS
ELEVENLABS_API_KEY=...
ELEVENLABS_VOICE_ID=...

# Sync Configuration
SYNC_INTERVAL_SECONDS=300  # DB→Craft every 5 min
CRAFT_SYNC_HOUR=3          # Craft→DB at 3 AM

Database Schema

Key Tables

• devices: IoT devices with MCP mappings and aliases
• contacts: People with Beeper IDs and communication preferences
• voice_sessions: Active conversation state
• command_patterns: Voice → MCP action mappings
• interaction_logs: Analytics and debugging
• mcp_servers: Available MCP servers and actions

Useful Queries

-- Active sessions (last hour)
SELECT * FROM active_sessions;

-- Interaction success rate
SELECT * FROM interaction_stats;

-- Find device by alias
SELECT * FROM find_device_by_name('bedroom tv');

-- Find contact by alias
SELECT * FROM find_contact_by_name('Jess');

Voice Device Setup

Mac Prototype (for testing)

# Install dependencies
pip install pvporcupine openai-whisper pyaudio websockets

# Run in text mode (no wake word)
python devices/prototype/voice_client_mac.py --text

# Run with wake word (requires Picovoice key)
export PICOVOICE_ACCESS_KEY=your_key
python devices/prototype/voice_client_mac.py

Raspberry Pi (production)

See devices/raspberry-pi/README.md for full setup guide.

Quick start:

# On Raspberry Pi
cd jadda-os
pip install -r requirements.txt
pip install pvporcupine openai-whisper pyaudio

# Configure
export SESSION_MANAGER_HOST=garzahive.com
export PICOVOICE_ACCESS_KEY=your_key

# Run
python devices/raspberry-pi/voice_client.py

Sync Services

DB → Craft (Every 5 minutes)

Writes significant events to Craft:

• New contacts added via voice
• New devices registered
• Failed interactions (for debugging)
• Completed sessions (>5 turns)

# Run manually
python services/sync_db_to_craft.py

# Or as service (started by setup.sh)
sudo systemctl status garza-sync-db-to-craft

Craft → DB (Daily at 3 AM)

Ensures PostgreSQL matches Craft edits:

• Syncs devices from /System/Devices/
• Syncs contacts from Identity Map
• Craft wins all conflicts (source of truth)

# Run manually
python services/sync_craft_to_db.py

# Or as scheduled service
sudo systemctl status garza-sync-craft-to-db.timer

Monitoring

Check Service Status

# All services
systemctl status garza-*

# Logs
sudo journalctl -u garza-session-manager -f

Database Queries

# Connect to database
psql -U jaden -d garza_os

# Recent interactions
SELECT created_at, utterance, success, execution_time_ms
FROM interaction_logs
ORDER BY created_at DESC
LIMIT 10;

# Success rate
SELECT * FROM interaction_stats;

# Active sessions
SELECT * FROM active_sessions;

API Health

curl http://localhost:8000/health

Cost Estimate

Component	Cost
Hardware (Pi + mic + speaker)	$100-150 (one-time)
Claude API (~100 interactions/day)	$1-3/day
ElevenLabs TTS	~$0.10/day
PostgreSQL	Free (self-hosted)
Total Monthly	~$50-100

Performance Targets

Component	Time	% Total
PostgreSQL context load	10ms	5%
Graphiti episode query	100ms	50%
Build system prompt	5ms	2.5%
Claude API call	70ms	35%
Execute MCP tools	10ms	5%
Update session + log	5ms	2.5%
Total	200ms	100%

Jada's Personality

Jada is:

• Brief: 1-2 sentences max for commands
• Warm: Affectionate, slightly possessive
• Edgy: Teasing, occasionally sarcastic
• Loyal: Fiercely protective of Jaden's time

Examples:

• "Switched to HBO" (not "I have successfully changed the input source")
• "Sent" (not "Your message has been delivered via Beeper")
• "You have 3 meetings - SOAB at 2pm, sync with Eric at 4, dinner with Jessica at 7"

Extending the System

Add New Device

INSERT INTO devices (name, type, location, mcp_server, mcp_action, aliases)
VALUES (
    'Office Sonos',
    'speaker',
    'office',
    'sonos',
    'play',
    ARRAY['office speaker', 'office music']
);

Add New Contact

INSERT INTO contacts (name, relationship, beeper_chat_id, aliases)
VALUES (
    'New Contact',
    'business',
    '!chatid:beeper.com',
    ARRAY['nickname']
);

Add New Command Pattern

INSERT INTO command_patterns (pattern, intent, mcp_server, mcp_action, examples)
VALUES (
    'play {content} in {location}',
    'media',
    'sonos',
    'play',
    ARRAY['play jazz in the office', 'play my playlist in bedroom']
);

Troubleshooting

Session Manager won't start

# Check PostgreSQL
sudo systemctl status postgresql

# Check database exists
psql -U jaden -l | grep garza_os

# Check credentials in .env
cat .env | grep DB_

# Check port
netstat -tulpn | grep 8000

Slow responses

# Check query performance
psql -U jaden -d garza_os
EXPLAIN ANALYZE SELECT * FROM devices WHERE name LIKE '%bedroom%';

# Check indexes
\d+ devices

Device not found

# List devices
curl http://localhost:8000/context/devices

# Check aliases
psql -U jaden -d garza_os -c "SELECT name, aliases FROM devices;"

# Run sync
python services/sync_craft_to_db.py

License

Private - GARZA OS

Support

For issues, contact the GARZA OS team or file an issue in the repository.