GovernsAI Precheck

Policy evaluation and PII redaction service for GovernsAI. This service provides real-time policy evaluation and PII detection/redaction for AI tool usage.

Features

• Policy Evaluation: Real-time evaluation of AI tool usage against configurable policies
• PII Detection & Redaction: Advanced PII detection using Microsoft Presidio with regex fallback
• API Key Authentication: Secure API key-based authentication
• Rate Limiting: Redis-based rate limiting (optional)
• Database Storage: SQLAlchemy-based storage with PostgreSQL/SQLite support
• Docker Support: Production-ready Docker containerization

Quick Start

Local Development

1. Install dependencies:

   pip install -r requirements.txt
   python -m spacy download en_core_web_sm

2. Run the service:

   python start.py
   # or
   uvicorn app.main:app --reload --host 0.0.0.0 --port 8080

3. Test the service:

   curl -X GET http://localhost:8080/api/v1/health

Docker

1. Build the image:

   docker build -t governsai-precheck .

2. Run the container:

   docker run -p 8080:8080 governsai-precheck

API Endpoints

Health Check

GET /api/v1/health

Returns service status and version information.

Precheck

POST /api/v1/precheck
X-Governs-Key: your-api-key
Content-Type: application/json

{
  "tool": "web.fetch",
  "scope": "net.external",
  "raw_text": "Please fetch data from https://example.com for user@example.com",
  "tags": ["research"],
  "corr_id": "req-123"
}

Response:

{
  "decision": "transform",
  "payload": {
    "url": "https://example.com",
    "email": "u***@example.com"
  },
  "reasons": ["pii.redacted:email"],
  "policy_id": "net-redact-presidio",
  "ts": 1703123456
}

Postcheck

POST /api/v1/postcheck
X-Governs-Key: your-api-key
Content-Type: application/json

Similar to precheck but for post-execution validation.

Configuration

The service can be configured via environment variables:

Variable	Default	Description
APP_BIND	0.0.0.0:8080	Server bind address
DB_URL	sqlite:///./local.db	Database connection URL
USE_PRESIDIO	true	Enable Presidio PII detection
PRESIDIO_MODEL	en_core_web_sm	spaCy model for Presidio
WEBHOOK_URL	None	Webhook URL for dashboard events
PRECHECK_DLQ	/tmp/precheck.dlq.jsonl	Dead letter queue file path

PII Detection

The service uses Microsoft Presidio for advanced PII detection with the following capabilities:

• Email addresses: Detected and masked
• Phone numbers: International format detection and masking
• Credit cards: Luhn algorithm validation and masking
• API keys: Custom patterns for various API key formats
• JWT tokens: Detection and redaction
• SSN: US Social Security Number detection
• IP addresses: IPv4/IPv6 detection

Fallback Mode

If Presidio fails to initialize, the service falls back to regex-based detection for basic PII types (email, phone, credit card).

Webhook Events

The service emits fire-and-forget webhook events for all precheck and postcheck decisions to enable dashboard integration and audit logging.

Event Structure

{
  "userId": "user123",
  "tool": "web.fetch",
  "scope": "net.external",
  "decision": "transform",
  "policyId": "net-redact-presidio",
  "reasons": ["pii.redacted:email"],
  "payloadHash": "sha256_hash_of_payload",
  "latencyMs": 45,
  "timestamp": 1703123456,
  "correlationId": "req-123",
  "tags": ["research"],
  "direction": "precheck"
}

Configuration

• Set WEBHOOK_URL environment variable to enable webhook emission
• Failed webhook deliveries are written to DLQ file (configurable via PRECHECK_DLQ)
• Webhook includes retry logic with exponential backoff (0.5s, 1s, 2s)

Testing

Use the provided webhook test URL for development:

export WEBHOOK_URL="https://webhook-test.com/1508b1ea2414ed242d2b8abf6ea66616"

Policy Configuration

Denied Tools

The following tools are automatically denied:

• python.exec
• bash.exec
• code.exec
• shell.exec

Network Tools

Tools with network scope or web/http prefixes trigger PII redaction:

• web.*
• http.*
• fetch.*
• request.*
• net.* scope

Development

Development Setup

# Install development dependencies
pip install -r requirements-dev.txt

# Download spaCy model
python -m spacy download en_core_web_sm

Running Tests

pytest tests/

Code Formatting

black app/ tests/
isort app/ tests/

Type Checking

mypy app/

Deployment Modes

Cloud Mode

• Bind to 0.0.0.0:8080
• Use PostgreSQL database
• Use Redis for rate limiting
• Set PUBLIC_BASE for external access

Local Sidecar Mode

• Bind to 127.0.0.1:7071
• Use SQLite database
• Optional Redis
• Local development setup

License

Elastic License 2.0 - see LICENSE file for details.