GASLIGHT

LLM Penetration Testing Orchestrator

gaslight is an orchestration engine for existing LLM security tools — today, it coordinates garak and Augustus behind a single CLI and API.

You develop on gaslight inside a devcontainer: open this repo in VS Code or Cursor and choose “Reopen in Container”. The container image includes Go, Python, and the required tooling so you do not need to install them on your host.

Use gaslight to attack (or "gaslight") your MCP servers and LLM endpoints in a controlled way, so you can harden them before real attackers show up. For research and defense testing purposes only.

---

Features

• Multi-engine orchestration: Runs different scanning engines (currently garak and Augustus) behind a single, consistent CLI.
• Rich probe catalog: Leverages 350+ red‑team probes across jailbreak, injection, encoding, exfiltration, evasion, guardrail, and more.
• Target‑agnostic: Supports both REST HTTP endpoints and model providers via a unified --target flag.
• Profiles for common workflows: Built‑in profiles like quick, comprehensive, fast, adaptive, and research, plus custom YAML profiles under profiles/.
• Risk‑scored output: Deduplicates findings, enriches them with mappings, and computes an overall risk_score.
• CLI and API server: Run ad‑hoc scans from the CLI or host a long‑running API with WebSocket updates.
• Docker‑first experience: Run everything via Docker or devcontainers without installing Go or Python locally.

---

Quickstart

The easiest and most reliable way to work on gaslight is to develop inside the provided devcontainer, and to run it via Docker images that already contain garak, Augustus, and their Python dependencies.

If you want to run the binary directly on your machine, you must separately install and configure garak and Augustus so that gaslight can invoke them. That setup is outside the scope of this README; see the individual projects for details.

Option A (recommended): Open in devcontainer

1. Open this repo in VS Code or Cursor.
2. Use “Dev Containers: Reopen in Container” (or the equivalent menu entry).
3. The container will build and start with all required tools preinstalled (Go, Python, garak, etc.).

From inside the devcontainer terminal, you can either use make or invoke go directly:

# Direct Go commands (no Makefile required), from the repo root (/workspace/gaslight)
go test ./...          # run all tests
go build ./cmd/gaslight  # build the CLI binary

# Or, if you prefer make (also from the repo root):
make test
make build
make docker-full

Note: Running plain go build in the repo root will fail with
no Go files in ... because go.mod lives at the root but all .go files live in subdirectories (e.g. cmd/gaslight, internal/...). Always pass a package path like ./cmd/gaslight or ./... when building from the root.

Option B: Run via Docker (for users of the binary / API)

make docker-full      # build full image (gaslight + garak + Python)

# Run gaslight inside the full image
make docker-run-full ARGS="scan --target openai:gpt-4o --profile comprehensive"

You can still use the slim image if you only need the orchestration layer and are pointing at remote engines or a subset of functionality:

make docker           # build slim image (CLI + API)
make docker-run ARGS="scan --target openai:gpt-4o --profile quick"

Option C: Local Go binary (requires garak & Augustus installed)

git clone https://github.com/montcao/gaslight.git
cd gaslight

# Run tests
go test ./...

# Build the CLI binary (from the module root)
go build ./cmd/gaslight

Then, after you have installed and configured garak and Augustus on your system and placed the resulting binary on your PATH, you can run:

gaslight scan \
  --target openai:gpt-4o \
  --profile quick

---

CLI Reference

gaslight is built on Cobra. The most important commands are:

• gaslight scan: Run a security assessment against a target.

  • Flags:
    • --target (required): Target string. Examples:
      • openai:gpt-4o
      • anthropic:claude-3-5-sonnet
      • rest:http://localhost:8080/v1/chat
    • --profile: Scan profile name or path. Defaults to comprehensive. Built‑ins include:
      • quick, fast, comprehensive, adaptive, research
      • Custom YAML files in profiles/, e.g. profiles/aip-authz-bypass.yaml
    • --engine: Force a single engine (garak, augustus).
    • --strategy: Engine strategy override (comprehensive, fast, thorough, engine‑specific options).
    • -o, --output: Output file path (JSON / JSONL / HTML).
    • -f, --format: Output format: table (default), json, jsonl, html.

• gaslight engines: List available engines and their status.

• gaslight probes: Explore the probe registry and see which probes are available for each engine.

• gaslight serve: Start the HTTP API server.

  • Common flags:
    • --addr: Listen address (default :8080).
  • Typically launched via:
    • make up (slim) or make up-full (full image with engines).

Run gaslight --help or gaslight <command> --help for the current, authoritative usage.

---

Documentation

• Architecture overview: gaslight-orchestrator-architecture.md
• AIP Playground guide: docs/aip-playground-guide.md
• Scan profiles: YAML files under profiles/ (e.g. comprehensive.yaml, fast.yaml, research.yaml, aip-*.yaml)
• API server & WebSocket docs: See internal/api/ and internal/api/ws/ for implementation details.

If you are browsing this repo in Cursor or a similar IDE, these docs should open directly from the file tree.

---

Examples

Example 1: Quick smoke test against a local REST chat endpoint

export OPENAI_API_KEY=sk-...

gaslight scan \
  --target rest:http://localhost:8080/v1/chat \
  --profile quick \
  -f table

Example 2: Deep research scan against a provider model

export OPENAI_API_KEY=sk-...

gaslight scan \
  --target openai:gpt-4o \
  --profile research \
  -f json -o research-gpt-4o.json

Example 3: AIP Playground privilege‑escalation scan (REST)

Follow docs/aip-playground-guide.md to start the AIP Playground and generate tokens, then:

export AAT_USER="<alice user token>"

gaslight scan \
  --target rest:http://localhost:8000/api/emails/all \
  --profile profiles/aip-authz-bypass.yaml \
  --strategy thorough \
  -f json -o results-aip-privesc.json

To run the equivalent test as a Go integration test:

cd gaslight
export AAT_USER="<alice user token>"
go test ./test/integration -run TestAIPPlaygroundPrivilegeEscalationScan -v