Skip to content
This repository was archived by the owner on Mar 14, 2026. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -30,3 +30,13 @@ PANW_MGMT_TSG_ID=
# ── Tuning ───────────────────────────────────────────────────────────
SCAN_CONCURRENCY=5
PROPAGATION_DELAY_MS=10000
# ACCUMULATE_TESTS=false
# MAX_ACCUMULATED_TESTS=

# ── Persistence ──────────────────────────────────────────────────────
# DATA_DIR=~/.daystrom/runs

# ── Memory ───────────────────────────────────────────────────────────
# MEMORY_ENABLED=true
# MEMORY_DIR=~/.daystrom/memory
# MAX_MEMORY_CHARS=3000
15 changes: 12 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ src/
β”‚ β”‚ β”œβ”€β”€ resume.ts # Resume paused/failed run from disk
β”‚ β”‚ β”œβ”€β”€ report.ts # View run results by ID
β”‚ β”‚ β”œβ”€β”€ list.ts # List all saved runs
β”‚ β”‚ β”œβ”€β”€ runtime.ts # Runtime scanning (scan, bulk-scan, resume-poll)
β”‚ β”‚ β”œβ”€β”€ runtime.ts # Runtime scanning (scan, bulk-scan, resume-poll) + config management (profiles, topics, api-keys, customer-apps, deployment-profiles, dlp-profiles, scan-logs)
β”‚ β”‚ β”œβ”€β”€ audit.ts # Profile-level multi-topic evaluation
β”‚ β”‚ β”œβ”€β”€ redteam.ts # Red team operations (scan, targets CRUD, prompt-sets CRUD, prompts CRUD, properties)
β”‚ β”‚ └── modelsecurity.ts # Model security operations (groups, rules, rule-instances, scans, labels, pypi-auth)
Expand All @@ -72,7 +72,7 @@ src/
β”œβ”€β”€ airs/
β”‚ β”œβ”€β”€ scanner.ts # AirsScanService + DebugScanService β€” syncScan + scanBatch
β”‚ β”œβ”€β”€ runtime.ts # SdkRuntimeService β€” sync scan, async bulk scan, poll results, CSV export
β”‚ β”œβ”€β”€ management.ts # SdkManagementService β€” topic CRUD + profile linking
β”‚ β”œβ”€β”€ management.ts # SdkManagementService β€” topic CRUD, profile CRUD, API keys, customer apps, deployment/DLP profiles, scan logs
β”‚ β”œβ”€β”€ promptsets.ts # SdkPromptSetService β€” custom prompt set CRUD via RedTeamClient
β”‚ β”œβ”€β”€ redteam.ts # SdkRedTeamService β€” red team scan CRUD, polling, reports
β”‚ β”œβ”€β”€ modelsecurity.ts # SdkModelSecurityService β€” security groups, rules, scans, labels
Expand Down Expand Up @@ -142,7 +142,7 @@ tests/
- **Detection**: Both block and allow intents use `triggered` (= `topic_violation`) as the sole guardrail detection signal. No category-based or action-based detection.
- **`DebugScanService`**: Wrapper that appends raw scan responses to a JSONL file when `--debug-scans` is passed
- **Prompt sets**: `SdkPromptSetService` wraps `RedTeamClient.customAttacks` for custom prompt set CRUD; `--create-prompt-set` auto-creates a prompt set from the best iteration's test cases
- **Management**: `ManagementClient` for topic CRUD + profile linking via OAuth2
- **Management**: `ManagementClient` via OAuth2 β€” topic CRUD, security profile CRUD, API key management, customer app management, deployment/DLP profile listing, scan log querying, plus profile linking for guardrail generation
- Profile updates create **new revisions with new UUIDs** β€” always reference profiles by name, never ID
- Topics must be added to profile's `model-protection` β†’ `topic-guardrails` β†’ `topic-list`
- AIRS rejects empty `topic-list` entries β€” only include entries with topics (no empty opposite-action entry)
Expand All @@ -155,6 +155,7 @@ tests/
### Runtime Scanning (`src/airs/runtime.ts`)
- `SdkRuntimeService` wraps SDK `Scanner` for sync and async scanning
- `scanPrompt()` β€” sync scan via `syncScan()`, normalizes to `RuntimeScanResult`
- **Detection scope**: `scanPrompt()` aggregates 6 detection types via OR (`topic_violation`, `injection`, `toxic_content`, `dlp`, `url_cats`, `malicious_code`). This is intentionally broader than the guardrail loop's `topic_violation`-only signal β€” runtime scanning is a general-purpose firewall check, not topic-specific evaluation.
- `submitBulkScan()` β€” batches prompts into groups of 5 `AsyncScanObject` items, calls `asyncScan()` per batch; optional `sessionId` for AIRS Sessions UI grouping
- `pollResults()` β€” sweeps all pending scan IDs in batches of 5 per cycle; retries on rate limit with exponential backoff (10s base); retry level decays by 1 after a full successful sweep (not per-batch); inter-batch and inter-sweep delays scale with rate limit pressure
- `formatResultsCsv()` β€” static method producing CSV from results
Expand All @@ -163,6 +164,14 @@ tests/
- Input file parsing: `.csv` files extract the `prompt` column by header; `.txt`/extensionless use line-per-prompt
- Bulk scan IDs are saved to `~/.daystrom/bulk-scans/` before polling β€” survives rate limit crashes
- CLI: `daystrom runtime resume-poll <stateFile> [--output <file>]` β€” resume polling from saved scan IDs
- CLI config management subcommand groups (all via `ManagementClient` OAuth2):
- `daystrom runtime profiles {list,get,create,update,delete}` β€” security profile CRUD (supports `--force --updated-by`)
- `daystrom runtime topics {list,get,create,update,delete}` β€” custom topic CRUD (supports `--force --updated-by`)
- `daystrom runtime api-keys {list,create,regenerate,delete}` β€” API key management (`regenerate` takes `--interval`/`--unit`)
- `daystrom runtime customer-apps {list,get,update,delete}` β€” customer app CRUD
- `daystrom runtime deployment-profiles {list}` β€” deployment profile listing (`--unactivated` filter)
- `daystrom runtime dlp-profiles {list}` β€” DLP profile listing
- `daystrom runtime scan-logs {query}` β€” scan log querying (`--interval`/`--unit hours`/`--filter`)

### Red Team (`src/airs/redteam.ts`, `src/airs/promptsets.ts`)
- `SdkRedTeamService` wraps `RedTeamClient` for scan CRUD, polling, reports, **target CRUD**
Expand Down
3 changes: 3 additions & 0 deletions src/airs/runtime.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,9 @@ export class SdkRuntimeService implements RuntimeService {
const res = await this.scanner.syncScan({ profile_name: profileName }, content, undefined);

const detected = (res.prompt_detected as Record<string, boolean> | undefined) ?? {};
// Runtime scanning aggregates all 6 detection types β€” intentionally broader than
// the guardrail loop's topic_violation-only signal. Runtime is a general-purpose
// firewall check, not topic-specific evaluation.
const triggered = !!(
detected.topic_violation ||
detected.injection ||
Expand Down
Loading