Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
8ad134a
test(security): add E2E tests for command injection and credential sa…
jyaunches Mar 25, 2026
3ca3da0
ci: temporarily disable repo guard for fork testing
jyaunches Mar 25, 2026
720b16f
ci: bump bootstrap timeout, skip vLLM on CPU E2E runs
jyaunches Mar 25, 2026
3626cee
ci: bump bootstrap timeout to 40 min for sandbox image build
jyaunches Mar 25, 2026
1e40af1
ci: bump Brev instance to 8x32 for faster Docker builds
jyaunches Mar 25, 2026
a56ddf5
ci: add real-time progress streaming for E2E bootstrap and tests
jyaunches Mar 25, 2026
054488f
ci: use NemoClaw launchable for E2E bootstrap
jyaunches Mar 25, 2026
ab32c2b
fix: prevent openshell sandbox create from hanging in non-interactive…
jyaunches Mar 25, 2026
dfe6c89
fix: source nvm in non-interactive SSH for launchable path
jyaunches Mar 25, 2026
fc9229a
fix: setup.sh respects NEMOCLAW_SANDBOX_NAME env var
jyaunches Mar 25, 2026
8704eaf
ci: bump full E2E test timeout to 15 min for install + sandbox build
jyaunches Mar 25, 2026
2fef180
ci: don't run full E2E alongside security tests (it destroys the sand…
jyaunches Mar 25, 2026
f83f0af
ci: pre-build base image locally when GHCR image unavailable
jyaunches Mar 25, 2026
f13e81f
ci: install nemoclaw CLI after bootstrap in non-launchable path
jyaunches Mar 26, 2026
8393d8a
fix: use npm_config_prefix for nemoclaw CLI install so it lands on PATH
jyaunches Mar 26, 2026
8335ba9
fix: npm link from repo root where bin.nemoclaw is defined
jyaunches Mar 26, 2026
306fc1f
fix(ci): register sandbox in nemoclaw registry after setup.sh bootstrap
jyaunches Mar 27, 2026
50ca58f
style: shfmt formatting fix in setup.sh
jyaunches Mar 27, 2026
de1aa1f
fix(test): exclude policy presets from C7 secret pattern scan
jyaunches Mar 27, 2026
2271a06
docs(ci): add test suite descriptions to e2e-brev workflow header
jyaunches Mar 30, 2026
73ab4f1
ci: re-enable repo guard for e2e-brev workflow
jyaunches Mar 30, 2026
6dc2493
fix(test): update setup-sandbox-name test for NEMOCLAW_SANDBOX_NAME e…
jyaunches Mar 30, 2026
7f04a9b
fix(lint): add shellcheck directives for injection test payloads and …
jyaunches Mar 30, 2026
5308e74
fix(lint): suppress SC2034 for status_output in credential sanitizati…
jyaunches Mar 30, 2026
32687e3
fix: address CodeRabbit review — timeout, pipefail, fail-closed probe…
jyaunches Mar 30, 2026
87996cc
Merge branch 'main' into feat/security-e2e-tests
cv Mar 30, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 35 additions & 2 deletions .github/workflows/e2e-brev.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,28 @@

name: e2e-brev

# Ephemeral Brev E2E: provisions a cloud instance, bootstraps NemoClaw,
# runs test suites remotely, then tears down. Use workflow_dispatch to
# trigger manually from the Actions tab, or workflow_call from other workflows.
#
# Test suites:
# full — Install → onboard → sandbox verify → live inference
# against NVIDIA Endpoints → CLI operations. Tests the
# complete user journey. (~10 min, destroys sandbox)
# credential-sanitization — 24 tests validating PR #743: credential stripping from
# migration snapshots, auth-profiles.json deletion, blueprint
# digest verification, symlink traversal protection, and
# runtime sandbox credential checks. Requires running sandbox.
# telegram-injection — 18 tests validating PR #584: command injection prevention
# through $(cmd), backticks, quote breakout, ${VAR} expansion,
# process table leak checks, and SANDBOX_NAME validation.
# Requires running sandbox.
# all — Runs credential-sanitization + telegram-injection (NOT full,
# which destroys the sandbox the security tests need).
#
# Required secrets: BREV_API_TOKEN, NVIDIA_API_KEY
# Instance cost: Brev CPU credits (~$0.10/run for 4x16 instance)

on:
workflow_dispatch:
inputs:
Expand All @@ -15,14 +37,20 @@ on:
required: false
default: ""
test_suite:
description: "Test suite to run"
description: "Test suite to run (see workflow header for descriptions)"
required: true
default: "full"
type: choice
options:
- full
- credential-sanitization
- telegram-injection
- all
use_launchable:
description: "Use NemoClaw launchable (true) or bare brev-setup.sh (false)"
required: false
type: boolean
default: true
keep_alive:
description: "Keep Brev instance alive after tests (for SSH debugging)"
required: false
Expand All @@ -41,6 +69,10 @@ on:
required: false
type: string
default: "full"
use_launchable:
required: false
type: boolean
default: true
keep_alive:
required: false
type: boolean
Expand All @@ -64,7 +96,7 @@ jobs:
e2e-brev:
if: github.repository == 'NVIDIA/NemoClaw'
runs-on: ubuntu-latest
timeout-minutes: 45
timeout-minutes: 90
steps:
- name: Checkout target branch
uses: actions/checkout@v6
Expand Down Expand Up @@ -110,6 +142,7 @@ jobs:
GITHUB_TOKEN: ${{ github.token }}
INSTANCE_NAME: e2e-pr-${{ inputs.pr_number || github.run_id }}
TEST_SUITE: ${{ inputs.test_suite }}
USE_LAUNCHABLE: ${{ inputs.use_launchable && '1' || '0' }}
KEEP_ALIVE: ${{ inputs.keep_alive }}
run: npx vitest run --project e2e-brev --reporter=verbose

Expand Down
Loading
Loading