Skip to content

fix(critical): close silent fail-open patterns in protect() init and rule eval#203

Open
anirudhp26 wants to merge 1 commit intomasterfrom
ani/audit-A-critical-failopen
Open

fix(critical): close silent fail-open patterns in protect() init and rule eval#203
anirudhp26 wants to merge 1 commit intomasterfrom
ani/audit-A-critical-failopen

Conversation

@anirudhp26
Copy link
Copy Markdown
Contributor

Audit Round 2 — PR A. Closes the two CRITICAL fail-open findings: silent paths that turned a misconfigured Veto into a working-looking allow-all.

Findings closed

1. protect() silently degraded to allow-all on init exception (Python)

Beforepackages/sdk-python/veto/core/protect.py:439-440:

try:
    instance = await Veto.init(...)
except Exception:
    instance = _create_allow_all_instance(options)  # silent

Any failure during init — typo in YAML, bad rule shape, transient network — silently replaced the Veto instance with an allow-all wrapper. Every tool call passed. No warning. No audit trail. The system looked healthy from the outside.

After:

  • Default: raise. The caller sees what broke.
  • Opt-in protect(safe_fallback=True): keeps the legacy degrade-to-allow-all but prints a loud stderr banner:
    [veto] WARNING: Veto initialization failed — falling back to
    ALLOW-ALL because safe_fallback=True was set. Every tool call
    will be permitted with no policy enforcement. Original error: ...
    

2. Veto.evaluateLocalExpression returned false on compile / eval errors (TS)

packages/sdk/src/core/veto.ts:2030-2046 caught both compile and evaluate errors, logged at warn, and returned false. A block rule with a parse error silently never matched → the call sailed through.

Both catches now log at error level and re-raise. The validator engine's existing fail-closed treatment of validator exceptions turns this into a deny — security-relevant rules can no longer be silently bypassed by a broken expression.

3. matches operator on a rejected regex now logs a one-time error (both SDKs)

The matches operator returns false when the safety heuristic rejects the pattern (preserves matches-operator semantics — a non-matching pattern isn't a hard error). Earlier this was completely silent; now a single [veto] ERROR: rule \matches` regex rejected by safety heuristic — the rule will never fire` line is written to stderr the first time each distinct pattern hits this path.

Pairs with the load-time scan added in #201 — that catches rules at startup; this catches rules added or mutated after init.

Tests

  • packages/sdk-python/tests/test_failopen_critical.py — 6 cases
  • packages/sdk/tests/core/failopen-critical.test.ts — 4 cases

Full suites green: TS 1386/1386, Python 334/334.

Behavioural notes for downstream

  • protect() callers that relied on the allow-all degradation as a soft fallback need to either fix their config or set safe_fallback=True and accept that every call is now permitted (with the loud banner).
  • Local rules with broken expressions now produce a deny instead of an allow. For most users this is the safer default; broken expressions should be fixed.

Test plan

  • In a project with a deliberately bad rule (malformed YAML or unparseable expression), confirm protect() raises and the error message points at the bad rule.
  • Set safe_fallback=True in the same scenario; confirm the loud stderr banner prints and tools still wrap.
  • Author a rule with matches: '(rm.*|wget.*)' (the heuristic rejects this); on first guard call, confirm a single ERROR line on stderr; on subsequent calls, confirm no further logs for that pattern.
  • Author a rule with a broken expression like args.x ==; confirm the first guard call denies (rather than silently allowing).

…rule eval

Audit Round 2, PR A — addresses the two CRITICAL findings: silent
fail-opens that masked broken policies as a working Veto instance.

1. **`protect()` no longer silently degrades to allow-all on init
   failure.** Previously, any exception during Veto initialization
   (malformed YAML, bad rule, transient network) was caught and
   replaced with an allow-all wrapper. Every tool call passed; no
   warning. The system became a no-op while looking healthy. Now:
     - Default behaviour: re-raise. The user sees what broke.
     - Opt-in `protect(safe_fallback=True)`: keeps the legacy
       degradation, but prints a loud `WARNING: Veto initialization
       failed — falling back to ALLOW-ALL` banner to stderr.

2. **`Veto.evaluateLocalExpression` (TS) now propagates compile and
   evaluate errors** instead of catching them, logging at `warn`,
   and returning `false`. Earlier behaviour: a `block` rule with a
   parse error silently never matched and the call went through.
   Now: error is logged at `error` level and re-raised; the
   validator engine treats validator exceptions as a deny — fail-
   closed by default for security-relevant rules.

3. **`matches` operator on a rejected regex now writes a one-time
   ERROR line to stderr** in both SDKs (Python `condition_evaluator`,
   TS `condition-evaluator`). The operator still returns `false`
   (preserves matches-operator semantics — a non-matching pattern
   isn't a hard error), but a fail-open block rule on a
   misconfigured regex is no longer invisible at runtime. Pairs
   with the load-time scan added in #201; this handles rules added
   or mutated after init.

Tests
-----
* `packages/sdk-python/tests/test_failopen_critical.py` — 6 cases
  (init-raises-by-default, opt-in fallback prints banner, success
  path silent, matches one-time log, safe pattern silent, two
  patterns log once each)
* `packages/sdk/tests/core/failopen-critical.test.ts` — 4 cases
  (matches one-time log, safe pattern silent, two patterns log
  once each, compile() malformed expression throws)
* Full suites: TS 1386/1386, Python 334/334

Behavioural change for downstream
---------------------------------
- `protect()` callers that depended on the allow-all degradation as a
  soft fallback need to either fix their config or pass
  `safe_fallback=True` (and accept every call is now permitted).
- Local rules with broken expressions now produce a deny instead of
  an allow. For most users this is the safer default.
@github-actions github-actions Bot added area:sdk Changes in the TypeScript SDK area:python Changes in the Python SDK area:docs Documentation updates labels Apr 27, 2026
@github-actions
Copy link
Copy Markdown
Contributor


Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA


You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

@github-actions
Copy link
Copy Markdown
Contributor

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:docs Documentation updates area:python Changes in the Python SDK area:sdk Changes in the TypeScript SDK

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant