Skip to content

feat: nightly hardening - strict configuration validation#1

Open
mouse-value-add wants to merge 1 commit intobrainsparker:mainfrom
mouse-value-add:chore/nightly-hardening-20260407-config-validation
Open

feat: nightly hardening - strict configuration validation#1
mouse-value-add wants to merge 1 commit intobrainsparker:mainfrom
mouse-value-add:chore/nightly-hardening-20260407-config-validation

Conversation

@mouse-value-add
Copy link
Copy Markdown
Contributor

Problem

PromptLens accepted invalid runtime configuration values (e.g., out-of-range temperatures, non-positive token limits, unsupported output formats), which could fail late at runtime and make evaluations brittle.

Approach

  • Added strict Pydantic validators across config models to fail fast on invalid values.
  • Enforced sane boundaries for temperatures, token/timeouts, and retry settings.
  • Added output format normalization + deduplication (markdown alias -> md) with explicit allowlist checks.
  • Enforced non-empty model configuration at run-level.
  • Added focused tests for validation behavior and normalization.
  • Documented the hardening capability in README features.

Verification

  • Ran: python3 -m pytest -q
  • Result: 5 tests passed.

Risks

  • Some previously accepted but invalid config values now fail early with validation errors.
  • Users relying on permissive/typoed output formats will need to update configs.

Rollback Plan

  • Revert this PR commit to restore previous permissive config behavior.
  • If needed, selectively relax individual validators while keeping test coverage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant