Skip to content

Conversation

@keshavramji
Copy link

  • Added TestBasedEval component for evaluation with generator and judge sessions.
  • Implements m eval run CLI command for configurable models and backends for the generator and juge.
  • Supports flexible test format with multiple examples per test, each containing input-target pairs evaluated against custom instructions.

@mergify
Copy link

mergify bot commented Nov 6, 2025

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert|release)(?:\(.+\))?:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant