feat(testing): Add Parameterized Test Support with Test Data Generator#1606
Open
Mustafa11300 wants to merge 2 commits intomofa-org:mainfrom
Open
feat(testing): Add Parameterized Test Support with Test Data Generator#1606Mustafa11300 wants to merge 2 commits intomofa-org:mainfrom
Mustafa11300 wants to merge 2 commits intomofa-org:mainfrom
Conversation
Implements parameterized scenario expansion for the mofa-testing crate.
One scenario template can now expand into many concrete test cases by
substituting {{variable}} placeholders with values from parameter sets.
New components:
- ParameterSet: named variable bindings for one test variant
- ParameterMatrix: Cartesian product expansion with safety limits
- ParameterizedScenario: template + parameter sets -> expanded scenarios
- ParameterizedScenarioFile: YAML/TOML/JSON file-backed loading
Includes:
- 30+ comprehensive tests covering expansion, substitution, file
loading, execution, edge cases, and error handling
- Example scenarios in examples/parameterized_test/ with YAML, TOML,
and matrix expansion demonstrations
- README with usage guide and code samples
Closes #<ISSUE_NUMBER>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🔍 Description
Adds parameterized scenario expansion to the
mofa-testingcrate. A single scenario template can now expand into many concrete test cases by substituting{{variable}}placeholders with values from parameter sets.This is Issue 3 from the testing platform roadmap (Test Definition Layer, Phase 2).
Closes #1603
Depends on #1599
📌 Changes
New:
tests/src/parameterized.rsCore parameterized test module with the following public components:
ParameterSetParameterMatrixParameterizedScenarioAgentTestScenariotemplate and expands it against parameter setsParameterizedScenarioFileParameterExpansionErrorPlaceholder substitution (
{{variable}}) is supported in:user_input(turn prompts)system_prompttextexpectations (respond_containing, respond_exact)patternexpectations (respond_matching_regex)nameexpectations (call_tool, call_tool_with)argumentsvalues (recursively in JSON)Expansion modes:
Modified:
tests/src/lib.rspub mod parameterizedmodule registrationParameterExpansionError,ParameterMatrix,ParameterSet,ParameterizedScenario,ParameterizedScenarioFileNew:
tests/tests/parameterized_tests.rs30+ comprehensive tests covering:
New:
examples/parameterized_test/README.mdscenario_weather_parameterized.yamlscenario_greetings_matrix.yamlscenario_support_parameterized.toml🧪 Testing
All new functionality is covered by
tests/tests/parameterized_tests.rswith 30+ test cases.Key test categories:
💡 Usage Example
Builder API (Rust)
File-backed (YAML)
✅ Checklist
lib.rswith public re-exportsparameterized_tests.rs)examples/parameterized_test/)