[SC-7759] Exposing static descriptions in test results

[juanmleng]

Internal Notes for Reviewers

Added a doc property to TestResult class to clearly separate the static test documentation from the LLM-generated descriptions. This change helps distinguish between the original test documentation (accessed via result.doc) and the dynamic, descriptions generated by the LLM (accessed via result.description). The doc property contains the test's docstring, providing users with direct access to the test's purpose and behavior.

External Release Notes

Added a doc property to TestResult class to clearly separate the static test documentation from the LLM-generated descriptions. This change helps distinguish between the original test documentation (accessed via result.doc) and the dynamic, descriptions generated by the LLM (accessed via result.description). The doc property contains the test's docstring, providing users with direct access to the test's purpose and behavior.

[johnwalz97]

lgtm 👏

[github-actions]

PR Summary

This pull request introduces several enhancements and bug fixes to the ValidMind project:

1. Test Result Documentation:

   • Added a doc attribute to the TestResult class to store documentation related to test results.
   • Updated various functions (build_test_result, _run_composite_test, _run_comparison_test, run_test) to include the doc parameter, ensuring that test documentation is properly captured and passed through the test result pipeline.

2. ROC Curve Test Enhancements:

   • Modified the test_roc_curve_structure and test_perfect_separation tests in test_ROCCurve.py to handle the updated return type of the ROCCurve function, which now returns a tuple containing RawData and Figure.
   • Added assertions to verify the structure and contents of the RawData object, ensuring it contains expected fields like fpr, tpr, and auc.

3. Test Execution Improvement:

   • Updated tests/test_unit_tests.py to exit with a failure status if any tests fail, improving the robustness of the test execution process.

These changes aim to enhance the documentation of test results, improve the structure and validation of ROC curve tests, and ensure more reliable test execution.

Test Suggestions

• Verify that the doc attribute is correctly populated and accessible in TestResult instances.
• Ensure that the ROCCurve function returns a tuple with RawData and Figure and that the tests correctly validate this structure.
• Test the exit behavior of the test suite to confirm it exits with a failure status when tests fail.
• Check that the RawData object contains the expected fields (fpr, tpr, auc) and that these fields are correctly validated in the tests.