Skip to content

Conversation

@juanmleng
Copy link
Contributor

Internal Notes for Reviewers

Fixed static descriptions handling for composite tests by combining docstrings from individual unit metrics. Previously, this was throwing an error in the notebook run_unit_metrics.ipynb as it was assuming a non-existing test_id for composite unit metrics tests. Now the docstring is built from a combination of the docstrings of the individual unit metrics.

External Release Notes

@juanmleng juanmleng added bug Something isn't working internal Not to be externalized in the release notes labels Dec 16, 2024
@juanmleng juanmleng self-assigned this Dec 16, 2024
Copy link
Contributor

@johnwalz97 johnwalz97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks

@github-actions
Copy link
Contributor

PR Summary

This pull request introduces an enhancement to the validmind/tests/run.py file, specifically in the _run_comparison_test function. The change improves the generation of test documentation by checking if unit_metrics are present. If they are, it constructs a detailed documentation string by iterating over each unit_metric, retrieving its name and documentation, and joining them with newlines. If unit_metrics are not present, it defaults to the previous behavior of loading the documentation for the test_id. This enhancement allows for more comprehensive documentation when multiple unit metrics are involved in a test.

The version number in pyproject.toml and validmind/__version__.py has been incremented from 2.7.1 to 2.7.2 to reflect this change.

Test Suggestions

  • Test the _run_comparison_test function with a non-empty unit_metrics list to ensure the documentation is generated correctly for each metric.
  • Test the _run_comparison_test function with an empty unit_metrics list to verify that it falls back to the original behavior.
  • Verify that the output format of the documentation string is as expected when multiple unit metrics are provided.
  • Check for any performance impacts when processing a large number of unit metrics.

@juanmleng juanmleng merged commit 71f812d into main Dec 16, 2024
5 checks passed
@johnwalz97 johnwalz97 deleted the juan5508/sc-7759/exposing-static-descriptions-in-test-results branch January 3, 2025 15:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working internal Not to be externalized in the release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants