Skip to content

[SC-8568] Add input ids to RawData for enhanced test comparison post processing#317

Merged
juanmleng merged 11 commits intomainfrom
juan5508/sc-8568/add-input-ids-to-rawdata-for-enhanced-test-comparison-post-processing
Feb 28, 2025
Merged

[SC-8568] Add input ids to RawData for enhanced test comparison post processing#317
juanmleng merged 11 commits intomainfrom
juan5508/sc-8568/add-input-ids-to-rawdata-for-enhanced-test-comparison-post-processing

Conversation

@juanmleng
Copy link
Contributor

Internal Notes for Reviewers

  • Updated vm-library tests to include model and dataset input IDs in RawData objects for comparison tests. This update enables post-processing functions to track and reference which specific models and datasets generated each test result, improving support for comparison test scenarios.
  • Added a section in how_to/understand_utilize_rawdata.ipynb for illustrating the comparison use case scenario.

External Release Notes

@juanmleng juanmleng added internal Not to be externalized in the release notes chore Chore tasks that aren't bugs or new features labels Feb 18, 2025
@juanmleng juanmleng self-assigned this Feb 18, 2025
Copy link
Contributor

@cachafla cachafla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome 🫡

Copy link
Contributor

@AnilSorathiya AnilSorathiya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

@github-actions
Copy link
Contributor

PR Summary

This pull request introduces significant enhancements to the ValidMind library by improving the utilization of the RawData object across various test functions. The changes include:

  1. Enhancements to RawData Utilization:

    • The RawData object now includes additional metadata such as model.input_id and dataset.input_id where applicable. This allows for better tracking and customization of test results.
    • Functions across multiple files have been updated to return RawData with enriched information, ensuring that all relevant data is captured for post-processing.
  2. Updates to Unit Tests:

    • Unit tests have been updated to accommodate the changes in function signatures, particularly the inclusion of RawData in return values.
    • Tests now include assertions to verify that RawData objects are correctly instantiated and contain expected data.
  3. Documentation and Code Quality:

    • Inline comments and docstrings have been updated to reflect the new functionality and provide clarity on the usage of RawData.
    • Minor formatting adjustments have been made for consistency and readability.

These changes aim to enhance the robustness and flexibility of the ValidMind library, particularly in handling raw data for machine learning model validation and testing.

Test Suggestions

  • Verify that all updated functions return the expected RawData objects with correct metadata.
  • Ensure that unit tests correctly assert the presence and structure of RawData in function outputs.
  • Test the integration of RawData with post-processing functions to confirm that additional metadata is utilized effectively.
  • Run end-to-end tests to validate that the changes do not break existing functionality.
  • Check for any performance impacts due to the additional data being processed and returned.

@juanmleng juanmleng merged commit 90d2c77 into main Feb 28, 2025
6 checks passed
@johnwalz97 johnwalz97 deleted the juan5508/sc-8568/add-input-ids-to-rawdata-for-enhanced-test-comparison-post-processing branch August 20, 2025 17:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

chore Chore tasks that aren't bugs or new features internal Not to be externalized in the release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants