Skip to content

[SC-7456] intro_for_model_developers.ipynb throws SHAPGlobalImportance.py error#341

Merged
juanmleng merged 2 commits intomainfrom
juan/sc-7456/fix-shap-test-for-logistic-regression
Apr 1, 2025
Merged

[SC-7456] intro_for_model_developers.ipynb throws SHAPGlobalImportance.py error#341
juanmleng merged 2 commits intomainfrom
juan/sc-7456/fix-shap-test-for-logistic-regression

Conversation

@juanmleng
Copy link
Contributor

Internal Notes for Reviewers

When running the 102-start_development_process.ipynb notebook the SHAPGlobalImportance test fails with:

ERROR(validmind.vm_models.test_suite.test): Failed to run test 'validmind.model_validation.sklearn.SHAPGlobalImportance': (TypeError) loop of ufunc does not support argument 0 of type float which has no callable rint method

There seems to be an issue with the Logistic Regression setup in the notebook causing some SHAP values to have object data type instead of floats, which creates problems with the SHAP plots. The fix here is to convert these values to proper float arrays.

External Release Notes

@juanmleng juanmleng added bug Something isn't working internal Not to be externalized in the release notes labels Mar 31, 2025
@juanmleng juanmleng self-assigned this Mar 31, 2025
@juanmleng juanmleng requested review from cachafla and validbeck March 31, 2025 22:01
@github-actions
Copy link
Contributor

github-actions bot commented Apr 1, 2025

PR Summary

This pull request enhances the logic for selecting SHAP values in the select_shap_values function within the validmind/tests/model_validation/sklearn/SHAPGlobalImportance.py file. The changes include:

  • Improved handling of SHAP values for both regression and classification tasks.
  • For regression tasks, the SHAP values are returned as-is if they are not a list.
  • For binary classification, the function defaults to using SHAP values for class 1 if no specific class is specified.
  • For multi-class classification, the function now checks if the class_of_interest is valid and raises a ValueError if it is not.
  • Added type conversion to ensure the selected SHAP values are returned as a float array.

These changes aim to improve the robustness and clarity of the SHAP value selection process.

Test Suggestions

  • Test with regression SHAP values to ensure they are returned as-is.
  • Test binary classification SHAP values without specifying class_of_interest to ensure class 1 values are returned.
  • Test multi-class classification with valid class_of_interest to ensure correct SHAP values are selected.
  • Test multi-class classification with invalid class_of_interest to ensure a ValueError is raised.
  • Test the type conversion to ensure the returned SHAP values are of type float array.

@juanmleng juanmleng merged commit 27aa125 into main Apr 1, 2025
6 checks passed
@johnwalz97 johnwalz97 deleted the juan/sc-7456/fix-shap-test-for-logistic-regression branch August 20, 2025 17:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working internal Not to be externalized in the release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants