Skip to content

notebook: Enable PII detection in tests#416

Merged
validbeck merged 38 commits intomainfrom
beck/sc-11822/documentation-pii-detection
Aug 22, 2025
Merged

notebook: Enable PII detection in tests#416
validbeck merged 38 commits intomainfrom
beck/sc-11822/documentation-pii-detection

Conversation

@validbeck
Copy link
Collaborator

@validbeck validbeck commented Aug 21, 2025

Pull Request Description

What and why?

sc-11822

  • I took the sample notebook John introduced in SC-2886: PII Detection for Uploaded Test Results #410 and cleaned it up with some more distinct/expanded examples.
  • It has been renamed to follow our style guide conventions, etc. (configure_pii_detection.ipynb > enable_pii_detection.ipynb)

How to test

I checked that everything ran fine locally, but you might want to pull the file down just to check: https://github.com/validmind/validmind-library/blob/beck/sc-11822/documentation-pii-detection/notebooks/how_to/enable_pii_detection.ipynb

What needs special review?

I took some liberties with the examples so please make sure they make sense/are accurate.

Dependencies, breaking changes, and deployment notes

n/a

Release notes

Learn how to enable and configure Personally Identifiable Information (PII) detection when running tests with the ValidMind Library with our new notebook. Choose whether or not to include PII in test descriptions generated, or whether or not to include PII in test results logged to the ValidMind Platform.

Enable PII detection in tests

Checklist

  • What and why
  • Screenshots or videos (Frontend)
  • How to test
  • What needs special review
  • Dependencies, breaking changes, and deployment notes
  • Labels applied
  • PR linked to Shortcut
  • Unit tests added (Backend)
  • Tested locally
  • Documentation updated (if required)
  • Environment variable additions/changes documented (if required)

@validbeck validbeck self-assigned this Aug 21, 2025
@validbeck validbeck added documentation Improvements or additions to documentation highlight Feature to be curated in the release notes labels Aug 21, 2025
@validbeck validbeck requested a review from johnwalz97 August 21, 2025 21:34
@github-actions
Copy link
Contributor

PR Summary

This PR replaces the deprecated 'configure_pii_detection.ipynb' notebook with a new, comprehensive 'enable_pii_detection.ipynb' notebook. The new notebook provides detailed instructions on how to enable and configure PII detection within the ValidMind Library. It steps through the process of installing the library with the pii-detection extra, initializing the library, and creating custom tests that emit PII. Different PII detection modes (disabled, test_results, test_descriptions, and all) are demonstrated with code examples. In addition, the notebook illustrates how to override detection using an unsafe flag, while also explaining the potential risks. Overall, the changes aim to streamline the user experience for configuring PII detection and provide clearer guidance on managing sensitive data in testing scenarios.

Test Suggestions

  • Run the notebook under each PII detection mode (disabled, test_results, test_descriptions, and all) and verify that the output behavior matches the expected logging and description generation.
  • Test the override functionality with the unsafe flag to ensure that the blocking logic is bypassed as documented.
  • Confirm that the custom test output containing sample PII data properly triggers PII detection and handles exceptions as expected.
  • Verify that no sensitive PII is actually transmitted when not using the unsafe override.

Copy link
Contributor

@johnwalz97 johnwalz97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

beautiful stuff!

@validbeck validbeck merged commit a0bfd69 into main Aug 22, 2025
15 checks passed
@validbeck validbeck deleted the beck/sc-11822/documentation-pii-detection branch August 22, 2025 16:52
@nrichers nrichers added enhancement New feature or request and removed highlight Feature to be curated in the release notes labels Sep 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants