SC-2886: PII Detection for Uploaded Test Results

[johnwalz97]

Pull Request Description

What and why?

Using Microsoft Presidio to add PII detection to the library that is off by default but can be turned on with an environment variable. This will check every test result for PII before its uploaded and throw an error if its detected. The unsafe=True flag can be used to override for a specific test result.

How to test

What needs special review?

Dependencies, breaking changes, and deployment notes

Release notes

Checklist

• What and why
• Screenshots or videos (Frontend)
• How to test
• What needs special review
• Dependencies, breaking changes, and deployment notes
• Labels applied
• PR linked to Shortcut
• Unit tests added (Backend)
• Tested locally
• Documentation updated (if required)
• Environment variable additions/changes documented (if required)

[nibalizer]

Looks good to me. Consider allowing the user to PII scan data before it goes to the LLM. Possible to set the env var to a tuple: ["strict", "results", "annonymize", "off"]

[validbeck]

Just watching your demo during sprint kickoff — I don't see the demo code you're showing off in this PR, perhaps it needs to be added as a mini notebook to notebooks/how_to?

[johnwalz97]

Just watching your demo during sprint kickoff — I don't see the demo code you're showing off in this PR, perhaps it needs to be added as a mini notebook to notebooks/how_to?

Sorry just saw this @validbeck ... i created a notebook and its available in this PR: notebooks/how_to/configure_pii_detection.ipynb

[github-actions]

PR Summary

This PR introduces significant enhancements centered around two main areas:

1. Integration Workflow Updates

   • The GitHub Actions integration workflow has been refactored to remove the use of an older virtual environment (sdist-venv) in favor of a uniform environment (all-venv). This simplifies the workflow by installing the built package, additional dependencies, and creating a Jupyter kernel consistently from the same environment.
   • Logging improvements have been added to the end-to-end notebook runner. The script now prints the current PII detection mode for debugging purposes and accepts a CLI option to configure the PII detection mode dynamically.

2. PII Detection Feature

   • New documentation sections have been added to the README and a dedicated Jupyter Notebook to explain how to configure and test the optional PII detection capabilities using Microsoft Presidio. This feature allows users to detect and block sensitive data in test descriptions and results.
   • In the build configuration (pyproject.toml), dependencies for PII detection (presidio-analyzer and presidio-structured) have been added.
   • A new module (pii_filter.py) has been implemented which provides an enum for PII detection modes, along with utility functions (scan_text and scan_df) that detect and raise errors if PII is found in either a text string or a pandas DataFrame.
   • Functions in the result logging logic now incorporate PII detection by checking tables and description texts, raising errors when sensitive data is detected under specific PII detection modes. This helps prevent logging of sensitive data accidentally.
   • Minor code formatting improvements and test adjustments have also been made to align with these changes.

Overall, this PR not only streamlines the integration workflow but also enhances the software's capability to attempt automatic PII filtering, thereby helping to secure potentially sensitive output data.

Test Suggestions

• Test the integration workflow by running notebooks with different PII detection modes (disabled, test_results, test_descriptions, all) to verify the correct enabling/disabling of PII scanning.
• Create tests that feed both clean and PII-containing data to the scan_text and scan_df functions to ensure they correctly raise exceptions when appropriate.
• Run end-to-end tests that simulate a complete logging operation to verify that the PII checks correctly block the logging of sensitive data.
• Validate that the environment variable configuration and CLI option for PII detection mode propagate correctly throughout the system.
• Test backward compatibility ensuring normal functionality when PII detection is disabled.