Skip to content

FIX Reject empty WMDP category values#1497

Merged
romanlutz merged 2 commits intomicrosoft:mainfrom
biefan:reject-empty-wmdp-category
Mar 19, 2026
Merged

FIX Reject empty WMDP category values#1497
romanlutz merged 2 commits intomicrosoft:mainfrom
biefan:reject-empty-wmdp-category

Conversation

@biefan
Copy link
Copy Markdown
Contributor

@biefan biefan commented Mar 17, 2026

Summary

  • reject empty-string category values instead of treating them as "load all categories"
  • add regression coverage to ensure invalid empty values fail fast without calling load_dataset

Problem

fetch_wmdp_dataset() documents None as the way to load all WMDP categories, and invalid category values are supposed to raise ValueError. The implementation currently uses if not category, which also treats "" as "load all categories".

As a result, an explicit but invalid empty-string category is silently ignored and the function loads all three subsets instead of failing fast.

Testing

  • .venv/bin/pytest tests/unit/datasets/test_wmdp_dataset.py -q

@romanlutz romanlutz changed the title Reject empty WMDP category values FIX Reject empty WMDP category values Mar 19, 2026
@romanlutz romanlutz merged commit 80c1e6a into microsoft:main Mar 19, 2026
37 checks passed
riyosha pushed a commit to riyosha/PyRIT that referenced this pull request Mar 24, 2026
Co-authored-by: Roman Lutz <romanlutz13@gmail.com>
jbolor21 pushed a commit to jbolor21/jbolor-PyRIT that referenced this pull request Mar 25, 2026
Co-authored-by: Roman Lutz <romanlutz13@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants