Skip to content

Add an #[explicit_test_case] feature.#144

Open
DRMacIver wants to merge 8 commits intomainfrom
DRMacIver/explicit-test-case
Open

Add an #[explicit_test_case] feature.#144
DRMacIver wants to merge 8 commits intomainfrom
DRMacIver/explicit-test-case

Conversation

@DRMacIver
Copy link
Copy Markdown
Member

Depends on #143. Review that one first.

This mirrors Hypothesis's example decorator. It lets you explicitly
provide a test case as a set of inputs to your test function.

Here's it in action. Consider the following:

const BAD_VALUE: u32 = 124352450;

#[hegel::test]
#[hegel::explicit_test_case(
    x = BAD_VALUE
)]
fn my_test(tc: hegel::TestCase){
    let x = tc.draw(hegel::generators::integers::<u32>());
    assert!(x != BAD_VALUE);
}

If you run this you will see:

let x = BAD_VALUE; // = 124352450

thread 'my_test' (15608116) panicked at examples/explicit_test_case.rs:9:5:
assertion failed: x != BAD_VALUE

It will not work in all cases. Currently in some of those cases it will likely produce very bad error messages. I've tried to do my best to think of the common cases for this and provide good error messages there, ideally at compile time, but there are 100% going to be some missing ones that we can improve over time.

@DRMacIver DRMacIver marked this pull request as ready for review March 27, 2026 12:40
@DRMacIver DRMacIver force-pushed the DRMacIver/explicit-test-case branch 3 times, most recently from e31e709 to 89914c2 Compare April 4, 2026 01:44
DRMacIver and others added 7 commits April 5, 2026 17:34
This introduces draw_named as the primitive for when we are actually
printing, which provides much nicer output for values.

It then makes the hegel::test macro automatically rewrite draw()
to draw_named where it is safe to do so.
- Remove explicit 'a lifetime in is_tc_draw_binding (clippy::needless_lifetimes)
- Move NOTE text outside ```no_run code fence in draw() doc comment
- Allow clippy::let_and_return in closure test (needed for macro rewrite testing)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This mirrors Hypothesis's example decorator. It lets you explicitly
provide a test case as a set of inputs to your test function.

It will not work in all cases. Currently in some of those cases it
will likely produce very bad error messages. I've tried to do my
best to think of the common cases for this, but there are 100% going
to be some missing ones that we can improve over time.
The proc macro always rewrites draw() to draw_named(), so draw() itself
was never exercised in tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
These methods were unreachable through subprocess-based tests since
coverage instrumentation doesn't capture subprocess output.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@DRMacIver DRMacIver force-pushed the DRMacIver/explicit-test-case branch from 89914c2 to fad5927 Compare April 5, 2026 16:34
@Liam-DeVoe
Copy link
Copy Markdown
Member

Liam-DeVoe commented Apr 7, 2026

non-blocking thoughts:

I think @example was the best use of the name "example" in hypothesis and would be weakly in favor of reusing it here as #[hegel::example].

I think explicit_test_case is a fine placeholder name as we figure things out, but a bad permanent name. We should be wary of the scope-creep of test_case in the same way as example in hypothesis, but I would also be weakly in favor of calling this #[hegel::test_case].

@DRMacIver
Copy link
Copy Markdown
Member Author

I agree explicit_test_case isn't a great name, but I'm also not a fan of using test_case or example.

Bit left field. How would you feel about always_try?

@Liam-DeVoe
Copy link
Copy Markdown
Member

I don't mind always_try. What would the noun form of it be, eg when we talk about it in documentation? "we first execute all your always try's, and then...".

@DRMacIver
Copy link
Copy Markdown
Member Author

What would the noun form of it be, eg when we talk about it in documentation?

"explicitly provided test cases" still seems like the best noun to use here for me even if we don't want that in the attribute name.

@DRMacIver
Copy link
Copy Markdown
Member Author

Oh BTW it's not strictly necessary, but it would be nice to get #163 merged first, as I think this has a natural rebuild on top of the backend system. It's implementation only though so the two can logically go in any order really.

@Liam-DeVoe
Copy link
Copy Markdown
Member

An argument against always_try: you don't always try it, if you disable Phase.explicit.

@DRMacIver
Copy link
Copy Markdown
Member Author

An argument against always_try: you don't always try it, if you disable Phase.explicit.

Hmm. That's a pretty compelling argument.

How would you feel about example with a note saying "We're not super happy with this name, which was inherited from Hypothesis, and reserve the right to change it pre-1.0 if we can come up with a better one. The semantics will not change so this should just be a find and replace if we do." or words to that effect.

@Liam-DeVoe
Copy link
Copy Markdown
Member

yeah I'm happy with that. I'm also happy with keeping explicit_test_case with that same note.

@DRMacIver
Copy link
Copy Markdown
Member Author

yeah I'm happy with that. I'm also happy with keeping explicit_test_case with that same note.

Mild preference for keeping explicit_test_case over example then.

if source_normalized == debug_normalized {
eprintln!("let {} = {};", name, source);
} else {
eprintln!("let {} = {}; // = {}", name, source, debug);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a corresponding update to the output so that the format is a syntactically valid explicit test case could also be useful.

e.g. instead of Draw 1: 20828340, output #[hegel::explicit_test_case(x = 20828340).

We'd have to change the actual printer record_named_draw in TestCase (which I cannot comment on since it is unchanged in this PR).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants