Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
97 changes: 97 additions & 0 deletions proposals/expect-failure-enhancements.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
# Feature proposal: `expectFailure` enhancements

## Summary
Update the `expectFailure` option in `test()` to accept different types of values, enabling both **custom failure messages** and **error validation**. This proposal integrates the requirements from [nodejs/node#61570](https://github.com/nodejs/node/issues/61570), ensuring consistency with `skip`/`todo` while adding robust validation capabilities.

## API & Behavior

The behavior of `expectFailure` is strictly determined by the type of value provided:

### 1. String: Failure Reason
When a **non-empty string** is provided, it acts as a documentation message (reason), identical to `skip` and `todo` options.

```js
test('fails with a specific reason', {
expectFailure: 'Bug #123: Feature not implemented yet'
}, () => {
throw new Error('boom');
});
```
- **Behavior**: The test is expected to fail. The string is treated as a label/reason.
- **Validation**: None. It accepts *any* error.
- **Output**: The reporter displays the string (e.g., `# EXPECTED FAILURE Bug #123...`).

### 2. Matcher: RegExp, Class, or Error Object
When a **RegExp**, **Class** (Function), or **Error Object** is provided directly, it acts as the validation logic. This leverages `assert.throws` behavior directly.

```js
test('fails with matching error (RegExp)', {
expectFailure: /expected error message/
}, () => {
throw new Error('this is the expected error message');
});

test('fails with matching error (Class)', {
expectFailure: RangeError
}, () => {
throw new RangeError('Index out of bounds');
});
```

### 3. Configuration Object: Reason & Validation
When a **Plain Object** with specific properties (`with`, `message`) is provided, it allows specifying both a failure reason and validation logic simultaneously.

```js
test('fails with reason and specific error', {
expectFailure: {
message: 'Bug #123: Edge case behavior', // Reason
with: /Index out of bounds/ // Validation
}
}, () => {
throw new RangeError('Index out of bounds');
});
```
- **Properties**:
- `message` (String): The failure reason/label (displayed in reporter).
- `with` (RegExp | Object | Function | Class): Validation logic. This is passed directly to `assert.throws` validation argument, supporting all its capabilities.
- **Behavior**: The test passes **only if** the error matches the `with` criteria.
- **Output**: The reporter displays the `message`.

### Equivalence
The following configurations are equivalent in behavior (both set a failure reason without validation):
```js
expectFailure: 'reason'
expectFailure: { message: 'reason' }
```
Comment on lines +60 to +65

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For added clarity on the consequences of this design (which i think is perfectly fine), i think you should include the other equivalents: nodejs/node#61563 (comment)


## Ambiguity Resolution
Potential ambiguity between a **Matcher Object** and a **Configuration Object** is resolved as follows:

1. **String** → Reason.
2. **RegExp** or **Function** → Matcher (Validation).
3. **Object**:
* If the object contains `with` or `message` properties → **Configuration Object**.
* Otherwise → **Matcher Object** (passed to `assert.throws` for property matching).

## Alternatives Considered

### Flat Options (`expectFailureError`)
It was proposed to split the options into `expectFailure` (reason) and `expectFailureError` (validation).
This was rejected in favor of the nested/polymorphic structure using `with` and `message` properties. This syntax was selected as the preferred choice for its readability and clarity:
* `with`: Clearly indicates "fails **with** this error" (Validation).
* `message`: Clearly indicates the **reason** or label for the expected failure.
This approach keeps related configuration grouped without polluting the top-level options namespace.

## Implementation Details

### Validation Logic
The implementation leverages `assert.throws` internally to perform error validation.
- If `expectFailure` is a Matcher (RegExp, Class, Object), it is passed as the second argument to `assert.throws(fn, expectFailure)`.
- If `expectFailure` is a Configuration Object, `expectFailure.with` is passed to `assert.throws`.

## Edge Cases

### Empty String (`expectFailure: ''`)
To maintain consistency with `todo` and `skip` options, an empty string is treated as **truthy** (enabled).
* `expectFailure: ''` behaves like `expectFailure: true`.
* The feature is **enabled** (expects failure), but without a specific reason label.
Comment on lines +94 to +97

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the implementation should replicate the todo/skip logic exactly: any value other than undefined or false is treated as true, unless it is decided to change todo/skip as well.