From 036ee9c8c2e8926553ad02dd8b815f45eca77bab Mon Sep 17 00:00:00 2001 From: sangwook Date: Thu, 29 Jan 2026 11:23:38 +0900 Subject: [PATCH 1/6] feat: add expectFailure enhancements proposal --- proposals/expect-failure-enhancements.md | 64 ++++++++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100644 proposals/expect-failure-enhancements.md diff --git a/proposals/expect-failure-enhancements.md b/proposals/expect-failure-enhancements.md new file mode 100644 index 0000000..66ba1d9 --- /dev/null +++ b/proposals/expect-failure-enhancements.md @@ -0,0 +1,64 @@ +# Feature proposal: `expectFailure` enhancements + +## Summary +Update the `expectFailure` option in `test()` to accept different types of values, enabling both **custom failure messages** and **error validation**. This aligns with `skip` and `todo` options while adding capabilities similar to `assert.throws`. + +## API & Behavior + +The behavior of `expectFailure` is determined by the type of value provided: + +### 1. String: Failure Reason +When a **non-empty string** is provided, it acts as a documentation message (reason), identical to `skip` and `todo` options. + +```js +test('fails with a specific reason', { + expectFailure: 'Bug #123: Feature not implemented yet' +}, () => { + throw new Error('boom'); +}); +``` +- **Behavior**: The test is expected to fail. The string is treated as a label/reason. +- **Validation**: None. It accepts *any* error. +- **Output**: The reporter displays the string (e.g., `# EXPECTED FAILURE Bug #123...`). +- **Rationale**: Maintains consistency with existing `test` options where a string implies a reason/description. + +### 2. RegExp: Error Matcher +When a **RegExp** is provided, it acts as a validator for the thrown error. + +```js +test('fails with matching error', { + expectFailure: /expected error message/ +}, () => { + throw new Error('this is the expected error message'); +}); +``` +- **Behavior**: The test passes **only if** the thrown error matches the regular expression. +- **Validation**: Strict matching against the error message. +- **Output**: Standard expected failure output. + +## Ambiguity Resolution +Potential ambiguity between "String as Reason" and "String as Matcher" is resolved by strict type separation: +* `typeof value === 'string'` → **Reason** (Documentation only) +* `value instanceof RegExp` → **Matcher** (Validation) + +Users needing to match a specific string error message should use a RegExp (e.g., `/Error message/`) to avoid confusion. + +## Edge Cases & Implementation Details + +### Empty String (`expectFailure: ''`) +Following standard JavaScript truthiness rules, an empty string should be treated as **falsy**. + +* `expectFailure: ''` behaves exactly like `expectFailure: false`. +* The feature is **disabled**, and the test is expected to pass normally. + +### Type Safety for `this.passed` +The implementation must ensure that `this.passed` remains a strict `boolean`. +Assigning a string directly (e.g., `this.passed = this.expectFailure`) is unsafe as it introduces type pollution (assigning `""` or `"reason"` instead of `false`/`true`). + +**Recommended Implementation Logic:** +```javascript +// When an error is caught: +this.passed = !!this.expectFailure; // Forces conversion to boolean +``` +* If `expectFailure` is `"reason"` → `true` (Test Passes) +* If `expectFailure` is `""` → `false` (Test Fails, as expected failure was not active) From 049c2d3e83bbb675bdbf45cf8cc8a56db8e6d562 Mon Sep 17 00:00:00 2001 From: sangwook Date: Thu, 29 Jan 2026 13:40:11 +0900 Subject: [PATCH 2/6] docs: add object support for combined reason and validation (fixes #61570 requirement) --- proposals/expect-failure-enhancements.md | 36 +++++++++++++++++------- 1 file changed, 26 insertions(+), 10 deletions(-) diff --git a/proposals/expect-failure-enhancements.md b/proposals/expect-failure-enhancements.md index 66ba1d9..7993d08 100644 --- a/proposals/expect-failure-enhancements.md +++ b/proposals/expect-failure-enhancements.md @@ -1,11 +1,11 @@ # Feature proposal: `expectFailure` enhancements ## Summary -Update the `expectFailure` option in `test()` to accept different types of values, enabling both **custom failure messages** and **error validation**. This aligns with `skip` and `todo` options while adding capabilities similar to `assert.throws`. +Update the `expectFailure` option in `test()` to accept different types of values, enabling both **custom failure messages** and **error validation**. This proposal integrates the requirements from [nodejs/node#61570](https://github.com/nodejs/node/issues/61570), ensuring consistency with `skip`/`todo` while adding robust validation capabilities. ## API & Behavior -The behavior of `expectFailure` is determined by the type of value provided: +The behavior of `expectFailure` is strictly determined by the type of value provided: ### 1. String: Failure Reason When a **non-empty string** is provided, it acts as a documentation message (reason), identical to `skip` and `todo` options. @@ -20,7 +20,6 @@ test('fails with a specific reason', { - **Behavior**: The test is expected to fail. The string is treated as a label/reason. - **Validation**: None. It accepts *any* error. - **Output**: The reporter displays the string (e.g., `# EXPECTED FAILURE Bug #123...`). -- **Rationale**: Maintains consistency with existing `test` options where a string implies a reason/description. ### 2. RegExp: Error Matcher When a **RegExp** is provided, it acts as a validator for the thrown error. @@ -36,24 +35,41 @@ test('fails with matching error', { - **Validation**: Strict matching against the error message. - **Output**: Standard expected failure output. -## Ambiguity Resolution -Potential ambiguity between "String as Reason" and "String as Matcher" is resolved by strict type separation: -* `typeof value === 'string'` → **Reason** (Documentation only) -* `value instanceof RegExp` → **Matcher** (Validation) +### 3. Object: Reason & Validation +When an **Object** is provided, it allows specifying both a failure reason and validation logic simultaneously. + +```js +test('fails with reason and specific error', { + expectFailure: { + message: 'Bug #123: Edge case behavior', // Reason + match: /Index out of bounds/ // Validation + } +}, () => { + throw new RangeError('Index out of bounds'); +}); +``` +- **Properties**: + - `message` (String): The failure reason/label (displayed in reporter). + - `match` (RegExp | Object | Function): Validation logic (similar to `assert.throws` validation argument). +- **Behavior**: The test passes **only if** the error matches the `match` criteria. +- **Output**: The reporter displays the `message`. -Users needing to match a specific string error message should use a RegExp (e.g., `/Error message/`) to avoid confusion. +## Ambiguity Resolution +Potential ambiguity is resolved by strict type separation: +* `typeof value === 'string'` → **Reason** +* `value instanceof RegExp` → **Matcher** +* `typeof value === 'object'` → **Both** (Explicit properties) ## Edge Cases & Implementation Details ### Empty String (`expectFailure: ''`) Following standard JavaScript truthiness rules, an empty string should be treated as **falsy**. - * `expectFailure: ''` behaves exactly like `expectFailure: false`. * The feature is **disabled**, and the test is expected to pass normally. ### Type Safety for `this.passed` The implementation must ensure that `this.passed` remains a strict `boolean`. -Assigning a string directly (e.g., `this.passed = this.expectFailure`) is unsafe as it introduces type pollution (assigning `""` or `"reason"` instead of `false`/`true`). +Assigning a string directly (e.g., `this.passed = this.expectFailure`) is unsafe as it introduces type pollution. **Recommended Implementation Logic:** ```javascript From abb5912aa29d0de14e495ca0d79de1a277c661a2 Mon Sep 17 00:00:00 2001 From: sangwook Date: Fri, 30 Jan 2026 00:01:18 +0900 Subject: [PATCH 3/6] docs: update expectFailure proposal with validation details Update the `expectFailure` enhancements proposal based on feedback and implementation alignment: - Consolidate validation logic under the `with` property within an object. - Remove direct RegExp support in favor of the object syntax for consistency. - Specify usage of `assert.throws` for robust error validation. - Document alternatives considered (flat options). --- proposals/expect-failure-enhancements.md | 38 +++++++++++++++++------- 1 file changed, 27 insertions(+), 11 deletions(-) diff --git a/proposals/expect-failure-enhancements.md b/proposals/expect-failure-enhancements.md index 7993d08..122522b 100644 --- a/proposals/expect-failure-enhancements.md +++ b/proposals/expect-failure-enhancements.md @@ -21,19 +21,16 @@ test('fails with a specific reason', { - **Validation**: None. It accepts *any* error. - **Output**: The reporter displays the string (e.g., `# EXPECTED FAILURE Bug #123...`). -### 2. RegExp: Error Matcher -When a **RegExp** is provided, it acts as a validator for the thrown error. +### 2. RegExp: Error Matcher (via Object) +Use the object form with the `with` property. ```js test('fails with matching error', { - expectFailure: /expected error message/ + expectFailure: { with: /expected error message/ } }, () => { throw new Error('this is the expected error message'); }); ``` -- **Behavior**: The test passes **only if** the thrown error matches the regular expression. -- **Validation**: Strict matching against the error message. -- **Output**: Standard expected failure output. ### 3. Object: Reason & Validation When an **Object** is provided, it allows specifying both a failure reason and validation logic simultaneously. @@ -42,7 +39,7 @@ When an **Object** is provided, it allows specifying both a failure reason and v test('fails with reason and specific error', { expectFailure: { message: 'Bug #123: Edge case behavior', // Reason - match: /Index out of bounds/ // Validation + with: /Index out of bounds/ // Validation } }, () => { throw new RangeError('Index out of bounds'); @@ -50,15 +47,34 @@ test('fails with reason and specific error', { ``` - **Properties**: - `message` (String): The failure reason/label (displayed in reporter). - - `match` (RegExp | Object | Function): Validation logic (similar to `assert.throws` validation argument). -- **Behavior**: The test passes **only if** the error matches the `match` criteria. + - `with` (RegExp | Object | Function | Class): Validation logic. This is passed directly to `assert.throws` validation argument, supporting all its capabilities. +- **Behavior**: The test passes **only if** the error matches the `with` criteria. - **Output**: The reporter displays the `message`. ## Ambiguity Resolution Potential ambiguity is resolved by strict type separation: * `typeof value === 'string'` → **Reason** -* `value instanceof RegExp` → **Matcher** -* `typeof value === 'object'` → **Both** (Explicit properties) +* `typeof value === 'object'` → **Configuration Object** (`message` and/or `with`) + +## Alternatives Considered + +### Flat Options (`expectFailureError`) +It was proposed to split the options into `expectFailure` (reason) and `expectFailureError` (validation). +```js +{ + expectFailure: 'reason', + expectFailureError: /error/ +} +``` +This was rejected in favor of the nested object structure to: +1. Keep related configuration grouped. +2. Avoid polluting the top-level options namespace. +3. Allow future extensibility within the `expectFailure` object. + +## Implementation Details + +### Validation Logic +The implementation leverages `assert.throws` internally to perform error validation. This ensures consistency with the existing assertion ecosystem and supports advanced validation (Classes, Custom Functions) out of the box without code duplication. ## Edge Cases & Implementation Details From 87802e3e188aed71c5d5c86767906c2aafeb0533 Mon Sep 17 00:00:00 2001 From: sangwook Date: Fri, 30 Jan 2026 11:21:28 +0900 Subject: [PATCH 4/6] docs: update expectFailure proposal with direct matchers and rationale --- proposals/expect-failure-enhancements.md | 69 ++++++++++++------------ 1 file changed, 35 insertions(+), 34 deletions(-) diff --git a/proposals/expect-failure-enhancements.md b/proposals/expect-failure-enhancements.md index 122522b..c15c681 100644 --- a/proposals/expect-failure-enhancements.md +++ b/proposals/expect-failure-enhancements.md @@ -21,19 +21,25 @@ test('fails with a specific reason', { - **Validation**: None. It accepts *any* error. - **Output**: The reporter displays the string (e.g., `# EXPECTED FAILURE Bug #123...`). -### 2. RegExp: Error Matcher (via Object) -Use the object form with the `with` property. +### 2. Matcher: RegExp, Class, or Error Object +When a **RegExp**, **Class** (Function), or **Error Object** is provided directly, it acts as the validation logic. This leverages `assert.throws` behavior directly. ```js -test('fails with matching error', { - expectFailure: { with: /expected error message/ } +test('fails with matching error (RegExp)', { + expectFailure: /expected error message/ }, () => { throw new Error('this is the expected error message'); }); + +test('fails with matching error (Class)', { + expectFailure: RangeError +}, () => { + throw new RangeError('Index out of bounds'); +}); ``` -### 3. Object: Reason & Validation -When an **Object** is provided, it allows specifying both a failure reason and validation logic simultaneously. +### 3. Configuration Object: Reason & Validation +When a **Plain Object** with specific properties (`with`, `message`) is provided, it allows specifying both a failure reason and validation logic simultaneously. ```js test('fails with reason and specific error', { @@ -51,46 +57,41 @@ test('fails with reason and specific error', { - **Behavior**: The test passes **only if** the error matches the `with` criteria. - **Output**: The reporter displays the `message`. +### Equivalence +The following configurations are equivalent in behavior (both set a failure reason without validation): +```js +expectFailure: 'reason' +expectFailure: { message: 'reason' } +``` + ## Ambiguity Resolution -Potential ambiguity is resolved by strict type separation: -* `typeof value === 'string'` → **Reason** -* `typeof value === 'object'` → **Configuration Object** (`message` and/or `with`) +Potential ambiguity between a **Matcher Object** and a **Configuration Object** is resolved as follows: + +1. **String** → Reason. +2. **RegExp** or **Function** → Matcher (Validation). +3. **Object**: + * If the object contains `with` or `message` properties → **Configuration Object**. + * Otherwise → **Matcher Object** (passed to `assert.throws` for property matching). ## Alternatives Considered ### Flat Options (`expectFailureError`) It was proposed to split the options into `expectFailure` (reason) and `expectFailureError` (validation). -```js -{ - expectFailure: 'reason', - expectFailureError: /error/ -} -``` -This was rejected in favor of the nested object structure to: -1. Keep related configuration grouped. -2. Avoid polluting the top-level options namespace. -3. Allow future extensibility within the `expectFailure` object. +This was rejected in favor of the nested/polymorphic structure using `with` and `message` properties. This syntax was selected as the preferred choice for its readability and clarity: +* `with`: Clearly indicates "fails **with** this error" (Validation). +* `message`: Clearly indicates the **reason** or label for the expected failure. +This approach keeps related configuration grouped without polluting the top-level options namespace. ## Implementation Details ### Validation Logic -The implementation leverages `assert.throws` internally to perform error validation. This ensures consistency with the existing assertion ecosystem and supports advanced validation (Classes, Custom Functions) out of the box without code duplication. +The implementation leverages `assert.throws` internally to perform error validation. +- If `expectFailure` is a Matcher (RegExp, Class, Object), it is passed as the second argument to `assert.throws(fn, expectFailure)`. +- If `expectFailure` is a Configuration Object, `expectFailure.with` is passed to `assert.throws`. -## Edge Cases & Implementation Details +## Edge Cases ### Empty String (`expectFailure: ''`) Following standard JavaScript truthiness rules, an empty string should be treated as **falsy**. * `expectFailure: ''` behaves exactly like `expectFailure: false`. -* The feature is **disabled**, and the test is expected to pass normally. - -### Type Safety for `this.passed` -The implementation must ensure that `this.passed` remains a strict `boolean`. -Assigning a string directly (e.g., `this.passed = this.expectFailure`) is unsafe as it introduces type pollution. - -**Recommended Implementation Logic:** -```javascript -// When an error is caught: -this.passed = !!this.expectFailure; // Forces conversion to boolean -``` -* If `expectFailure` is `"reason"` → `true` (Test Passes) -* If `expectFailure` is `""` → `false` (Test Fails, as expected failure was not active) +* The feature is **disabled**, and the test is expected to pass normally. \ No newline at end of file From cb12492d9366fb91ca061ee512fa8b9cde477754 Mon Sep 17 00:00:00 2001 From: sangwook Date: Fri, 30 Jan 2026 11:35:22 +0900 Subject: [PATCH 5/6] docs: update expectFailure empty string behavior for consistency --- proposals/expect-failure-enhancements.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/expect-failure-enhancements.md b/proposals/expect-failure-enhancements.md index c15c681..2667106 100644 --- a/proposals/expect-failure-enhancements.md +++ b/proposals/expect-failure-enhancements.md @@ -92,6 +92,6 @@ The implementation leverages `assert.throws` internally to perform error validat ## Edge Cases ### Empty String (`expectFailure: ''`) -Following standard JavaScript truthiness rules, an empty string should be treated as **falsy**. -* `expectFailure: ''` behaves exactly like `expectFailure: false`. -* The feature is **disabled**, and the test is expected to pass normally. \ No newline at end of file +To maintain consistency with `todo` and `skip` options, an empty string is treated as **truthy** (enabled). +* `expectFailure: ''` behaves like `expectFailure: true`. +* The feature is **enabled** (expects failure), but without a specific reason label. \ No newline at end of file From 550464a42aedc018c7473925ae11621654252fa2 Mon Sep 17 00:00:00 2001 From: sangwook Date: Fri, 30 Jan 2026 12:24:47 +0900 Subject: [PATCH 6/6] docs: expand equivalence examples and clarify activation logic --- proposals/expect-failure-enhancements.md | 32 ++++++++++++++++-------- 1 file changed, 22 insertions(+), 10 deletions(-) diff --git a/proposals/expect-failure-enhancements.md b/proposals/expect-failure-enhancements.md index 2667106..88e5e06 100644 --- a/proposals/expect-failure-enhancements.md +++ b/proposals/expect-failure-enhancements.md @@ -58,12 +58,26 @@ test('fails with reason and specific error', { - **Output**: The reporter displays the `message`. ### Equivalence -The following configurations are equivalent in behavior (both set a failure reason without validation): +The following configurations are equivalent in behavior: + +**1. Reason only:** ```js expectFailure: 'reason' expectFailure: { message: 'reason' } ``` +**2. Validation only:** +```js +expectFailure: /error/ +expectFailure: { with: /error/ } +``` + +**3. Catch-all (Any Error):** +```js +expectFailure: true +expectFailure: {} +``` + ## Ambiguity Resolution Potential ambiguity between a **Matcher Object** and a **Configuration Object** is resolved as follows: @@ -73,7 +87,12 @@ Potential ambiguity between a **Matcher Object** and a **Configuration Object** * If the object contains `with` or `message` properties → **Configuration Object**. * Otherwise → **Matcher Object** (passed to `assert.throws` for property matching). -## Alternatives Considered +## Activation & Truthiness +To maintain strict consistency with `todo` and `skip` options: +* The feature is **disabled** only if `expectFailure` is `undefined` or `false`. +* **All other values** enable the feature (treat as truthy). + * `expectFailure: ''` (Empty String) → **Enabled** (treats as generic failure expectation). + * `expectFailure: 0` → **Enabled** (treated as a Matcher Object unless specific logic excludes numbers, but per consistency it enables the feature). ### Flat Options (`expectFailureError`) It was proposed to split the options into `expectFailure` (reason) and `expectFailureError` (validation). @@ -87,11 +106,4 @@ This approach keeps related configuration grouped without polluting the top-leve ### Validation Logic The implementation leverages `assert.throws` internally to perform error validation. - If `expectFailure` is a Matcher (RegExp, Class, Object), it is passed as the second argument to `assert.throws(fn, expectFailure)`. -- If `expectFailure` is a Configuration Object, `expectFailure.with` is passed to `assert.throws`. - -## Edge Cases - -### Empty String (`expectFailure: ''`) -To maintain consistency with `todo` and `skip` options, an empty string is treated as **truthy** (enabled). -* `expectFailure: ''` behaves like `expectFailure: true`. -* The feature is **enabled** (expects failure), but without a specific reason label. \ No newline at end of file +- If `expectFailure` is a Configuration Object, `expectFailure.with` is passed to `assert.throws`. \ No newline at end of file