Standardize maxVariation logic and rename difference calculation#400
Standardize maxVariation logic and rename difference calculation#400
Conversation
|
Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. WalkthroughConverts difference calculations from percentage (0–100) to fractional (0–1), renames Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro 📒 Files selected for processing (1)
🧰 Additional context used🧠 Learnings (17)📓 Common learnings📚 Learning: 2025-11-26T19:31:27.066ZApplied to files:
📚 Learning: 2025-11-26T19:31:27.066ZApplied to files:
📚 Learning: 2025-11-26T19:31:27.066ZApplied to files:
📚 Learning: 2025-11-26T19:31:27.066ZApplied to files:
📚 Learning: 2025-11-26T19:31:27.066ZApplied to files:
📚 Learning: 2025-11-26T19:31:27.066ZApplied to files:
📚 Learning: 2025-11-26T19:31:27.066ZApplied to files:
📚 Learning: 2025-11-26T19:31:27.066ZApplied to files:
📚 Learning: 2025-11-26T19:31:27.066ZApplied to files:
📚 Learning: 2025-11-26T19:31:27.066ZApplied to files:
📚 Learning: 2025-11-26T19:31:27.066ZApplied to files:
📚 Learning: 2025-11-26T19:31:27.066ZApplied to files:
📚 Learning: 2025-11-26T19:31:27.066ZApplied to files:
📚 Learning: 2025-11-26T19:31:27.066ZApplied to files:
📚 Learning: 2025-11-26T19:31:27.066ZApplied to files:
📚 Learning: 2025-11-26T19:31:27.066ZApplied to files:
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
🔇 Additional comments (3)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
src/tests/saveScreenshot.js (1)
289-289: Fractional image diff logic is consistent; consider clarifying messagesThe switch to
fractionalDiff = numDiffPixels / (width * height)and comparing directly againstmaxVariationaligns with the new fractional semantics and the defaults (0.05→ 5% tolerance). Logging withfractionalDiff.toFixed(2)is also consistent with the other step types.One minor clarity point: the warning message still refers to “command output content”, which is confusing in a screenshot step. Consider rewording to something like:
- result.description += ` The difference between the existing file content and command output content (${fractionalDiff.toFixed( + result.description += ` The difference between the existing screenshot and new screenshot (${fractionalDiff.toFixed( 2 )}) is greater than the max accepted variation (${ step.screenshot.maxVariation }).`;Also applies to: 340-345, 347-357, 360-362
src/tests/httpRequest.js (1)
7-7: HTTP fixture diffing correctly migrated to fractional differencesImporting
calculateFractionalDifferenceand usingfractionalDiff > step.httpRequest.maxVariationfor saved-response comparisons brings this step in line with the new fractionalmaxVariationsemantics. With the defaultmaxVariation: 0, any change in the serialized JSON will still cause a failure, which matches the previous strict behavior. The small doc/formatting tweaks infieldExistsAtPathare also fine.If you want the failure message to be less shell-specific, you could optionally adjust the wording:
- result.description += ` The difference between the existing file content and command output content (${fractionalDiff.toFixed( + result.description += ` The difference between the existing saved response and current response (${fractionalDiff.toFixed( 2 )}) is greater than the max accepted variation (${ step.httpRequest.maxVariation }).`;Also applies to: 456-462, 463-477, 491-535
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
AGENTS.md(1 hunks)src/tests/httpRequest.js(7 hunks)src/tests/runShell.js(2 hunks)src/tests/saveScreenshot.js(2 hunks)src/utils.js(2 hunks)
🧰 Additional context used
📓 Path-based instructions (4)
src/tests/*.js
📄 CodeRabbit inference engine (AGENTS.md)
Create new step type handlers in
src/tests/[actionName].jsexporting async functions that validate schema, resolve to object, set defaults, and return{ status: 'PASS'|'FAIL'|'WARNING', description: string, outputs: {} }
Files:
src/tests/runShell.jssrc/tests/saveScreenshot.jssrc/tests/httpRequest.js
src/**/*.js
📄 CodeRabbit inference engine (AGENTS.md)
src/**/*.js: Uselog(config, level, message)for logging where level is 'debug'|'info'|'warning'|'error', passing config object as first parameter
UsegetAvailableApps()to detect installed browsers rather than hardcoding browser paths, accounting for platform-specific locations
Files:
src/tests/runShell.jssrc/tests/saveScreenshot.jssrc/utils.jssrc/tests/httpRequest.js
src/tests/**/*.js
📄 CodeRabbit inference engine (AGENTS.md)
src/tests/**/*.js: Always handle driver cleanup in try/finally blocks to ensure WebDriver sessions are properly terminated
Set step variables viastep.variables = { MY_VAR: '$$response.body.token' }to store runtime values as environment variables for subsequent steps
UsesetViewportSize()to set inner viewport dimensions, which calculates delta from window size rather than setting window size directly
TreatmaxVariationpercentage values as decimals (0.1 = 10%) but multiply by 100 when performing percentage-based comparisons
File overwrite modes support three values: 'false' (never overwrite), 'true' (always overwrite), 'aboveVariation' (overwrite only if content differs > maxVariation)
Files:
src/tests/runShell.jssrc/tests/saveScreenshot.jssrc/tests/httpRequest.js
src/tests/httpRequest.js
📄 CodeRabbit inference engine (AGENTS.md)
Reference OpenAPI operations by
operationId(e.g.,step.httpRequest.openApi.operationId = 'getUserById') rather than path/method combinations
Files:
src/tests/httpRequest.js
🧠 Learnings (7)
📓 Common learnings
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Applies to src/tests/**/*.js : Treat `maxVariation` percentage values as decimals (0.1 = 10%) but multiply by 100 when performing percentage-based comparisons
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Applies to src/tests/**/*.js : Treat `maxVariation` percentage values as decimals (0.1 = 10%) but multiply by 100 when performing percentage-based comparisons
Applied to files:
src/tests/runShell.jssrc/tests/saveScreenshot.jsAGENTS.mdsrc/utils.jssrc/tests/httpRequest.js
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Applies to src/tests/**/*.js : File overwrite modes support three values: 'false' (never overwrite), 'true' (always overwrite), 'aboveVariation' (overwrite only if content differs > maxVariation)
Applied to files:
src/tests/runShell.jssrc/tests/saveScreenshot.jsAGENTS.mdsrc/tests/httpRequest.js
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Applies to src/tests/**/*.js : Use `setViewportSize()` to set inner viewport dimensions, which calculates delta from window size rather than setting window size directly
Applied to files:
src/tests/runShell.jssrc/tests/saveScreenshot.jsAGENTS.md
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Applies to src/**/*.js : Use `getAvailableApps()` to detect installed browsers rather than hardcoding browser paths, accounting for platform-specific locations
Applied to files:
AGENTS.md
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Applies to src/tests/httpRequest.js : Reference OpenAPI operations by `operationId` (e.g., `step.httpRequest.openApi.operationId = 'getUserById'`) rather than path/method combinations
Applied to files:
src/tests/httpRequest.js
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Applies to src/openapi.js : In OpenAPI integration, extract request/response examples from spec, validate payloads via AJV against schemas, and support mock responses by setting `step.httpRequest.openApi.mockResponse = true`
Applied to files:
src/tests/httpRequest.js
🧬 Code graph analysis (3)
src/tests/runShell.js (2)
src/tests/httpRequest.js (2)
fractionalDiff(457-460)existingFile(454-454)src/utils.js (1)
fractionalDiff(279-279)
src/tests/saveScreenshot.js (3)
src/tests/httpRequest.js (2)
fractionalDiff(457-460)result(12-12)src/tests/runShell.js (2)
fractionalDiff(138-141)result(15-25)src/utils.js (1)
fractionalDiff(279-279)
src/utils.js (3)
src/tests/httpRequest.js (1)
fractionalDiff(457-460)src/tests/runShell.js (1)
fractionalDiff(138-141)src/tests/saveScreenshot.js (1)
fractionalDiff(289-289)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)
- GitHub Check: test (macos-latest, 24)
- GitHub Check: test (macos-latest, 20)
- GitHub Check: test (windows-latest, 22)
- GitHub Check: test (macos-latest, 22)
- GitHub Check: test (windows-latest, 24)
- GitHub Check: test (windows-latest, 20)
- GitHub Check: test (ubuntu-latest, 20)
- GitHub Check: test (ubuntu-latest, 24)
- GitHub Check: test (ubuntu-latest, 22)
🔇 Additional comments (3)
AGENTS.md (1)
112-112: Fractional variation doc matches new semanticsThe updated
maxVariationdescription (fractional 0–1, e.g.,0.1= 10%) correctly aligns with the new comparison logic used in the tests and utilities.src/tests/runShell.js (1)
5-6: Shell output diff now correctly uses fractional semanticsUsing
calculateFractionalDifferenceon the actual file contents and comparingfractionalDiff > step.runShell.maxVariationbrings this step in line with the new fractionalmaxVariationcontract. Overwrite behavior for"false","true", and"aboveVariation"remains intact.No functional issues spotted here; any further tweaks (e.g., message wording) would be purely cosmetic.
Also applies to: 137-153
src/utils.js (1)
15-15: NewcalculateFractionalDifferenceimplementation is correctLevenshtein distance normalized by max string length with proper zero-length guard yields accurate 0–1 fractional differences. All call sites in
runShell.js,httpRequest.js, andsaveScreenshot.jscorrectly compare fractional differences directly againstmaxVariationthresholds as decimal values (0.1 for 10% tolerance). No stale references to the previouscalculatePercentageDifferenceAPI remain in the codebase.
|
📝 Documentation updates detected! New suggestion: Update maxVariation documentation to use fractional terminology |
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
src/tests/httpRequest.js (1)
305-319: LGTM! Formatting improvements enhance readability.The whitespace additions in the required fields validation block (lines 305-319) and the
fieldExistsAtPathfunction (lines 496-537) improve code organization without any functional changes.Also applies to: 496-537
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
src/tests/httpRequest.js(7 hunks)src/tests/runShell.js(2 hunks)src/tests/saveScreenshot.js(2 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- src/tests/runShell.js
🧰 Additional context used
📓 Path-based instructions (4)
src/tests/*.js
📄 CodeRabbit inference engine (AGENTS.md)
Create new step type handlers in
src/tests/[actionName].jsexporting async functions that validate schema, resolve to object, set defaults, and return{ status: 'PASS'|'FAIL'|'WARNING', description: string, outputs: {} }
Files:
src/tests/httpRequest.jssrc/tests/saveScreenshot.js
src/**/*.js
📄 CodeRabbit inference engine (AGENTS.md)
src/**/*.js: Uselog(config, level, message)for logging where level is 'debug'|'info'|'warning'|'error', passing config object as first parameter
UsegetAvailableApps()to detect installed browsers rather than hardcoding browser paths, accounting for platform-specific locations
Files:
src/tests/httpRequest.jssrc/tests/saveScreenshot.js
src/tests/**/*.js
📄 CodeRabbit inference engine (AGENTS.md)
src/tests/**/*.js: Always handle driver cleanup in try/finally blocks to ensure WebDriver sessions are properly terminated
Set step variables viastep.variables = { MY_VAR: '$$response.body.token' }to store runtime values as environment variables for subsequent steps
UsesetViewportSize()to set inner viewport dimensions, which calculates delta from window size rather than setting window size directly
TreatmaxVariationpercentage values as decimals (0.1 = 10%) but multiply by 100 when performing percentage-based comparisons
File overwrite modes support three values: 'false' (never overwrite), 'true' (always overwrite), 'aboveVariation' (overwrite only if content differs > maxVariation)
Files:
src/tests/httpRequest.jssrc/tests/saveScreenshot.js
src/tests/httpRequest.js
📄 CodeRabbit inference engine (AGENTS.md)
Reference OpenAPI operations by
operationId(e.g.,step.httpRequest.openApi.operationId = 'getUserById') rather than path/method combinations
Files:
src/tests/httpRequest.js
🧠 Learnings (6)
📓 Common learnings
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Applies to src/tests/**/*.js : Treat `maxVariation` percentage values as decimals (0.1 = 10%) but multiply by 100 when performing percentage-based comparisons
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Applies to src/tests/**/*.js : Treat `maxVariation` percentage values as decimals (0.1 = 10%) but multiply by 100 when performing percentage-based comparisons
Applied to files:
src/tests/httpRequest.jssrc/tests/saveScreenshot.js
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Applies to src/tests/**/*.js : File overwrite modes support three values: 'false' (never overwrite), 'true' (always overwrite), 'aboveVariation' (overwrite only if content differs > maxVariation)
Applied to files:
src/tests/httpRequest.jssrc/tests/saveScreenshot.js
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Applies to src/tests/httpRequest.js : Reference OpenAPI operations by `operationId` (e.g., `step.httpRequest.openApi.operationId = 'getUserById'`) rather than path/method combinations
Applied to files:
src/tests/httpRequest.js
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Applies to src/openapi.js : In OpenAPI integration, extract request/response examples from spec, validate payloads via AJV against schemas, and support mock responses by setting `step.httpRequest.openApi.mockResponse = true`
Applied to files:
src/tests/httpRequest.js
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Applies to src/tests/**/*.js : Use `setViewportSize()` to set inner viewport dimensions, which calculates delta from window size rather than setting window size directly
Applied to files:
src/tests/saveScreenshot.js
🧬 Code graph analysis (2)
src/tests/httpRequest.js (2)
src/tests/runShell.js (5)
require(1-1)require(2-6)result(15-25)fractionalDiff(138-141)existingFile(135-135)src/utils.js (1)
fractionalDiff(279-279)
src/tests/saveScreenshot.js (2)
src/tests/runShell.js (2)
fractionalDiff(138-141)result(15-25)src/utils.js (1)
fractionalDiff(279-279)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)
- GitHub Check: test (macos-latest, 24)
- GitHub Check: test (ubuntu-latest, 20)
- GitHub Check: test (macos-latest, 20)
- GitHub Check: test (macos-latest, 22)
- GitHub Check: test (windows-latest, 24)
- GitHub Check: test (windows-latest, 22)
- GitHub Check: test (windows-latest, 20)
- GitHub Check: test (ubuntu-latest, 24)
- GitHub Check: test (ubuntu-latest, 22)
🔇 Additional comments (3)
src/tests/saveScreenshot.js (1)
289-366: LGTM! Fractional difference standardization implemented correctly.The conversion from percentage-based to fractional-based difference calculation is well-executed:
- Line 339 correctly calculates
fractionalDiffasnumDiffPixels / (width * height)(0-1 scale) instead of multiplying by 100 (0-100 scale)- The comparison at line 347 now directly compares fractional values without scaling
- Log messages and user-facing descriptions consistently use fractional formatting with
.toFixed(2)This standardization simplifies the logic by keeping both
fractionalDiffandmaxVariationon the same 0-1 scale throughout.Based on learnings, the previous approach required multiplying by 100 for percentage-based comparisons, but this PR eliminates that complexity.
src/tests/httpRequest.js (2)
7-7: LGTM! Import updated for renamed function.The import correctly reflects the function rename from
calculatePercentageDifferencetocalculateFractionalDifference.
456-480: LGTM! Fractional difference logic correctly implemented.The changes properly convert from percentage-based to fractional-based comparison:
- Function call updated to
calculateFractionalDifference(line 457)- Direct comparison of fractional values at line 463
- Messages correctly format fractional values with
.toFixed(2)- Logic is consistent with the same changes in
saveScreenshot.jsThe default
maxVariationof 0 (line 188) appropriately enforces exact matches for HTTP responses unless explicitly configured otherwise.
There was a problem hiding this comment.
Pull request overview
This PR standardizes the handling of variation thresholds across the codebase by refactoring from percentage-based (0-100) to fractional-based (0-1) calculations. The function calculatePercentageDifference() is renamed to calculateFractionalDifference() and now returns a decimal fraction instead of a percentage string. This change eliminates inconsistencies where some code paths were multiplying by 100 while others were not, making the comparison logic clearer and more uniform.
Key changes:
- Renamed
calculatePercentageDifference()tocalculateFractionalDifference()with updated return type (number 0-1 instead of percentage string) - Updated all variation threshold comparisons across test utilities to use fractional values directly
- Improved user-facing messages to display fractional differences with 2 decimal places
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/utils.js | Renamed function to calculateFractionalDifference, added JSDoc, removed percentage conversion, added empty string edge case handling, returns raw fractional value |
| src/tests/saveScreenshot.js | Updated variable names, comparison logic, and warning/success messages to use fractional values instead of percentages |
| src/tests/runShell.js | Updated variable names, comparison logic, log messages, and error messages to use fractional values instead of percentages |
| src/tests/httpRequest.js | Updated variable names, comparison logic, log messages, error messages to use fractional values; removed erroneous * 100 multiplication; includes minor formatting cleanup |
| AGENTS.md | Updated documentation to clarify that maxVariation uses fractional values (0.1 = 10% tolerance) and comparisons use fractions directly |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
AGENTS.md(5 hunks)src/tests/httpRequest.js(6 hunks)src/tests/runShell.js(3 hunks)src/tests/saveScreenshot.js(5 hunks)
🚧 Files skipped from review as they are similar to previous changes (3)
- src/tests/runShell.js
- src/tests/httpRequest.js
- src/tests/saveScreenshot.js
🧰 Additional context used
🧠 Learnings (17)
📓 Common learnings
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Applies to src/tests/**/*.js : Treat `maxVariation` percentage values as decimals (0.1 = 10%) but multiply by 100 when performing percentage-based comparisons
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Update Resolver workflow syncs dependencies: installs specified resolver version + matching `doc-detective-common` version, runs tests, bumps patch version, creates release with aggregated PR notes and resolver changelog
Applied to files:
AGENTS.md
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Applies to src/tests.js : Core test runner `src/tests.js` orchestrates Appium/WebDriver sessions and manages spec → test → context → step hierarchy (600+ LOC orchestrator)
Applied to files:
AGENTS.md
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Input resolution via `doc-detective-resolver` detects tests from docs/specs and resolves to executable format as first step in test execution flow
Applied to files:
AGENTS.md
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Tests follow nested hierarchy: spec (file) → test → context (browser/platform combo) → step (action), with contexts running serially and skipping if unsupported
Applied to files:
AGENTS.md
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Test & Publish workflow runs cross-platform tests (Ubuntu/Windows/macOS × Node 18/20/22/24) before publishing; triggers downstream `doc-detective` update via repository dispatch
Applied to files:
AGENTS.md
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Each step type has dedicated handler module in `src/tests/` (e.g., `httpRequest.js`, `runShell.js`) implementing specific action logic
Applied to files:
AGENTS.md
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Applies to src/index.js : Entry point `src/index.js` must expose `runTests()` function as the primary API for test execution
Applied to files:
AGENTS.md
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Run local test suite with `npm test` (mocha), use `node dev` for development/manual testing, run `npm run depcheck` to check unused dependencies
Applied to files:
AGENTS.md
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Auto Dev Release workflow skips on `[skip ci]` commits, release commits, or docs-only changes; increments dev version format `X.Y.Z-dev.N`
Applied to files:
AGENTS.md
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Applies to src/tests.js : Register new step types in `src/tests.js`: add action to `driverActions` array if requires browser, add case in `runStep()` switch statement
Applied to files:
AGENTS.md
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Applies to scripts/postinstall.js : Post-install script (`scripts/postinstall.js`) must auto-download Chrome/Firefox/ChromeDriver to `browser-snapshots/` directory matching current platform
Applied to files:
AGENTS.md
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Applies to src/tests/*.js : Create new step type handlers in `src/tests/[actionName].js` exporting async functions that validate schema, resolve to object, set defaults, and return `{ status: 'PASS'|'FAIL'|'WARNING', description: string, outputs: {} }`
Applied to files:
AGENTS.md
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Applies to src/tests.js : Configure Appium drivers (`chromium`, `gecko`, `safari` on macOS) with default 10-minute timeout via `newCommandTimeout: 600`
Applied to files:
AGENTS.md
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Applies to src/tests/**/*.js : Set step variables via `step.variables = { MY_VAR: '$$response.body.token' }` to store runtime values as environment variables for subsequent steps
Applied to files:
AGENTS.md
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Applies to src/tests/**/*.js : Always handle driver cleanup in try/finally blocks to ensure WebDriver sessions are properly terminated
Applied to files:
AGENTS.md
📚 Learning: 2025-11-26T19:31:27.066Z
Learnt from: CR
Repo: doc-detective/core PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-26T19:31:27.066Z
Learning: Applies to src/config.js : Validate configuration objects via `doc-detective-common` schemas using `validate({ schemaKey: 'config_v3', object })`
Applied to files:
AGENTS.md
🪛 markdownlint-cli2 (0.18.1)
AGENTS.md
88-88: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)
- GitHub Check: test (macos-latest, 24)
- GitHub Check: test (windows-latest, 24)
- GitHub Check: test (windows-latest, 20)
- GitHub Check: test (windows-latest, 22)
- GitHub Check: test (macos-latest, 20)
- GitHub Check: test (macos-latest, 22)
- GitHub Check: test (ubuntu-latest, 24)
- GitHub Check: test (ubuntu-latest, 22)
- GitHub Check: test (ubuntu-latest, 20)
🔇 Additional comments (1)
AGENTS.md (1)
130-130: Documentation accurately reflects the fractional comparison logic.The documentation on line 130 correctly states that
maxVariationuses fractional values directly (0.1 = 10% tolerance) without percentage conversion. This is confirmed by thecalculateFractionalDifference()function in src/utils.js, which explicitly documents: "Compare against maxVariation thresholds directly (e.g., 0.1 for 10% tolerance)." The PR successfully standardizes to fractional differences (0–1) throughout the codebase.
Standardize the logic and logging for
maxVariationto use fractional differences instead of percentages. RenamecalculatePercentageDifference()tocalculateFractionalDifference()and update related documentation accordingly.Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.