Skip to content

Feature/ci html budgets #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Agent-Hellboy merged 8 commits into from
Apr 13, 2026
Merged

Feature/ci html budgets #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
b69980d
Add budgets, HTML report, and CI workflow
Agent-Hellboy Apr 13, 2026
d2037ca
Fix regex parsing and crop robustness
Agent-Hellboy Apr 13, 2026
cc2a7fd
Modularize audits and evaluation
Agent-Hellboy Apr 13, 2026
e547398
Reduce dead-link and lazy image false positives
Agent-Hellboy Apr 13, 2026
53a3950
Reduce crop failures and stabilize scroll state
Agent-Hellboy Apr 13, 2026
8681e09
Resolve review comments and harden CI
Agent-Hellboy Apr 13, 2026
12e7eec
Refactor audit flow into pipeline
Agent-Hellboy Apr 13, 2026
443ee66
refactor: organize src structure
Agent-Hellboy Apr 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
name: CI

on:
push:
branches: [main]
tags:
- 'v*.*.*'
pull_request:
branches: [main]

jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: npm
- name: Install dependencies
run: npm ci
- name: Install Playwright browsers (chromium only)
run: npx playwright install chromium --with-deps
- name: Smoke run against example.com
run: npm run smoke
- name: Upload smoke artifacts (report + shots)
if: always()
uses: actions/upload-artifact@v4
with:
name: smoke-artifacts
path: |
/tmp/uxray-ci.json
/tmp/uxray-shots

publish:
needs: test
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/v')
permissions:
contents: read
id-token: write
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
registry-url: https://registry.npmjs.org
- name: Install dependencies
run: npm ci
- name: Publish to npm
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
run: npm publish --provenance
88 changes: 38 additions & 50 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,17 +1,10 @@
# UXRay

Audit any hosted web app for quick UI/UX health:

- Layout issues: horizontal overflow offenders, scroll snapshots.
- Touch ergonomics: finds tap targets smaller than 44px.
- Accessibility: runs axe-core via Playwright.
- Theme signals: samples top colors and fonts.
- Stability: logs console errors/warnings and failed network requests.
- Evidence: full-page + stepped viewport screenshots.
Fast UI/UX audit CLI for live web apps. Produces a JSON report plus evidence screenshots, and can emit an HTML summary.

## Prereqs
- Node.js 18+ (Playwright uses modern APIs).
- One-time: download bundled browsers
- Node.js 20+
- One-time browser install:

```bash
npx playwright install
Expand All @@ -23,10 +16,10 @@ npx playwright install
npm install
```

Or run directly after cloning with `npx` (no global install):
Or run without installing (after cloning):

```bash
npx ./ui-review.js --url https://example.com
npx uxray --url https://example.com
```

## Usage
Expand All @@ -37,47 +30,42 @@ npm run review -- --url https://your-app.com \
--viewport 1366x768 \
--steps 4 \
--wait 2000 \
--wait-until load \
--ready-selector '#app' \
--target-policy wcag22-aa \
--axe-tags wcag21aa,wcag2aa \
--html \
--max-a11y 5 \
--max-small-targets 10 \
--max-overflow 2 \
--max-console 3 \
--max-http-errors 0 \
--trace \
--out ./reports/uxray-report.json \
--shots ./reports/shots
```

Short flags:
- `--url` (required) target page.
- `--mobile` also run an iPhone 12 emulation pass.
- `--viewport` desktop viewport, e.g. `1440x900` (default 1280x720).
- `--steps` number of viewport screenshots while scrolling (default 4).
- `--wait` extra ms to settle after load (default 1500ms).
- `--out` output report path (JSON).
- `--shots` screenshots root folder.

After a run you’ll see:
- `reports/ui-report-<timestamp>.json` with counts and offenders.
- `reports/shots-<timestamp>/desktop|mobile` PNGs.
## CLI flags (high-value)
- `--url` target page (required)
- `--mobile` add iPhone 12 pass
- `--viewport 1440x900`
- `--steps` viewport screenshots while scrolling
- `--wait` extra settle time (ms)
- `--wait-until` load|domcontentloaded|networkidle|commit
- `--ready-selector` wait for selector after navigation
- `--target-policy` wcag22-aa | wcag21-aaa | lighthouse
- `--axe-tags` comma tags for axe rules
- `--html` emit HTML report (optional path)
- `--trace` capture Playwright trace
- Budget gates: `--max-a11y`, `--max-small-targets`, `--max-overflow`, `--max-console`, `--max-http-errors`

## JSON report shape (excerpt)

```json
{
"url": "https://example.com",
"runAt": "2026-04-10T18:00:00.000Z",
"desktop": {
"viewport": "1280x720",
"navTimeMs": 2310,
"perf": { "domContentLoaded": 980, "load": 1500, "renderBlocking": 210 },
"overflow": { "hasOverflowX": false, "offenders": [] },
"tapTargets": { "smallTargetCount": 2, "samples": [...] },
"viewportMeta": true,
"styles": { "topColors": [...], "topFonts": [...] },
"axe": { "summary": { "violations": 3, "passes": 150, "incomplete": 0 }, "topViolations": [...] },
"screenshots": ["reports/shots.../desktop-full.png", "..."],
"networkIssues": [],
"consoleIssues": []
},
"mobile": { ... }
}
```
## Outputs
- JSON report: `reports/ui-report-<timestamp>.json`
- Screenshots: `reports/shots-<timestamp>/desktop|mobile`
- Optional HTML: `reports/ui-report-<timestamp>.html`
- Optional trace: `reports/shots-<timestamp>/desktop|mobile-trace.zip`
- Crops: `reports/shots-<timestamp>/desktop|mobile/crops/`

## Notes
- Tool runs headless; remove `headless: true` in `ui-review.js` if you want to watch it.
- If Playwright can’t launch browsers on macOS due to permissions, try running outside sandboxed shells or re-run `npx playwright install`.
- Reports and screenshots are git-ignored.
## CI & release
- GitHub Actions runs `npm run smoke` on PRs and publishes artifacts.
- Tag `v*.*.*` to publish to npm (requires `NPM_TOKEN`).
4 changes: 3 additions & 1 deletion package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@
"uxray": "./ui-review.js"
},
"scripts": {
"review": "node ui-review.js"
"review": "node ui-review.js",
"smoke": "node scripts/smoke.js",
"ci": "npm run smoke"
},
"repository": {
"type": "git",
Expand Down
21 changes: 21 additions & 0 deletions scripts/smoke.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
#!/usr/bin/env node
const os = require('os');
const path = require('path');
const { spawnSync } = require('child_process');

const outFile = path.join(os.tmpdir(), 'uxray-ci.json');
const shotsDir = path.join(os.tmpdir(), 'uxray-shots');

const args = [
path.join(__dirname, '..', 'ui-review.js'),
'--url', 'https://example.com',
'--steps', '1',
'--wait', '800',
'--wait-until', 'load',
'--target-policy', 'wcag21-aaa',
'--out', outFile,
'--shots', shotsDir,
];

const result = spawnSync('node', args, { stdio: 'inherit' });
process.exit(result.status === null ? 1 : result.status);
Comment on lines +20 to +21
Copy link
Copy Markdown

@coderabbitai coderabbitai bot Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify timeout/error-handling are currently absent/present in the smoke runner.
rg -n -C3 "spawnSync\\(|timeout|killSignal|result\\.error|process\\.exit\\(" scripts/smoke.js

Repository: Agent-Hellboy/UXRay

Length of output: 224

🏁 Script executed:

cat -n scripts/smoke.js

Repository: Agent-Hellboy/UXRay

Length of output: 808

Add timeout + explicit spawn error handling for CI reliability.

Line 20 runs a blocking child process without timeout. If the test or underlying Playwright process hangs, this can stall the CI job until the global workflow timeout. Additionally, line 21 does not check result.error, which means spawn failures (e.g., ENOENT, permission issues) go undiagnosed in logs.

Proposed fix 
-const result = spawnSync('node', args, { stdio: 'inherit' });
-process.exit(result.status === null ? 1 : result.status);
+const result = spawnSync(process.execPath, args, {
+  stdio: 'inherit',
+  timeout: 180000,
+  killSignal: 'SIGKILL',
+});
+
+if (result.error) {
+  console.error('Smoke run failed to start:', result.error.message);
+  process.exit(1);
+}
+
+process.exit(typeof result.status === 'number' ? result.status : 1);
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const result = spawnSync('node', args, { stdio: 'inherit' });
process.exit(result.status === null ? 1 : result.status);
const result = spawnSync(process.execPath, args, {
stdio: 'inherit',
timeout: 180000,
killSignal: 'SIGKILL',
});
if (result.error) {
console.error('Smoke run failed to start:', result.error.message);
process.exit(1);
}
process.exit(typeof result.status === 'number' ? result.status : 1);
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@scripts/smoke.js` around lines 20 - 21, The current call to spawnSync('node',
args, { stdio: 'inherit' }) should be hardened for CI by adding a timeout (e.g.,
timeout: <ms>) and explicit spawn error handling: pass a timeout option to
spawnSync, check result.error after the call and log the error to stderr (or
process.stderr.write) with context including result.error.code/message, and then
exit with a non-zero status; retain the existing exit behavior for result.status
when there is no error but ensure that a timeout or spawn failure causes a clear
error log and non-zero exit. Use the existing symbols spawnSync, args, result,
and process.exit when applying the change.

27 changes: 27 additions & 0 deletions src/audits/a11y.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
const { AxeBuilder } = require('@axe-core/playwright');

async function runAxe(page, { axeTags }) {
let builder = new AxeBuilder({ page });
if (axeTags && Array.isArray(axeTags) && axeTags.length) {
builder = builder.withTags(axeTags);
}
const results = await builder.analyze();
return {
summary: {
violations: results.violations.length,
passes: results.passes.length,
incomplete: results.incomplete.length,
},
topViolations: results.violations.slice(0, 10).map((v) => ({
id: v.id,
description: v.description,
impact: v.impact,
help: v.help,
nodes: v.nodes.slice(0, 4).map((n) => ({ target: n.target, summary: n.failureSummary })),
})),
};
}

module.exports = {
runAxe,
};
Loading
Loading