Skip to content

refactor: improve Percy build verification process#7

Open
devin-ai-integration[bot] wants to merge 3 commits intomainfrom
devin/1768170748-refactor-percy-verification
Open

refactor: improve Percy build verification process#7
devin-ai-integration[bot] wants to merge 3 commits intomainfrom
devin/1768170748-refactor-percy-verification

Conversation

@devin-ai-integration
Copy link
Copy Markdown

@devin-ai-integration devin-ai-integration Bot commented Jan 11, 2026

refactor: improve Percy build verification with centralized error handling

Summary

This PR refactors the Percy build verification process to make it more robust and maintainable. The main changes include:

New centralized error handling utility (percy-error.ts) that provides:

  • Custom PercyError class with standardized error codes
  • PercyResult<T> type for consistent success/failure handling
  • withRetry utility with configurable exponential backoff
  • Logging utilities (logDebug, logWarning, logError)
  • Build status formatting (formatBuildStatus, isAlertWorthy)

Improved build ID extraction (read-build-number-from-logs.ts):

  • Replaced fragile string parsing with regex-based extraction
  • Added build ID validation
  • New extractBuildIdWithDetails function with detailed result information

Simplified GitHub Actions workflow (e2e.yml):

  • Extracted check_logs function for log analysis
  • Created is_retryable_error function for retry logic
  • Cleaner retry mechanism with dedicated functions

Better affected detection (e2e-affected/action.yml):

  • Improved Percy token validation with helpful error messages
  • Extracted helper functions for commit fetching
  • Parallel execution of commit fetching operations

Updates since last revision

  • Added tests to achieve 100% branch coverage (was 98.77%)
  • Removed unnecessary nullish coalescing in isAlertWorthy function that created unreachable code
  • Added test for non-Error thrown values in file read operations

Review & Testing Checklist for Human

This is a medium-risk change affecting critical CI/CD infrastructure. Please verify:

  • Regex patterns: The build ID extraction patterns were changed from [^/]+\/[^/]+ (exactly 2 path segments) to .* (any segments). Verify this matches all expected Percy URL formats like https://percy.io/82a32315/web/skyux-integration-e2e/builds/41420284
  • Retry logic in e2e.yml: The bash script changes should be tested with actual Percy failures to ensure retries work correctly
  • Backward compatibility: The refactoring adds new module dependencies - verify existing workflows still function correctly

Recommended test plan:

  1. Run the E2E workflow on a test PR to verify Percy integration works end-to-end
  2. Manually trigger a scenario where Percy fails to verify retry logic
  3. Check that build ID extraction works with various Percy URL formats

Notes

  • All 188 tests pass with 100% code coverage (statements, branches, functions, lines)
  • Lint checks pass
  • CI checks for build, formatting, and unit tests all pass
  • E2E test failures in CI are due to missing Percy tokens in the fork environment, not code issues

Link to Devin run: https://app.devin.ai/sessions/51b556e9f4df4de095b9c69e0b4b5641
Requested by: @bcmake

- Create centralized error handling utility (percy-error.ts) with:
  - Custom PercyError class with standardized error codes
  - Result types for consistent error handling patterns
  - Retry utilities with configurable options and exponential backoff
  - Logging utilities for debug, warning, and error messages
  - Log pattern matching for retryable errors
  - Build status formatting and alert-worthy detection

- Refactor percy-api.ts to use centralized error handling:
  - Improved logging with context tracking
  - Better error messages with error codes

- Refactor read-build-number-from-logs.ts:
  - Replace fragile string parsing with regex-based extraction
  - Add build ID validation
  - Add detailed result types for better error tracking

- Simplify e2e.yml workflow:
  - Extract check_logs function for log analysis
  - Create is_retryable_error function for retry logic
  - Simplify retry mechanism with dedicated functions

- Improve e2e-affected/action.yml:
  - Better Percy token validation with error messaging
  - Extract constants for better maintainability
  - Add helper functions for parallel commit fetching

- Expand test coverage for all new utilities

Co-Authored-By: benc@cognition.ai <Benc@windsurf.com>
@devin-ai-integration
Copy link
Copy Markdown
Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

Co-Authored-By: benc@cognition.ai <Benc@windsurf.com>
Co-Authored-By: benc@cognition.ai <Benc@windsurf.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants