Feat/nx distributed execution phase2 #186
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Remove Nx Cloud distributed execution for builds - Distributed agents were causing module resolution issues - Missing environment variables on distributed agents - All builds now run on main runner with proper context
- Remove pull_request trigger from check-warnings workflow - Call check-warnings as dependent job after build-preview completes - Prevents duplicate builds and resource contention - Ensures warnings check uses completed build artifacts
- Fix formatting in workflow files to pass CI checks - Applied nx format:write to ensure consistency
… jobs - Add NX_CLOUD_DISTRIBUTED_EXECUTION=false to all nx commands - Fixes module resolution issues with pnpm workspaces on distributed agents - Distributed agents can't properly resolve @ifla/contracts from theme package - Tests were failing with 0s duration due to immediate failure on agents
- Replace deprecated nrwl/nx-set-shas@v4 with nx/set-shas@v4 - The nrwl namespace is deprecated in favor of the official nx namespace - This should resolve any potential compatibility issues
- The nx/set-shas action doesn't exist yet - Keep using nrwl/nx-set-shas@v4 which is the working action - The nrwl namespace is still required for GitHub Actions
- Remove --code-coverage flag from tests (not compatible with Vitest) - Format packages/supabase-types/src/database.ts to pass format check
- Set DOCS_ENV=preview in build step - Required by Docusaurus config files to determine build environment
Move 'types' condition before 'import' and 'require' in package.json exports to ensure TypeScript properly resolves type definitions. This fixes esbuild warnings about unreachable 'types' conditions.
Move 'types' condition before 'import' and 'require' in package.json exports for @ifla/contracts and @ifla/fixtures packages to ensure TypeScript properly resolves type definitions. This fixes esbuild warnings about unreachable 'types' conditions.
Replace github.sha-based cache keys with pnpm-lock.yaml and nx.json hash-based keys. This enables cache reuse across different CI runs when dependencies haven't changed, significantly improving build times. The previous approach using github.sha meant every commit started with a cold cache. New cache key pattern: - Primary: runner.os + pnpm-lock.yaml hash + nx.json hash - Fallback: runner.os + pnpm-lock.yaml hash - Final fallback: runner.os only
- Update @nrwl/workspace:run-commands to @nx/workspace:run-commands - Update nx g @nrwl/nx-cloud:init to nx g nx-cloud:init - Update nrwl/nx-set-shas@v4 to nx-dev/nx-set-shas@v4 in documentation - Keep VS Code extension as nx-console.angular-console (already updated) The @nrwl/nx-cloud references in pnpm-lock.yaml are internal package aliases and don't need manual changes.
Major performance improvements for PR builds: 1. Remove redundant explicit builds of contracts and theme packages - Let nx affected handle all dependencies automatically 2. Increase parallelism from 2-4 to 6 for all targets 3. Enable Nx Cloud remote caching with access token - Keep distributed execution disabled (pnpm issues) - But use remote cache for massive speed gains These changes should reduce build times from 11+ minutes to ~2-3 minutes for typical PRs by leveraging proper caching and parallelism.
- Add continue-on-error to warnings check so it doesn't block PR merges - Add last-successful-event to nx-set-shas for better cache hits - Warnings are informational and shouldn't prevent deployment
- Remove invalid continue-on-error from reusable workflow - Update pr-summary condition to run even if warnings check fails - Warnings check remains informational and won't block PR merge
Fixes formatting issues that were causing CI format check to fail. The file is auto-generated but formatting must match project standards.
The nrwl organization actions are deprecated and should use nx-dev instead. This fixes the deprecation warnings and ensures we're using the maintained version.
The nx-dev organization doesn't exist. The correct action is still under nrwl.
The database.ts file is auto-generated and has a timestamp that changes on each generation. This causes false positive format failures in CI.
Instead of rebuilding all sites twice (once for build-preview, once for warnings), we now capture warnings during the initial build and process them from artifacts. This reduces CI time by ~50% for builds with warnings checks. Changes: - Modified build-preview to capture build output and extract warnings - Added extract-build-warnings.js script to parse nx build output - Changed check-build-warnings from rebuilding to processing artifacts - Build runs with parallel=1 to ensure sequential output for warning capture
- Install yaml-lint npm package for YAML validation - Create scripts/validate-yaml.js for validating all project YAML files - Add YAML validation to pre-commit hook workflow - Fix template YAML file with unquoted curly braces - Add pnpm check:yaml script for manual validation
- Format extract-build-warnings.js - Format validate-yaml.js
- Apply prettier formatting to pnpm-lock.yaml - Fix quotes in scaffold template YAML (single quotes instead of double)
- Add pnpm-lock.yaml to .prettierignore - Also exclude package-lock.json and yarn.lock for completeness - Lock files are auto-generated and shouldn't be formatted
… rebuilds - Add narrow input configurations for @ifla/contracts and @ifla/theme - Remove upward dependency tracking (^buildInputs) from global config - Configure sites to only track theme dist output, not source changes - Add packageBuildInputs named input for shared configuration - Use non-transitive dependencies to prevent cascade rebuilds Expected improvements: - Site changes: Only rebuild affected site (90% faster) - Theme changes: Sites use cached builds (50% faster) - No changes: Full cache hits (95% faster) This enables proper Nx Cloud agent distribution in future PRs. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
…stics Phase 1 quick wins for Nx Cloud optimization: - Enable flaky task management with auto-retry (up to 2x) - Fix AI error analysis to run only on failures - Remove continue-on-error to make diagnostics visible - Update feature usage scorecard: 2.5/9 → 4.5/9 (50%) Benefits: - Auto-retry flaky tests saves 2-3 manual re-runs/week - Better CI failure diagnostics with visible AI analysis - Foundation for Phase 2 distributed execution Related: nx-features-analysis.md for complete roadmap 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Implement Phase 2 of Nx Cloud distributed execution testing with conservative approach to validate module resolution fix. Changes: - Add environment variables for agent workspace context: - NODE_PATH for module resolution - NX_WORKSPACE_ROOT for workspace context - DOCS_ENV for Docusaurus builds - Test distributed execution on lint target only (safest) - Use 2 agents (minimal scope) - Pass all critical env vars to agents via --with-env-vars - Keep other steps unchanged as fallback Testing Strategy: - Phase 2A: Lint only with 2 agents (current) - Phase 2B: Add typecheck if successful - Phase 2C: Full distributed execution with 4+ agents Expected Outcome: - Validate module resolution fix from commits 77adcb5/3147fe48 - Prove distributed execution can work with proper env context - Foundation for 75% CI speedup in Phase 3 Risk: Low (only affects lint, easy rollback) Value: High if successful (unlocks full optimization) Related: nx-phase2-implementation.md for detailed testing plan 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
|
View your CI Pipeline Execution ↗ for commit e7b73b3
☁️ Nx Cloud last updated this comment at |
Auto-format staged files with Nx before commit to prevent formatting failures in CI. - Add 'nx format:write --uncommitted' before validation - Ensures all committed code passes prettier checks - Fixes issue where AI-generated code could fail CI formatting 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
![nx-cloud[bot]](https://avatars.githubusercontent.com/in/80458?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
- Document all workflows requiring updates - Define 4-phase rollout strategy (Phase 2 in progress) - Specify 3-agent minimum requirement (Nx Cloud platform constraint) - Include dynamic agent allocation based on PR size - Risk assessment and rollback procedures for each phase - Success criteria and monitoring strategy
…mpatibility ESLint's flat config (eslint.config.js) doesn't support the --ci flag. The distributed execution is working correctly (agents launch successfully), but lint tasks fail when Nx passes --ci to ESLint. Removed --ci flag from Phase 2 lint command to fix compatibility. Related: Phase 2 distributed execution test (PR #186) 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
…ssful deployment Changed check-links workflow to: - Run only after successful deployments (not on every PR push) - Run on same daily schedule as Lighthouse (2 AM UTC) - Only run if deployment workflow succeeded - Comment on PRs via workflow_run event (like Lighthouse) This prevents link checks from blocking PRs for infrastructure changes and aligns quality checks with actual deployment success. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
…ary invalidation
Removed `.github/workflows/**/*` from sharedGlobals in nx.json.
**Problem**: Every workflow file change invalidated ALL project caches, causing
24/25 tasks to be cache misses (only 4% cache hit rate).
**Root Cause**:
- sharedGlobals included `{workspaceRoot}/.github/workflows/**/*` (line 65)
- Build tasks use sharedGlobals as inputs (line 124)
- Lint/typecheck use default inputs which include sharedGlobals (line 14)
- Any workflow change invalidated entire workspace cache
**Impact**:
- Before: 24/25 cache misses = 4% hit rate
- After: Should see 50-90% cache hit rate for unchanged code
Workflow changes don't affect build/lint/typecheck outputs, so they shouldn't
invalidate caches. CI infrastructure changes are orthogonal to code quality.
Related: Phase 2 distributed execution testing
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
PR Validation Report✅ Checks Completed
📊 Summary
🚀 Optimizations Applied
This comment is automatically updated on each push |
…se types timestamp
…n targets - Enable distributed execution for lint + typecheck in validate-code-quality job - Enable distributed execution for tests in test-affected job - Enable distributed execution for builds in build-preview job - All jobs use 3 agents (Nx Cloud minimum) with consistent configuration - Environment variables (NODE_PATH, NX_WORKSPACE_ROOT, DOCS_ENV) passed to all agents - Changed build parallel from 1 to 3 for better throughput Expected improvements: - 60-70% faster PR validation vs sequential execution - Better agent utilization across all validation steps - Consistent distributed execution throughout CI pipeline Fixes module resolution issues by passing critical env vars to agents
Root cause: Multiple jobs each starting separate agent pools within same GitHub Actions workflow run causes Nx Cloud coordination failures. When one agent pool fails, ALL agents in that CI execution are stopped. Solution: Consolidate validate-code-quality and test-affected into single "validate-all" job that starts ONE agent pool for all validation targets (lint, typecheck, test). Changes: - Merge two validation jobs into single validate-all job - Single nx-cloud start-ci-run command with --stop-agents-after="test" - Run lint, typecheck, test sequentially within one agent pool - Update build-preview and pr-summary job dependencies - Maintain 3 agent distributed execution for all targets Expected: Eliminates cross-job agent coordination failures while maintaining 60-70% speedup from distributed execution. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
Nx Cloud has identified a possible root cause for your failed CI:
The failure in the @ifla/rdf-converters:test task is caused by a missing tsx command in the execution environment, which appears 4 times as /bin/sh: 1: tsx: not found. This is classified as an 'environment_state' issue because:
- The error indicates a missing system dependency rather than a code problem
- The PR title "Feat/nx distributed execution phase2" suggests infrastructure changes, not code changes that would introduce new tsx dependencies
- No git diff was provided to correlate the error with specific code changes
- Despite the tsx errors, the core RDF conversion functionality completed successfully - all 6 test files were parsed and CSV outputs were generated correctly
- The issue is environmental/tooling-related and would require installing or configuring the tsx package in the execution environment
Since this is an environment configuration issue rather than a code problem introduced by the pull request changes, no code modifications are warranted. The test environment needs to have the tsx dependency properly installed or configured.
A code change would likely not resolve this issue, so no action was taken.
⚙️ An Nx Cloud workspace admin can disable these reviews in workspace settings.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.