Flaky E2E Test: github-telegram-integration times out when run after acp-integration

[tbrandenburg]

Problem

The E2E test tests/e2e/github-telegram-integration.test.ts fails intermittently with a 30-second timeout when run as part of the full test suite, but passes consistently when run in isolation.

Symptoms

• Failure Mode: Test times out after 30 seconds
• Test: GitHub Auth + Telegram Notification E2E > should complete GitHub CLI auth workflow with current repository
• Trigger: Only fails when run after tests/e2e/acp-integration.test.ts in the full suite
• Isolation: Passes 100% of the time when run alone

Reproduction

# Fails (times out)
npm test -- tests/e2e/acp-integration.test.ts tests/e2e/github-telegram-integration.test.ts

# Passes
npm test -- tests/e2e/github-telegram-integration.test.ts

Analysis

Test Execution Order in Full Suite

1. acp-integration.test.ts runs first (26s)
2. github-telegram-integration.test.ts runs second - times out at 30s

Investigation Results

Process Leaks: ✅ None found

• No ACP processes left running after acp-integration.test.ts
• Cleanup handlers work correctly

Test Isolation: ⚠️ Potential issue

• Both tests use temporary directories with proper cleanup
• However, they may share resources:
  • GitHub API client (rate limiting?)
  • Context state in .work/contexts.json
  • Test execution environment variables

Timing Evidence:

# Isolated run
✓ github-telegram-integration (2 tests) 44s
  ✓ Test 1: 22s
  ✓ Test 2: 22s

# After ACP test
FAIL github-telegram-integration timeout after 30s

Likely Root Causes

1. GitHub API Rate Limiting

   • ACP test creates/modifies issues
   • Telegram test immediately tries to create more issues
   • Secondary rate limits may kick in

2. Test Teardown Race Condition

   • afterEach in ACP test may not fully complete before next test starts
   • Shared state in GitHub API client or authentication

3. Insufficient Timeout

   • Test has 30s default timeout
   • When run after ACP test, initialization takes longer
   • Adding delay between tests might help

Suggested Fixes

Option 1: Increase Test Timeout

it('should complete GitHub CLI auth workflow with current repository', async () => {
  // ... test code
}, 60000); // Increase from 30s to 60s

Option 2: Add Test Isolation

afterEach(async () => {
  // ... existing cleanup
  
  // Add delay to allow API rate limits to reset
  await new Promise(resolve => setTimeout(resolve, 2000));
});

Option 3: Mock GitHub API in E2E Tests

• Use nock or similar to mock GitHub API responses
• Eliminates rate limiting issues
• Improves test reliability and speed

Option 4: Run Tests in Parallel Groups

Configure Vitest to run these tests in separate groups:

// vitest.config.ts
export default {
  test: {
    sequence: {
      shuffle: false,
    },
    poolOptions: {
      threads: {
        singleThread: true, // Run E2E tests serially
      },
    },
  },
};

Impact

• Severity: Low (test is flaky, not a production bug)
• Frequency: Intermittent (occurs in full suite, not in isolation)
• Workaround: Run tests individually or re-run CI

Additional Context

Discovered during investigation of PR #1499 (issue #1362). The flaky test is unrelated to the PR changes - it's a pre-existing test ordering dependency.

Test file: tests/e2e/github-telegram-integration.test.ts
Lines: 48-113 (first test in the suite)

Environment

• Test Framework: Vitest 3.2.4
• Test Type: E2E (requires real GitHub API and Telegram bot)
• Dependencies: GitHub CLI, Telegram bot credentials

[CopilotMe]

"Passes in isolation, fails in suite" is a classic test interference pattern. FixSense is a free GitHub Action that analyzes failing E2E tests with AI — posts root cause, flakiness score, and app bug vs test bug classification as a PR comment.

- name: FixSense Analysis
  if: failure()
  uses: CopilotMe/fixsense-action@v1
  with:
    api-key: ${{ secrets.FIXSENSE_API_KEY }}

Free plan: 350 analyses/month.