feat: replace Playwright auth with CDP for private tweets by RichardBray · Pull Request #16 · RichardBray/x-dl

RichardBray · 2026-03-27T20:22:30Z

Summary

Replace deprecated Playwright persistent-context auth (--profile, --login, --verify-auth) with CDP-based approach via x-dl cdp <url>
New src/cdp.ts module connects to user's real Chrome browser via Chrome DevTools Protocol for authenticated tweet downloads
Auto-detects running Chrome, launches headlessly if needed, and triggers login flow when not authenticated

Test plan

bun test test/unit/ — 63 tests passing
x-dl <public-tweet-url> — regression test, public tweets still work
x-dl cdp <private-tweet-url> — CDP mode with Chrome remote debugging enabled
x-dl cdp <url> with Chrome not running — auto-launch flow
x-dl --help — shows CDP MODE section, no deprecated flags

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

devin-ai-integration

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

…ages Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace connectOverCDP (which hangs on Chrome 146) with Playwright's launchPersistentContext using channel: 'chrome' and a dedicated profile at ~/.x-dl-chrome-profile. Remove --port flag, update help text and tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Remove references to --port, chrome://inspect, and remote debugging. Document the new persistent profile flow at ~/.x-dl-chrome-profile. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Twitter blocks login in Playwright-controlled browsers. Instead, launch real Chrome (not Playwright) for the login flow using the same profile dir. User logs in normally, closes Chrome, then Playwright picks up the persisted cookies headlessly for extraction. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Opens real Chrome with the dedicated profile dir so the user can log in without Playwright automation detection. Session persists for cdp mode. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Real Chrome and Playwright don't share cookie storage, so using real Chrome for login and Playwright for extraction doesn't work. Instead, use Playwright for both but with --disable-blink-features=AutomationControlled and navigator.webdriver override to avoid Twitter's automation detection. Login subcommand now uses Playwright headed mode. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

devin-ai-integration

Devin Review found 2 new potential issues.

View 7 additional findings in Devin Review.

devin-ai-integration · 2026-03-27T21:27:50Z

src/index.ts

+  const connection = await launchPrivateBrowser({ headed: true });
+
+  await connection.page.goto('https://x.com/i/flow/login', {
+    waitUntil: 'domcontentloaded',
+    timeout: 30000,
+  });
+
+  // Poll for auth_token cookie (5 min timeout)
+  const start = Date.now();
+  const timeoutMs = 300000;
+  while (Date.now() - start < timeoutMs) {
+    const cookies = await connection.context.cookies('https://x.com');
+    if (cookies.some(c => c.name === 'auth_token')) break;
+    await new Promise(r => setTimeout(r, 2000));
+  }
+
+  const cookies = await connection.context.cookies('https://x.com');
+  await connection.cleanup();
+
+  if (!cookies.some(c => c.name === 'auth_token')) {
+    console.error('❌ Login timed out (5 minutes). Please try again.\n');
+    process.exit(1);
+  }
+
+  console.log('✅ Login successful! You can now use: x-dl cdp <url>\n');


🔴 process.exit(1) inside try/finally bypasses async browser cleanup in handleLoginMode

In handleLoginMode, when login times out, process.exit(1) is called at line 604 inside a try...finally block. The finally block (line 608-610) contains await connection.cleanup(), which closes the Chrome browser process. Because process.exit() terminates the process synchronously — before the async await in finally can complete — the Chrome browser process is left running as an orphan.

The correct pattern is already used in handleCdpMode (src/index.ts:380-533): use an exitCode variable with return to trigger the finally block naturally, then call process.exit() after cleanup completes.

Was this helpful? React with 👍 or 👎 to provide feedback.

Good catch — fixed in 02b98fd. handleLoginMode now wraps everything after launchPrivateBrowser() in a try/finally so connection.cleanup() always runs.

src/index.ts

Replace process.exit() calls inside try blocks with return + exitCode pattern so the finally block always runs connection.cleanup(). Also wrap handleLoginMode in try/finally. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

RichardBray · 2026-03-28T15:56:45Z

src/extractor.ts

  }

-  async extract(url: string): Promise<ExtractResult> {
+  async extract(url: string, externalPage?: Page): Promise<ExtractResult> {


what is externalPage used for?

externalPage allows CDP mode (x-dl cdp) to pass in a Playwright Page that's already connected to a logged-in Chrome session. When provided, the extractor reuses that page instead of launching its own browser. This is how private tweets work — the caller manages the browser lifecycle (login, cookies) and the extractor just navigates and intercepts video URLs on the pre-authenticated page.

For the normal public flow (x-dl <url>), externalPage is undefined and the extractor creates/closes its own browser as before.

Renamed to authenticatedPage in 56a3a97 — makes the intent clearer.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Richard Oliver Bray and others added 8 commits March 27, 2026 19:59

refactor: remove deprecated auth types and deprecation tests

4db0ba0

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

refactor: remove isPrivateTweet and auth cookie utils

456749b

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

refactor: remove auth methods from VideoExtractor, accept external page

7561dfd

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

refactor: remove --profile, --login, --verify-auth CLI flags

f59d9b3

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

feat: add CDP module with Chrome detection, connection, and login flow

61e2ac9

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

feat: add x-dl cdp subcommand for private tweet downloads

3b69a08

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

test: add CDP subcommand CLI tests, remove old auth flag tests

a9684f1

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

docs: update README with CDP mode, remove deprecated auth docs

02b419c

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

devin-ai-integration bot reviewed Mar 27, 2026

View reviewed changes

Richard Oliver Bray and others added 7 commits March 27, 2026 20:32

fix: handle Chrome running without remote debugging, clean error mess…

a1c30fc

…ages Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

docs: update README for persistent context approach

086b26e

Remove references to --port, chrome://inspect, and remote debugging. Document the new persistent profile flow at ~/.x-dl-chrome-profile. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

feat: add x-dl login subcommand to open Chrome for X/Twitter login

e149e24

Opens real Chrome with the dedicated profile dir so the user can log in without Playwright automation detection. Session persists for cdp mode. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

docs: add login subcommand to README

b57eb15

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

devin-ai-integration bot reviewed Mar 27, 2026

View reviewed changes

RichardBray commented Mar 28, 2026

View reviewed changes

refactor: rename externalPage to authenticatedPage for clarity

56a3a97

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

RichardBray merged commit d8e32e7 into main Mar 28, 2026
1 check passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: replace Playwright auth with CDP for private tweets#16

feat: replace Playwright auth with CDP for private tweets#16
RichardBray merged 17 commits intomainfrom
feature/cdp-private-tweets

RichardBray commented Mar 27, 2026 •

edited by devin-ai-integration bot

Loading

Uh oh!

devin-ai-integration bot left a comment

Uh oh!

devin-ai-integration bot left a comment

Uh oh!

devin-ai-integration bot Mar 27, 2026 •

edited

Loading

Uh oh!

RichardBray Mar 27, 2026

Uh oh!

Uh oh!

RichardBray Mar 28, 2026

Uh oh!

RichardBray Mar 28, 2026

Uh oh!

RichardBray Mar 28, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

RichardBray commented Mar 27, 2026 • edited by devin-ai-integration bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

devin-ai-integration bot left a comment

Choose a reason for hiding this comment

✅ Devin Review: No Issues Found

Uh oh!

devin-ai-integration bot left a comment

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration bot Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RichardBray Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

RichardBray Mar 28, 2026

Choose a reason for hiding this comment

Uh oh!

RichardBray Mar 28, 2026

Choose a reason for hiding this comment

Uh oh!

RichardBray Mar 28, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

RichardBray commented Mar 27, 2026 •

edited by devin-ai-integration bot

Loading

devin-ai-integration bot Mar 27, 2026 •

edited

Loading