fix(intercept): IIFE wrapping for installInterceptor/getInterceptedRequests#100
Conversation
…r/getInterceptedRequests
Root cause: daemon migration changed these methods from this.evaluate()
to direct sendCommand('exec'), losing the wrapForEval() IIFE wrapping.
CDP received bare arrow functions that were never invoked.
Fixes jackwener#98
…TERCEPT commands - followers/following: install interceptor on profile page, then click followers/following link (SPA navigation preserves JS context). Use JSON.stringify for targetUser to prevent injection. Throw on navigation failure. Update selector: /verified_followers. - notifications: install interceptor on home, then pushState+popstate to /notifications. Validate navigation URL. - search: fix author resolution (core.screen_name, not legacy). - All: fix GraphQL data path (remove extra .data level), update author resolution to try core.screen_name before legacy.screen_name. - followers: remove erroneous .filter(r => r?.url) — interceptor stores response body JSON, URL filtering happens at capture time.
8b11115 to
f9d304d
Compare
|
Thanks for the contribution. This was a very helpful fix. Restoring the interceptor path in page.ts addresses the actual regression from the daemon migration, and the follow-up Twitter adapter updates make the INTERCEPT commands usable again on the current X frontend. I especially appreciated the careful notes in the PR description and the extra hardening around SPA navigation and JSON.stringify for injected values. |
|
Thank you! Glad the investigation into the IIFE wrapping root cause was useful. Debugging the SPA navigation was a fun puzzle — the Note: the upstream Happy to help with more adapters if needed! |
Summary
Fixes #98 — INTERCEPT strategy commands fail to capture data.
1. Interceptor pipeline fix (
page.ts)Root cause:
installInterceptor()andgetInterceptedRequests()were changed fromthis.evaluate()to directsendCommand('exec')during the daemon migration. This bypassedwrapForEval(), so CDP received bare arrow functions that were defined but never invoked.Fix: restore
this.evaluate()for both methods, sowrapForEval()wraps the arrow function as an IIFE(code)().2. Twitter adapter fixes (
followers,following,search,notifications)With the interceptor pipeline working, discovered three additional issues preventing data capture:
page.goto()resets JS context, losing the installed interceptor. Changed to install interceptor first, then navigate via SPA (click for followers/following,pushState+popstatefor search/notifications) to preserve JS context.{ data: { user: ... } }, not{ data: { data: { user: ... } } }. Removed the extra.datalevel.screen_namefromlegacytocore. Updated to trycore.screen_namefirst withlegacyfallback./followersto/verified_followers.3. Security hardening
JSON.stringifyfor all user input interpolated intoevaluate()JS stringsAlso removed erroneous
.filter(r => r?.url?.includes('Followers'))infollowers.ts— interceptor stores response body JSON (nourlfield), pattern filtering happens at capture time.Test plan
npm run typecheckpassesnpx vitest run src/— 213 unit tests passtwitter followers --user elonmusk --limit 5— returns datatwitter following --user elonmusk --limit 5— returns datatwitter search --query "opencli" --limit 5— returns data with correct authortwitter search --query "it's a test"— single quote in query handled safelytwitter notifications --limit 5— returns data