Skip to content

feat(cli): record browser agent video#1337

Open
icycodes wants to merge 6 commits intomainfrom
feat/cli-browser-agent-video
Open

feat(cli): record browser agent video#1337
icycodes wants to merge 6 commits intomainfrom
feat/cli-browser-agent-video

Conversation

@icycodes
Copy link
Member

@icycodes icycodes commented Mar 6, 2026

Summary

  • Added support for recording browser agent sessions to MP4 video files in the CLI.
  • Introduced fluent-ffmpeg for video conversion from MJPEG frames.
  • Implemented real-time MJPEG stream processing via WebSocket in the browser agent subtask.

Test plan

  • Run Pochi and use the browser agent.
  • Verify that a .mp4 file is generated in the specified directory.
  • Check if the video correctly captures the browser session steps.

🤖 Generated with Pochi | Task

@icycodes icycodes marked this pull request as draft March 6, 2026 12:28
Base automatically changed from fix/cli-improve-output to main March 9, 2026 15:44
@icycodes icycodes force-pushed the feat/cli-browser-agent-video branch from 43ba767 to c6441b3 Compare March 10, 2026 03:20
@codecov-commenter
Copy link

codecov-commenter commented Mar 10, 2026

Codecov Report

❌ Patch coverage is 5.85586% with 209 lines in your changes missing coverage. Please review.
✅ Project coverage is 18.11%. Comparing base (5e5a10f) to head (d7f873a).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
packages/cli/src/lib/ffmpeg-mjpeg-to-mp4.ts 5.30% 125 Missing ⚠️
packages/cli/src/tools/new-task.ts 8.95% 61 Missing ⚠️
...ckages/common/src/browser/browser-session-store.ts 0.00% 12 Missing ⚠️
packages/cli/src/cli.ts 0.00% 10 Missing ⚠️
packages/cli/src/task-runner.ts 0.00% 1 Missing ⚠️

❌ Your patch status has failed because the patch coverage (5.85%) is below the target coverage (75.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1337      +/-   ##
==========================================
+ Coverage   13.94%   18.11%   +4.17%     
==========================================
  Files         683      687       +4     
  Lines       62913    63582     +669     
  Branches     1748     1483     -265     
==========================================
+ Hits         8774    11520    +2746     
+ Misses      53858    51787    -2071     
+ Partials      281      275       -6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@icycodes icycodes marked this pull request as ready for review March 10, 2026 08:54
@icycodes icycodes requested a review from Copilot March 10, 2026 08:54
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds CLI-side support for recording browser-agent sessions by consuming the browser agent’s WebSocket MJPEG frame stream and converting it into an MP4 via fluent-ffmpeg.

Changes:

  • Extend browser session registration to optionally enable a local WebSocket stream outside VSCode.
  • Add an MJPEG→MP4 converter module using fluent-ffmpeg, and wire recording/finalization into the newTask tool.
  • Introduce new CLI plumbing (blob store in tool options, --ffmpeg flag) and add required dependencies.

Reviewed changes

Copilot reviewed 8 out of 9 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
packages/common/src/browser/browser-session-store.ts Allows enabling browser stream when not in VSCode; stream URL handling updated.
packages/cli/src/types.ts Adds blobStore to ToolCallOptions so tools can persist generated binaries.
packages/cli/src/tools/new-task.ts Connects to the browser stream, collects frames, converts to MP4, and stores a URL reference in the task file system.
packages/cli/src/tools/tests/background-job-integration.test.ts Updates test setup to provide blobStore in tool options.
packages/cli/src/task-runner.ts Passes blobStore into tool call options.
packages/cli/src/lib/ffmpeg-mjpeg-to-mp4.ts New converter implementation based on ffmpeg stdin MJPEG piping and timestamp-based frame repetition.
packages/cli/src/cli.ts Adds --ffmpeg <path> option to configure ffmpeg binary location.
packages/cli/package.json Adds ffmpeg + websocket dependencies required for recording.
bun.lock Locks newly added dependencies (including fluent-ffmpeg transitive deps).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

@wsxiaoys wsxiaoys requested a review from zhanba March 11, 2026 02:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants