Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
120 changes: 120 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ A CLI tool that turns **any website** into a command-line interface — Bilibili
- [Built-in Commands](#built-in-commands)
- [Output Formats](#output-formats)
- [For AI Agents (Developer Guide)](#for-ai-agents-developer-guide)
- [Remote Chrome (Server/Headless)](#remote-chrome-serverheadless)
- [Testing](#testing)
- [Troubleshooting](#troubleshooting)
- [Releasing New Versions](#releasing-new-versions)
Expand Down Expand Up @@ -197,6 +198,125 @@ opencli cascade https://api.example.com/data

Explore outputs to `.opencli/explore/<site>/` (manifest.json, endpoints.json, capabilities.json, auth.json).

## Remote Chrome (Server/Headless)

For server environments without a display, connect OpenCLI to a Chrome browser running on your local machine via Chrome DevTools Protocol (CDP). Two methods are available:

| Method | Chrome restart? | Chrome version | Endpoint format |
|--------|:-:|:-:|:-:|
| **A. Chrome 144+ Auto-Discovery** | No | ≥ 144 | `ws://` |
| **B. Classic `--remote-debugging-port`** | Yes | Any | `http://` |

---

### Method A: Chrome 144+ (No Restart Required)

Use your **already-running Chrome** — no command-line flags needed.

**Step 1 — Enable remote debugging in Chrome**

Navigate to `chrome://inspect#remote-debugging` and check "Allow remote debugging".

**Step 2 — Get the WebSocket URL**

Read Chrome's `DevToolsActivePort` file to get the port and browser GUID:

```bash
# macOS (Chrome)
cat ~/Library/Application\ Support/Google/Chrome/DevToolsActivePort

# macOS (Edge)
cat ~/Library/Application\ Support/Microsoft\ Edge/DevToolsActivePort

# Linux (Chrome)
cat ~/.config/google-chrome/DevToolsActivePort

# Linux (Chromium)
cat ~/.config/chromium/DevToolsActivePort
```

```cmd
:: Windows (Chrome)
type "%LOCALAPPDATA%\Google\Chrome\User Data\DevToolsActivePort"

:: Windows (Edge)
type "%LOCALAPPDATA%\Microsoft\Edge\User Data\DevToolsActivePort"
```

Output:
```
61882
/devtools/browser/9f395fbe-24cb-4075-b58f-dd1c4f6eb172
```

**Step 3 — SSH tunnel + run OpenCLI**

```bash
# On your local machine — forward the port to your server
ssh -R 61882:localhost:61882 your-server

# On the server
export OPENCLI_CDP_ENDPOINT="ws://localhost:61882/devtools/browser/9f395fbe-..."
opencli doctor # Verify connection
opencli bilibili hot --limit 5 # Test a command
```

> **Same-machine shortcut**: If Chrome and OpenCLI run on the same machine, auto-discovery reads `DevToolsActivePort` automatically — no env var needed, or set `OPENCLI_CDP_ENDPOINT=1` to force it.

> **Note**: The port and GUID change every time Chrome restarts or re-enables remote debugging. You'll need to re-read `DevToolsActivePort` and update the env var.

---

### Method B: Classic `--remote-debugging-port` (Any Chrome Version)

Requires restarting Chrome with a flag, but works with any Chrome version and provides a stable HTTP endpoint.

**Step 1 — Start Chrome with remote debugging**

```bash
# macOS
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome \
--remote-debugging-port=9222

# Linux
google-chrome --remote-debugging-port=9222

# Windows
"C:\Program Files\Google\Chrome\Application\chrome.exe" ^
--remote-debugging-port=9222
```

**Step 2 — Log into target websites** in that Chrome window.

**Step 3 — SSH tunnel + run OpenCLI**

```bash
# On your local machine
ssh -R 9222:localhost:9222 your-server

# On the server
export OPENCLI_CDP_ENDPOINT="http://localhost:9222"
opencli doctor # Verify connection
opencli bilibili hot --limit 5 # Test a command
```

---

### Environment Variables

| Variable | Description | Example |
|----------|-------------|---------|
| `OPENCLI_CDP_ENDPOINT` | CDP endpoint URL (`ws://` or `http://`), or `1` to force auto-discovery | `ws://localhost:61882/devtools/browser/...` |
| `CHROME_USER_DATA_DIR` | Override Chrome user data directory for `DevToolsActivePort` discovery | `/home/user/.config/google-chrome` |

### Persistent Configuration

Add to your shell profile (`~/.bashrc` or `~/.zshrc`):

```bash
export OPENCLI_CDP_ENDPOINT="ws://localhost:61882/devtools/browser/..."
```

## Testing

See **[TESTING.md](./TESTING.md)** for the full testing guide, including:
Expand Down
120 changes: 120 additions & 0 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ OpenCLI 将任何网站变成命令行工具 — B站、知乎、小红书、Twi
- [内置命令](#内置命令)
- [输出格式](#输出格式)
- [致 AI Agent(开发者指南)](#致-ai-agent开发者指南)
- [远程 Chrome(服务器/无头环境)](#远程-chrome服务器无头环境)
- [常见问题排查](#常见问题排查)
- [版本发布](#版本发布)
- [License](#license)
Expand Down Expand Up @@ -196,6 +197,125 @@ opencli cascade https://api.example.com/data

探索结果输出到 `.opencli/explore/<site>/`。

## 远程 Chrome(服务器/无头环境)

在服务器(无显示器)环境中,通过 Chrome DevTools Protocol (CDP) 连接到本地电脑上运行的 Chrome。支持两种方式:

| 方式 | 需重启 Chrome? | Chrome 版本 | 端点格式 |
|------|:-:|:-:|:-:|
| **A. Chrome 144+ 自动发现** | 否 | ≥ 144 | `ws://` |
| **B. 经典 `--remote-debugging-port`** | 是 | 任意 | `http://` |

---

### 方式 A:Chrome 144+(无需重启)

直接复用**已运行的 Chrome**,不需要任何命令行参数。

**第一步 — 在 Chrome 中开启远程调试**

打开 `chrome://inspect#remote-debugging`,勾选"允许远程调试"。

**第二步 — 获取 WebSocket URL**

读取 Chrome 的 `DevToolsActivePort` 文件获取端口和浏览器 GUID:

```bash
# macOS (Chrome)
cat ~/Library/Application\ Support/Google/Chrome/DevToolsActivePort

# macOS (Edge)
cat ~/Library/Application\ Support/Microsoft\ Edge/DevToolsActivePort

# Linux (Chrome)
cat ~/.config/google-chrome/DevToolsActivePort

# Linux (Chromium)
cat ~/.config/chromium/DevToolsActivePort
```

```cmd
:: Windows (Chrome)
type "%LOCALAPPDATA%\Google\Chrome\User Data\DevToolsActivePort"

:: Windows (Edge)
type "%LOCALAPPDATA%\Microsoft\Edge\User Data\DevToolsActivePort"
```

输出示例:
```
61882
/devtools/browser/9f395fbe-24cb-4075-b58f-dd1c4f6eb172
```

**第三步 — SSH 隧道 + 运行 OpenCLI**

```bash
# 本地电脑 — 将端口转发到服务器
ssh -R 61882:localhost:61882 your-server

# 在服务器上
export OPENCLI_CDP_ENDPOINT="ws://localhost:61882/devtools/browser/9f395fbe-..."
opencli doctor # 验证连接
opencli bilibili hot --limit 5 # 测试命令
```

> **同机快捷方式**:如果 Chrome 和 OpenCLI 在同一台机器上运行,auto-discovery 会自动读取 `DevToolsActivePort`,无需设置环境变量,或设置 `OPENCLI_CDP_ENDPOINT=1` 强制启用。

> **注意**:每次 Chrome 重启或重新启用远程调试后,端口和 GUID 会改变,需要重新读取 `DevToolsActivePort` 并更新环境变量。

---

### 方式 B:经典 `--remote-debugging-port`(任意 Chrome 版本)

需要用命令行参数重启 Chrome,但兼容所有 Chrome 版本,且 HTTP 端点稳定不变。

**第一步 — 启动带远程调试的 Chrome**

```bash
# macOS
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome \
--remote-debugging-port=9222

# Linux
google-chrome --remote-debugging-port=9222

# Windows
"C:\Program Files\Google\Chrome\Application\chrome.exe" ^
--remote-debugging-port=9222
```

**第二步 — 登录目标网站**

**第三步 — SSH 隧道 + 运行 OpenCLI**

```bash
# 本地电脑
ssh -R 9222:localhost:9222 your-server

# 在服务器上
export OPENCLI_CDP_ENDPOINT="http://localhost:9222"
opencli doctor # 验证连接
opencli bilibili hot --limit 5 # 测试命令
```

---

### 环境变量

| 变量 | 说明 | 示例 |
|------|------|------|
| `OPENCLI_CDP_ENDPOINT` | CDP 端点 URL(`ws://` 或 `http://`),或 `1` 强制自动发现 | `ws://localhost:61882/devtools/browser/...` |
| `CHROME_USER_DATA_DIR` | 自定义 Chrome 用户数据目录(用于 `DevToolsActivePort` 发现) | `/home/user/.config/google-chrome` |

### 持久化配置

写入 shell 配置文件(`~/.bashrc` 或 `~/.zshrc`):

```bash
export OPENCLI_CDP_ENDPOINT="ws://localhost:61882/devtools/browser/..."
```

## 常见问题排查

- **"Failed to connect to Playwright MCP Bridge"** 报错
Expand Down
33 changes: 33 additions & 0 deletions src/browser.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,39 @@ describe('browser helpers', () => {
}
});

it('builds CDP MCP args when cdpEndpoint is provided', () => {
const savedCI = process.env.CI;
delete process.env.CI;
try {
// CDP mode: --cdp-endpoint takes priority, no --extension
expect(__test__.buildMcpArgs({
mcpPath: '/tmp/cli.js',
cdpEndpoint: 'ws://localhost:9222/devtools/browser/abc-123',
})).toEqual([
'/tmp/cli.js',
'--cdp-endpoint',
'ws://localhost:9222/devtools/browser/abc-123',
]);

// CDP with executable path — executable path is ignored in CDP mode
expect(__test__.buildMcpArgs({
mcpPath: '/tmp/cli.js',
cdpEndpoint: 'http://localhost:9222',
executablePath: '/usr/bin/chromium',
})).toEqual([
'/tmp/cli.js',
'--cdp-endpoint',
'http://localhost:9222',
]);
} finally {
if (savedCI !== undefined) {
process.env.CI = savedCI;
} else {
delete process.env.CI;
}
}
});

it('times out slow promises', async () => {
await expect(__test__.withTimeoutMs(new Promise(() => {}), 10, 'timeout')).rejects.toThrow('timeout');
});
Expand Down
80 changes: 77 additions & 3 deletions src/browser/discover.ts
Original file line number Diff line number Diff line change
Expand Up @@ -75,13 +75,87 @@ export function findMcpServerPath(): string | null {
return _cachedMcpServerPath;
}

export function buildMcpArgs(input: { mcpPath: string; executablePath?: string | null }): string[] {
/**
* Chrome 144+ auto-discovery: read DevToolsActivePort file to get CDP endpoint.
*
* Starting with Chrome 144, users can enable remote debugging from
* chrome://inspect#remote-debugging without any command-line flags.
* Chrome writes the active port and browser GUID to a DevToolsActivePort file
* in the user data directory, which we read to construct the WebSocket endpoint.
*/
export function discoverChromeEndpoint(): string | null {
const candidates: string[] = [];

// User-specified Chrome data dir takes highest priority
if (process.env.CHROME_USER_DATA_DIR) {
candidates.push(path.join(process.env.CHROME_USER_DATA_DIR, 'DevToolsActivePort'));
}

// Standard Chrome/Edge user data dirs per platform
if (process.platform === 'win32') {
const localAppData = process.env.LOCALAPPDATA ?? path.join(os.homedir(), 'AppData', 'Local');
candidates.push(path.join(localAppData, 'Google', 'Chrome', 'User Data', 'DevToolsActivePort'));
candidates.push(path.join(localAppData, 'Microsoft', 'Edge', 'User Data', 'DevToolsActivePort'));
} else if (process.platform === 'darwin') {
candidates.push(path.join(os.homedir(), 'Library', 'Application Support', 'Google', 'Chrome', 'DevToolsActivePort'));
candidates.push(path.join(os.homedir(), 'Library', 'Application Support', 'Microsoft Edge', 'DevToolsActivePort'));
} else {
candidates.push(path.join(os.homedir(), '.config', 'google-chrome', 'DevToolsActivePort'));
candidates.push(path.join(os.homedir(), '.config', 'chromium', 'DevToolsActivePort'));
candidates.push(path.join(os.homedir(), '.config', 'microsoft-edge', 'DevToolsActivePort'));
}

for (const filePath of candidates) {
try {
const content = fs.readFileSync(filePath, 'utf-8').trim();
const lines = content.split('\n');
if (lines.length >= 2) {
const port = parseInt(lines[0], 10);
const browserPath = lines[1]; // e.g. /devtools/browser/<GUID>
if (port > 0 && browserPath.startsWith('/devtools/browser/')) {
return `ws://127.0.0.1:${port}${browserPath}`;
}
}
} catch {}
}
return null;
}

export function resolveCdpEndpoint(): { endpoint?: string; requestedCdp: boolean } {
const envVal = process.env.OPENCLI_CDP_ENDPOINT;
if (envVal === '1' || envVal?.toLowerCase() === 'true') {
const autoDiscovered = discoverChromeEndpoint();
return { endpoint: autoDiscovered ?? envVal, requestedCdp: true };
}

if (envVal) {
return { endpoint: envVal, requestedCdp: true };
}

// Fallback to auto-discovery if not explicitly set
const autoDiscovered = discoverChromeEndpoint();
if (autoDiscovered) {
return { endpoint: autoDiscovered, requestedCdp: true };
}

return { requestedCdp: false };
}

export function buildMcpArgs(input: { mcpPath: string; executablePath?: string | null; cdpEndpoint?: string }): string[] {
const args = [input.mcpPath];

// Priority 1: CDP endpoint (remote Chrome debugging or local Auto-Discovery)
if (input.cdpEndpoint) {
args.push('--cdp-endpoint', input.cdpEndpoint);
return args;
}

// Priority 2: Extension mode (local Chrome with MCP Bridge extension)
if (!process.env.CI) {
// Local: always connect to user's running Chrome via MCP Bridge extension
args.push('--extension');
}
// CI: standalone mode — @playwright/mcp launches its own browser (headed by default).

// CI/standalone mode: @playwright/mcp launches its own browser (headed by default).
// xvfb provides a virtual display for headed mode in GitHub Actions.
if (input.executablePath) {
args.push('--executable-path', input.executablePath);
Expand Down
Loading