Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,23 @@
"version": "2.0.0"
},
"plugins": [
{
"name": "apify-agent-onboarding",
"source": "./skills/apify-agent-onboarding",
"skills": "./",
"description": "Set up Apify for any use case. Routes to the correct installation path based on what the user wants to do: MCP server, Agent Skills + CLI, or API client.",
"keywords": [
"apify",
"onboarding",
"setup",
"mcp",
"agent",
"installation",
"getting-started"
],
"category": "agent-onboarding",
"version": "2.0.0"
},
{
"name": "apify-ultimate-scraper",
"source": "./skills/apify-ultimate-scraper",
Expand Down Expand Up @@ -88,4 +105,4 @@
]
}
]
}
}
2 changes: 2 additions & 0 deletions agents/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ You have additional SKILLs documented in directories containing a "SKILL.md" fil
These skills are:
- apify-actor-development -> "skills/apify-actor-development/SKILL.md"
- apify-actorization -> "skills/apify-actorization/SKILL.md"
- apify-agent-onboarding -> "skills/apify-agent-onboarding/SKILL.md"
- apify-generate-output-schema -> "skills/apify-generate-output-schema/SKILL.md"
- apify-ultimate-scraper -> "skills/apify-ultimate-scraper/SKILL.md"

Expand All @@ -14,6 +15,7 @@ IMPORTANT: You MUST read the SKILL.md file whenever the description of the skill

apify-actor-development: `Develop, debug, and deploy Apify Actors - serverless cloud programs for web scraping, automation, and data processing. Use when creating new Actors, modifying existing ones, or troubleshooting Actor code.`
apify-actorization: `Convert existing projects into Apify Actors - serverless cloud programs. Actorize JavaScript/TypeScript (SDK with Actor.init/exit), Python (async context manager), or any language (CLI wrapper). Use when migrating code to Apify, wrapping CLI tools as Actors, or adding Actor SDK to existing projects.`
apify-agent-onboarding: `Set up Apify for any use case. Routes to the correct installation path based on what the user wants to do - MCP server, Agent Skills + CLI, or API client. Triggers on "install apify", "set up apify", "use apify", "connect apify", "apify getting started", "which apify package", "apify MCP". Do NOT use when Apify is already configured and the user wants to run a specific Actor (use apify-ultimate-scraper) or build an Actor (use apify-actor-development).`
apify-generate-output-schema: `Generate output schemas (dataset_schema.json, output_schema.json, key_value_store_schema.json) for an Apify Actor by analyzing its source code. Use when creating or updating Actor output schemas.`
apify-ultimate-scraper: `Universal AI-powered web scraper for any platform. Scrape data from Instagram, Facebook, TikTok, YouTube, Google Maps, Google Search, Google Trends, Booking.com, and TripAdvisor. Use for lead generation, brand monitoring, competitor analysis, influencer discovery, trend research, content analytics, audience analysis, or any data extraction task.`
</available_skills>
Expand Down
156 changes: 156 additions & 0 deletions skills/apify-agent-onboarding/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
---
name: apify-agent-onboarding
description: Set up Apify for any use case. Routes to the correct installation path based on what the user wants to do - MCP server, Agent Skills + CLI, or API client. Triggers on "install apify", "set up apify", "use apify", "connect apify", "apify getting started", "which apify package", "apify MCP". Do NOT use when Apify is already configured and the user wants to run a specific Actor (use apify-ultimate-scraper) or build an Actor (use apify-actor-development).
---

# Apify agent onboarding

You are an AI agent setting up Apify for your user. Apify is a platform with thousands of serverless cloud tools called **Actors** for web scraping, browser automation, and data extraction.

**Before installing anything, determine what your user needs:**

| Signal | Route |
|--------|-------|
| Universal catch-all solution ideal for research and exploration | **Path A: MCP server** (Apify Store search, small scale ad-hoc scraping, and documentation access) |
| User wants to scrape web data with multi-step agentic workflows | **Path B: Agent Skills** (ultimate-scraper) |
| User wants to build, test, or deploy a custom Actor | **Path B: Agent Skills + CLI** (actor-development, actorization, generate-output-schema) |
| User wants to add Apify to an existing JS/Python app | **Path C: API integration** |
| Ambiguous | Ask: "Do you want to (a) extract data from websites, (b) build and deploy a custom scraper, or (c) add web scraping to an existing app?" |

> **NAMING TRAP:** The `apify` npm package is the SDK for *building* Actors. The `apify-client` package is the API client for *calling* Actors. If your user wants to call Actors from their app, use `apify-client`.

---

## Path A: MCP server

Connect to Apify via the Model Context Protocol. No local install needed for remote-capable clients.

### Exploration mode (no account needed)

These tools work **without authentication** - use them freely for research and Actor discovery:
- `search-actors` - find Actors by keyword
- `fetch-actor-details` - get Actor specs, input schema, pricing
- `search-apify-docs` / `fetch-apify-docs` - browse documentation

### Setup

**Remote (streamable HTTP):**
```json
{
"mcpServers": {
"apify": {
"url": "https://mcp.apify.com"
}
}
}
```
Clients that support remote MCP (Claude Code, Cursor, VS Code, Copilot) use OAuth - your user will be prompted to sign in on first use.

**Local (stdio):**
```json
{
"mcpServers": {
"apify": {
"command": "npx",
"args": ["-y", "@apify/actors-mcp-server"],
"env": { "APIFY_TOKEN": "<token>" }
}
}
}
```
Requires an API token. Direct your user to **Console > Settings > Integrations** at https://console.apify.com/settings/integrations to create one. If they don't have an account, open https://console.apify.com/sign-up in their browser - sign-up is free, no credit card needed.

For client-specific configuration, direct your user to https://mcp.apify.com which generates ready-to-paste configs for all supported clients.

### Verify

Call `search-actors` with query `"web scraper"`. If results return, the connection works.

---

## Path B: Agent Skills

Install pre-built skill workflows for data extraction and Actor development.

```bash
npx skills add apify/agent-skills
```


**Available skills**

| Skill | Purpose |
|-------|---------|
| `apify-ultimate-scraper` | Web scraping with multi-step agentic workflows |
| `apify-actor-development` | Full Actor build/test/deploy lifecycle |
| `apify-actorization` | Convert an existing project into an Actor |
| `apify-generate-output-schema` | Auto-generate output schemas from source code |


### For data extraction

Use the `apify-ultimate-scraper` skill for scraping web data via agentic workflows. Create complex multi-step data pipelines with for use cases like lead generation, deep research, social media monitoring, ecommerce intelligence etc.

**Requires:** `APIFY_TOKEN` environment variable. Ask your user for their API token, or direct them to https://console.apify.com/settings/integrations to create one. If they don't have an account, open https://console.apify.com/sign-up (free, no credit card).

### For Actor development

Use the `apify-actor-development` or `apify-actorization` skills in conjuction with `apify-generate-output-schema` skill for a guided workflow covering scaffolding, testing, and deployment of Actors on the Apify platform.

**Requires Apify CLI:**
```bash
npm install -g apify-cli
apify login
```

**Workflow:** `apify create my-actor` > develop in `src/` > `apify run` (test locally) > `apify push` (deploy to cloud).

---

## Path C: API integration

For adding Apify to an existing application. Install `apify-client` (NOT `apify`).

**JavaScript/TypeScript:**
```bash
npm install apify-client
```
```typescript
import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('apify/web-scraper').call({ startUrls: [{ url: 'https://example.com' }] });
const { items } = await client.dataset(run.defaultDatasetId).listItems();
```

**Python:**
```bash
pip install apify-client
```
```python
from apify_client import ApifyClient
client = ApifyClient(token=os.environ['APIFY_TOKEN'])
run = client.actor('apify/web-scraper').call(run_input={'startUrls': [{'url': 'https://example.com'}]})
items = client.dataset(run['defaultDatasetId']).list_items().items
```

**REST API** (any language): `POST https://api.apify.com/v2/acts/{actorId}/runs` with `Authorization: Bearer <APIFY_TOKEN>`. Full reference: https://docs.apify.com/api/v2

### Documentation context for agentic development
If you need Apify documentation while building an integration, consider configuring the MCP server to access docs tools: `search-apify-docs`, `fetch-apify-docs`

Alternatively refer to following resources:
- Docs index: https://docs.apify.com/llms.txt
- Full docs: https://docs.apify.com/llms-full.txt
- Any Actor details in clean markdown: append `.md` to its Apify Store URL

---

## Quick reference

| I want to... | Path | Install | Auth |
|---|---|---|---|
| Use Apify platform with my AI agent | MCP server | None (remote) | OAuth |
| Run multi-step scraping workflows | Agent Skills | `npx skills add apify/agent-skills` | `APIFY_TOKEN` |
| Build and deploy a custom Actor | Agent Skills + CLI | `npm i -g apify-cli` | `apify login` |
| Call Actors from Node.js/Python | API client | `npm i apify-client` / `pip install apify-client` | `APIFY_TOKEN` |
| Call Actors from any language | REST API | None | Bearer token |
Loading