agentic-seo

Audit your documentation and website for Agentic Engine Optimization (AEO): ensure your content is discoverable, parseable, and useful to AI coding agents.

AI coding agents like Claude Code, Cursor, Cline, and Aider consume documentation fundamentally differently from humans. They issue single HTTP requests, strip HTML, count tokens, and either use your content as context or silently discard it. agentic-seo checks whether your documentation is ready for this new reality.

Quick Start

# Audit current directory (auto-detects framework)
npx agentic-seo

# Audit a specific directory
npx agentic-seo ./my-docs-site

# Audit a live URL
npx agentic-seo --url https://docs.example.com

# Scaffold missing AEO files
npx agentic-seo init

What It Checks

agentic-seo runs 10 checks across 5 categories, scoring your site out of 100:

Discovery (25 points)

Check	Points	What it looks for
robots-txt	10	AI crawlers not blocked, explicit allow rules
llms-txt	10	Structured index with descriptions and token counts
agents-md	5	AGENTS.md/CLAUDE.md with project context

Content Structure (25 points)

Check	Points	What it looks for
content-structure	15	Heading hierarchy, semantic HTML, code examples, tables
markdown-availability	10	Markdown source available, low HTML noise, no JS dependency

Token Economics (25 points)

Check	Points	What it looks for
token-budget	15	Per-page token counts, no oversized pages
meta-tags	10	AI-friendly meta tags, descriptions, token count metadata

Capability Signaling (15 points)

Check	Points	What it looks for
skill-md	10	Capability descriptions, inputs, constraints
agent-permissions	5	Agent access rules and rate limits

UX Bridge (10 points)

Check	Points	What it looks for
copy-for-ai	10	Copy-for-AI buttons, copy-to-clipboard, raw view links

Grading

Grade	Score	Meaning
A	90-100%	Excellent agent readiness
B	75-89%	Good with minor improvements needed
C	60-74%	Functional but missing key signals
D	40-59%	Significant gaps in agent readiness
F	0-39%	Not optimized for AI agents

Installation

# Global install
npm install -g agentic-seo

# Or use npx (no install needed)
npx agentic-seo

Usage

Audit a Local Directory

# Auto-detects framework (Next.js, Docusaurus, 11ty, Astro, Hugo, etc.)
agentic-seo ./my-project

# Explicitly specify build output directory
agentic-seo --output-dir ./my-project/build

Supported frameworks: Next.js, Docusaurus, Eleventy, Astro, Hugo, Jekyll, Gatsby, VitePress, MkDocs, Sphinx, Vite.

Audit with Local Server

# Spin up a server and run HTTP-based checks
agentic-seo --serve ./build

Audit a Live URL

agentic-seo --url https://docs.example.com

CI Mode

# Exit with code 1 if score below threshold
agentic-seo --json --threshold 60

# Just the score
agentic-seo score --json

Scaffold AEO Files

# Create missing llms.txt, AGENTS.md, skill.md, agent-permissions.json
agentic-seo init

Run Specific Checks

agentic-seo --checks robots-txt,llms-txt,token-budget

Configuration

Create .aeorc.json in your project root, or add an "aeo" key to package.json:

{
  "outputDir": "_site",
  "checks": {
    "token-budget": { "maxTokensPerPage": 25000 },
    "robots-txt": { "requiredAgents": ["ClaudeBot", "GPTBot"] }
  },
  "ignore": ["**/node_modules/**", "**/vendor/**"],
  "threshold": 60
}

Programmatic API

import { audit, auditWithServer } from 'agentic-seo';

// Audit a directory
const report = await audit('./my-site');
console.log(report.grade);      // 'B'
console.log(report.percentage);  // 78
console.log(report.findings.errors); // [{ severity: 'error', message: '...' }]

// Audit with a local server
const report2 = await auditWithServer('./build');

// Access individual checkers
import { checkers } from 'agentic-seo';

How It Works

agentic-seo does not require an API key. All checks are structural and heuristic:

1. Framework Detection: Auto-detects your build tool and locates the output directory
2. File Analysis: Checks for robots.txt, llms.txt, AGENTS.md, skill.md, and agent-permissions.json
3. Content Analysis: Parses HTML with Cheerio and Markdown with Remark to evaluate structure
4. Token Counting: Uses gpt-tokenizer to measure real token counts per page
5. Agent Simulation: Evaluates what AI agents would actually see when fetching your pages

CLI Options

Options:
  --url, -u         Audit a live URL
  --serve, -s       Start local server and audit via HTTP
  --json            Output results as JSON
  --verbose, -v     Show all findings including info messages
  --threshold, -t   Minimum score percentage (exit 1 if below)
  --checks          Comma-separated checker IDs to run
  --output-dir      Explicitly specify build output directory
  --help            Show help
  --version         Show version

Where to Start

If you're new to AEO, here's the recommended priority order:

1. Audit robots.txt: Prevents silent agent lockout (10 min)
2. Add llms.txt: Immediate discoverability gains (a few hours)
3. Measure token counts: High-leverage insight into page sizes (weekend project)
4. Write skill.md: Start with your most-used APIs
5. Add "Copy for AI" buttons: Low effort, high signal for human+agent workflows
6. Set up AI traffic monitoring: Gives you data to justify further investment

Disclaimer

This is not an official Google project and is not endorsed by Google or Google Search. The concepts, checks, and recommendations in this tool are a best-effort community offering based on publicly available research and emerging practices around AI agent documentation consumption. Scores and recommendations should be taken as directional guidance, not as guarantees of any particular outcome.

License

MIT