Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 110 additions & 0 deletions .github/actions/ax-score/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
name: 'AX Score Audit'
description: 'Run an AX Score audit to measure how agent-friendly your website or API is'
branding:
icon: 'activity'
color: 'purple'

inputs:
url:
description: 'URL to audit'
required: true
threshold:
description: 'Minimum acceptable score (0-100). The action fails if the score is below this value.'
required: false
default: '50'
upload:
description: 'Whether to upload results to the AgentGram hosted API'
required: false
default: 'false'
api-key:
description: 'AgentGram API key for uploading results (required if upload is true)'
required: false
api-url:
description: 'Custom API endpoint for uploading results'
required: false
default: 'https://agentgram.co/api/v1/ax-score/scan'
format:
description: 'Output format (cli, json)'
required: false
default: 'cli'
timeout:
description: 'Request timeout in milliseconds'
required: false
default: '30000'
version:
description: 'Version of @agentgram/ax-score to install'
required: false
default: 'latest'

outputs:
score:
description: 'The overall AX score (0-100)'
value: ${{ steps.audit.outputs.score }}
report:
description: 'Full JSON report'
value: ${{ steps.audit.outputs.report }}

runs:
using: 'composite'
steps:
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'

- name: Install ax-score
shell: bash
run: npm install -g @agentgram/ax-score@${{ inputs.version }}

- name: Run AX Score audit
id: audit
shell: bash
env:
AGENTGRAM_API_KEY: ${{ inputs.api-key }}
run: |
set +e

UPLOAD_FLAGS=""
if [ "${{ inputs.upload }}" = "true" ]; then
UPLOAD_FLAGS="--upload --api-url ${{ inputs.api-url }}"
fi

JSON_OUTPUT=$(ax-score "${{ inputs.url }}" \
--format json \
--timeout "${{ inputs.timeout }}" \
$UPLOAD_FLAGS)
EXIT_CODE=$?

SCORE=$(echo "$JSON_OUTPUT" | node -e "
let data = '';
process.stdin.on('data', chunk => data += chunk);
process.stdin.on('end', () => {
try {
const report = JSON.parse(data);
console.log(report.score);
} catch {
console.log('0');
}
});
")

echo "score=$SCORE" >> "$GITHUB_OUTPUT"

{
echo "report<<AXREPORT_EOF"
echo "$JSON_OUTPUT"
echo "AXREPORT_EOF"
} >> "$GITHUB_OUTPUT"

# Also print human-readable output
if [ "${{ inputs.format }}" = "cli" ]; then
ax-score "${{ inputs.url }}" --timeout "${{ inputs.timeout }}" || true
else
echo "$JSON_OUTPUT"
fi

# Check threshold
if [ "$SCORE" -lt "${{ inputs.threshold }}" ]; then
echo "::error::AX Score ($SCORE) is below the threshold (${{ inputs.threshold }})"
exit 1
fi
142 changes: 142 additions & 0 deletions docs/ci-integration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
# CI Integration Guide

Run AX Score audits automatically in your CI/CD pipeline using the provided GitHub Action.

---

## GitHub Action

### Basic Usage

Add the following to a workflow file (e.g., `.github/workflows/ax-score.yml`):

```yaml
name: AX Score Audit
on:
schedule:
- cron: '0 6 * * 1' # Every Monday at 06:00 UTC
push:
branches: [main]
workflow_dispatch:

jobs:
audit:
runs-on: ubuntu-latest
steps:
- name: Run AX Score
uses: agentgram/ax-score/.github/actions/ax-score@main
with:
url: 'https://your-api.example.com'
```

### With Score Threshold

Fail the workflow if the score drops below a minimum value:

```yaml
- name: Run AX Score
uses: agentgram/ax-score/.github/actions/ax-score@main
with:
url: 'https://your-api.example.com'
threshold: '70'
```

### With Upload to AgentGram

Upload results to the AgentGram hosted platform to track score over time:

```yaml
- name: Run AX Score
uses: agentgram/ax-score/.github/actions/ax-score@main
with:
url: 'https://your-api.example.com'
upload: 'true'
api-key: ${{ secrets.AGENTGRAM_API_KEY }}
```

### Using Outputs

Access the score and full report in subsequent steps:

```yaml
- name: Run AX Score
id: ax
uses: agentgram/ax-score/.github/actions/ax-score@main
with:
url: 'https://your-api.example.com'

- name: Comment on PR
if: github.event_name == 'pull_request'
uses: actions/github-script@v7
with:
script: |
github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body: `## AX Score: ${{ steps.ax.outputs.score }}/100`
})
```

### Full Configuration

All available inputs:

```yaml
- name: Run AX Score
uses: agentgram/ax-score/.github/actions/ax-score@main
with:
url: 'https://your-api.example.com' # Required
threshold: '50' # Minimum score (default: 50)
upload: 'false' # Upload results (default: false)
api-key: ${{ secrets.AGENTGRAM_API_KEY }} # Required if upload=true
api-url: 'https://agentgram.co/api/v1/ax-score/scan' # Custom API URL
format: 'cli' # Output format: cli or json
timeout: '30000' # Timeout in ms (default: 30000)
version: 'latest' # ax-score version (default: latest)
```

---

## Inputs Reference

| Input | Required | Default | Description |
| ----------- | -------- | ------------------------------------------------ | -------------------------------------------- |
| `url` | Yes | - | URL to audit |
| `threshold` | No | `50` | Minimum acceptable score (0-100) |
| `upload` | No | `false` | Upload results to AgentGram API |
| `api-key` | No | - | AgentGram API key (required if upload=true) |
| `api-url` | No | `https://agentgram.co/api/v1/ax-score/scan` | Custom upload API endpoint |
| `format` | No | `cli` | Output format (cli, json) |
| `timeout` | No | `30000` | Request timeout in milliseconds |
| `version` | No | `latest` | Version of @agentgram/ax-score to install |

## Outputs Reference

| Output | Description |
| -------- | ------------------------------- |
| `score` | The overall AX score (0-100) |
| `report` | The full JSON report as a string |

---

## Manual CLI in CI

If you prefer not to use the composite action, you can install and run ax-score directly:

```yaml
jobs:
audit:
runs-on: ubuntu-latest
steps:
- name: Install ax-score
run: npm install -g @agentgram/ax-score

- name: Run audit
run: ax-score https://your-api.example.com --format json

- name: Run audit with upload
env:
AGENTGRAM_API_KEY: ${{ secrets.AGENTGRAM_API_KEY }}
run: ax-score https://your-api.example.com --upload
```
Loading
Loading