Skip to content

testing.md

Latest commit

 

History

History
159 lines (115 loc) · 4.74 KB

testing.md

File metadata and controls

159 lines (115 loc) · 4.74 KB

Testing the AgentMeter Action

Unit tests

npm test

Watch mode during development:

npm run test:watch

The test suite covers:

  • token-extractor.test.ts — JSON and regex-based token extraction, turns extraction (Claude Code JSON num_turns, Codex exec JSONL turn.completed count, regex fallback, null cases), priority logic, edge cases
  • context.test.ts — GitHub event → trigger type mapping, trigger ref extraction for all event types, issue_comment PR vs issue distinction
  • comment.test.ts — Markdown comment formatting, status emojis, cost formatting, multi-run accumulation, newest-first ordering, 5-run visible limit with collapsible history, old/new column format compatibility
  • ingest.test.ts — API client success/failure handling, retry logic, Authorization header
  • workflow-run.test.ts — workflow_run_id auto-resolution: gate logic, status normalization, trigger number fallback, zip artifact parsing, partial artifact handling
  • pricing.test.ts — fetchPricing success/failure/timeout/malformed-response handling, getPricing exact and prefix matching, case insensitivity, null cache pricing

Local integration test

To test the full compiled action against a real or mocked AgentMeter API:

1. Build the bundle

npm run build

2. Set environment variables

Create a test event payload file:

cat > /tmp/event.json << 'EOF'
{
  "action": "labeled",
  "issue": {
    "number": 1,
    "pull_request": null
  },
  "label": {
    "name": "agent"
  }
}
EOF

Export the required env vars:

export INPUT_API_KEY="am_sk_your_key_here"
export INPUT_MODEL="claude-sonnet-4-5"
export INPUT_STATUS="success"
export INPUT_ENGINE="claude"
export INPUT_POST_COMMENT="false"
export INPUT_API_URL="http://localhost:3000"   # or https://agentmeter.app

export GITHUB_REPOSITORY="yourorg/your-repo"
export GITHUB_RUN_ID="12345678"
export GITHUB_WORKFLOW="agent-implement"
export GITHUB_EVENT_NAME="issues"
export GITHUB_EVENT_PATH="/tmp/event.json"
export GITHUB_TOKEN="your_github_pat"   # only needed if post_comment=true

3. Run the compiled action

node dist/index.js

4. Test with explicit token counts

export INPUT_INPUT_TOKENS="5000"
export INPUT_OUTPUT_TOKENS="2000"
export INPUT_CACHE_READ_TOKENS="15000"
export INPUT_CACHE_WRITE_TOKENS="3000"
export INPUT_TURNS="12"
node dist/index.js

5. Test with JSON agent output (auto-extraction)

export INPUT_AGENT_OUTPUT='{"usage":{"input_tokens":1000,"output_tokens":500,"cache_read_input_tokens":200,"cache_creation_input_tokens":100}}'
node dist/index.js

6. Test turns auto-extraction

# Claude Code JSON with num_turns
export INPUT_AGENT_OUTPUT='{"num_turns":7,"usage":{"input_tokens":1000,"output_tokens":500}}'
node dist/index.js

# Codex exec JSONL turn.completed events
export INPUT_AGENT_OUTPUT='{"type":"turn.completed"}
{"type":"turn.completed"}
{"type":"turn.completed"}'
node dist/index.js

# Explicit turns override (auto-extraction ignored)
export INPUT_TURNS="5"
node dist/index.js

Testing against a local AgentMeter API

export INPUT_API_URL="http://localhost:3000"
export INPUT_API_KEY="am_sk_your_local_key"
node dist/index.js

Testing on a PR branch (inline test workflow)

The repo includes .github/workflows/agentmeter-inline-test.yml which fires on every pull_request event and runs the action directly from the branch code (uses: ./.). This is the primary way to test action changes without merging to main first.

It uses hardcoded synthetic token counts and a sleep 10 step so the reported duration is non-zero.

CI

.github/workflows/ci.yml runs on every push and PR:

  1. npm run lint — Biome linting
  2. npm run type-check — TypeScript strict mode
  3. npm test — Full test suite

.github/workflows/build.yml also runs on every push and PR to verify dist/index.js is not stale.

Publishing a new version

  1. Ensure all tests pass: npm test
  2. Run type check: npm run type-check
  3. Commit everything — the pre-commit hook runs lint-staged (Biome auto-fix), then npm run lint as a hard gate, then npm run type-check, then npm run build, and finally stages dist/index.js and dist/licenses.txt
  4. Push to main
  5. Create a GitHub release with a semver tag: v1.0.0
  6. Update the major version tag: git tag -f v1 && git push -f origin v1
  7. In the GitHub release UI, check "Publish this Action to GitHub Marketplace"

Note: dist/ must be committed. GitHub Actions checks out the repo at the referenced tag and runs dist/index.js directly — it does not run npm install or npm run build. The pre-commit hook ensures dist/ is always up-to-date before any commit lands.