diff --git a/skills/README.md b/skills/README.md index 8d6e81a..50647f2 100644 --- a/skills/README.md +++ b/skills/README.md @@ -7,6 +7,8 @@ Agent skills for AI coding assistants (Amp, Claude Code, etc.) that automate com | Skill | Description | |-------|-------------| | [`upgrading-sdk-v2/`](upgrading-sdk-v2/) | Upgrades an integration from SDK 1.x to 2.0.0 | +| [`writing-unit-tests/`](writing-unit-tests/) | Writes pytest unit tests for an integration using mock_context + FetchResponse | +| [`writing-integration-tests/`](writing-integration-tests/) | Writes pytest e2e integration tests that call real APIs using the live_context fixture | ## Setup diff --git a/skills/upgrading-sdk-v2/SKILL.md b/skills/upgrading-sdk-v2/SKILL.md index d965d17..c28b271 100644 --- a/skills/upgrading-sdk-v2/SKILL.md +++ b/skills/upgrading-sdk-v2/SKILL.md @@ -20,6 +20,9 @@ return ActionResult(data={"result": False, "error": str(e), "items": []}, cost_u # After — ActionError (returns ResultType.ACTION_ERROR, skips schema validation) return ActionError(message=str(e)) + +# After — ActionError with cost (when a billable API call was made before the error) +return ActionError(message=str(e), cost_usd=0.01) ``` `ActionError` is a dataclass, not an exception — **return it, do not raise it.** @@ -28,6 +31,7 @@ Convert ALL of these patterns: - `return ActionResult(data={"error": ...})` → `return ActionError(message=...)` - `return ActionResult(data={"result": False, "error": ..., })` → `return ActionError(message=...)` (extra keys like `"items": []` are dropped — ActionError only carries a message) - Exception catch blocks: `return ActionResult(data={"error": str(e)})` → `return ActionError(message=str(e))` +- Cost-bearing error paths: `return ActionResult(data={"error": str(e)}, cost_usd=0.01)` → `return ActionError(message=str(e), cost_usd=0.01)` — preserve the cost so billing is accurate ### SDK 2.0.0 — FetchResponse (breaking change) @@ -89,6 +93,7 @@ For every `context.fetch()` call site: 2. Convert every `return ActionResult(data={"error": ...})` to `return ActionError(message=...)` 3. Convert every `return ActionResult(data={"result": False, "error": ...})` to `return ActionError(message=...)` 4. Convert every `except Exception as e: return ActionResult(data={"error": str(e)})` to `return ActionError(message=str(e))` +5. Remove the `"error"` property (and any `"result": bool` property used only for error signalling) from each action's output schema in `config.json` — these fields are no longer returned in the action output **Do NOT change:** - Error handling (`try/except`) — exceptions are raised the same way @@ -249,6 +254,7 @@ Before considering an integration upgraded, verify: - [ ] All `context.fetch()` return values access `.data` for the body - [ ] All error paths return `ActionError(message=...)` instead of `ActionResult` with error data - [ ] `ActionError` is imported from the SDK +- [ ] `"error"` and error-only `"result"` properties removed from output schemas in `config.json` - [ ] `requirements.txt` pins `autohive-integrations-sdk~=2.0.0` - [ ] `config.json` version is bumped to `2.0.0` - [ ] Unit test mocks wrap return values in `FetchResponse(...)` diff --git a/skills/writing-integration-tests/SKILL.md b/skills/writing-integration-tests/SKILL.md new file mode 100644 index 0000000..80186f2 --- /dev/null +++ b/skills/writing-integration-tests/SKILL.md @@ -0,0 +1,429 @@ +--- +name: writing-integration-tests +description: "Writes pytest end-to-end integration tests for an Autohive integration that call real APIs using the live_context fixture pattern. Use when asked to write integration tests, add e2e tests, create live API tests, or test an integration against a real service. Covers file structure, live_context fixture variants, environment variable handling, destructive markers, and test organization." +--- + +# Writing Integration Tests for an Integration + +## Prerequisites + +- The integration must be on **SDK 2.0.0** (`autohive-integrations-sdk~=2.0.0` in `requirements.txt`) +- If the integration is still on SDK 1.x, upgrade it first using the `upgrading-sdk-v2` skill +- `aiohttp` must be available in the test environment (used by the `live_context` fixture) +- A valid API key or OAuth token for the target service + +## How Integration Tests Differ from Unit Tests + +| Aspect | Unit Tests | Integration Tests | +|---|---|---| +| Network calls | Mocked via `mock_context` | Real via `live_context` + aiohttp | +| File suffix | `_unit.py` | `_integration.py` | +| Marker | `pytest.mark.unit` | `pytest.mark.integration` | +| CI | Runs by default | Never runs in CI | +| Credentials | Fake test tokens | Real API keys/tokens from env vars | +| Speed | Milliseconds | Seconds (network I/O) | + +## File Structure + +### Naming Convention + +Test files live in `/tests/` and follow the pattern `test__integration.py`. + +Integration tests are always a single file per integration — do not split by domain. + +``` +myintegration/tests/ +├── __init__.py +├── test_myintegration_unit.py +└── test_myintegration_integration.py +``` + +### Double Exclusion from CI + +Integration tests are excluded from CI and default `pytest` runs by **two mechanisms**: + +1. **File naming**: `pyproject.toml` has `python_files = ["test_*_unit.py"]`, so `test_*_integration.py` files are never collected +2. **Marker**: `addopts = "-m unit"` in `pyproject.toml` filters to unit-only by default + +This double exclusion ensures integration tests never run accidentally. + +### File Header (boilerplate) + +Every integration test file must start with this exact boilerplate. Replace `myintegration` with the actual integration name: + +```python +""" +End-to-end integration tests for the MyIntegration integration. + +These tests call the real MyService API and require a valid access token +set in the MYINTEGRATION_ACCESS_TOKEN environment variable (via .env or export). + +Run with: + pytest myintegration/tests/test_myintegration_integration.py -m integration + +Never runs in CI — the default pytest marker filter (-m unit) excludes these, +and the file naming (test_*_integration.py) is not matched by python_files. +""" + +import os +import sys +import importlib + +_parent = os.path.abspath(os.path.join(os.path.dirname(__file__), "..")) +_deps = os.path.abspath(os.path.join(os.path.dirname(__file__), "../dependencies")) +sys.path.insert(0, _parent) +sys.path.insert(0, _deps) + +import pytest # noqa: E402 +from unittest.mock import MagicMock, AsyncMock # noqa: E402 +from autohive_integrations_sdk import FetchResponse # noqa: E402 + +_spec = importlib.util.spec_from_file_location("myintegration_mod", os.path.join(_parent, "myintegration.py")) +_mod = importlib.util.module_from_spec(_spec) +_spec.loader.exec_module(_mod) + +myintegration = _mod.myintegration # the Integration instance + +pytestmark = pytest.mark.integration +``` + +## Environment Variables + +### Token and ID Setup + +Define environment variables at module level. Use `os.environ.get` with an empty string default: + +```python +ACCESS_TOKEN = os.environ.get("MYINTEGRATION_ACCESS_TOKEN", "") +TEST_ITEM_ID = os.environ.get("MYINTEGRATION_TEST_ITEM_ID", "") +``` + +### require_* Skip Helpers + +For tests that need specific object IDs, create `require_*` helpers that skip gracefully: + +```python +def require_item_id(): + if not TEST_ITEM_ID: + pytest.skip("MYINTEGRATION_TEST_ITEM_ID not set") +``` + +Call these at the start of any test method that needs the ID: + +```python +class TestGetItem: + async def test_returns_item(self, live_context): + require_item_id() + result = await myintegration.execute_action("get_item", {"id": TEST_ITEM_ID}, live_context) + ... +``` + +### .env.example Documentation + +Document all required and optional environment variables in the integration's `.env.example`: + +```bash +# -- MyIntegration -- +MYINTEGRATION_ACCESS_TOKEN= +MYINTEGRATION_TEST_ITEM_ID= +MYINTEGRATION_TEST_PROJECT_ID= +``` + +## The live_context Fixture + +The `live_context` fixture provides a `MagicMock` of the SDK's `ExecutionContext` but replaces the `fetch` method with a real async HTTP client using `aiohttp`. This lets the integration code run unchanged while making real network calls. + +### Variant 1: No Auth (public APIs) + +For integrations that call public APIs with no authentication (e.g., Hacker News): + +```python +@pytest.fixture +def live_context(): + import aiohttp + + async def real_fetch(url, *, method="GET", json=None, headers=None, **kwargs): + async with aiohttp.ClientSession() as session: + async with session.request(method, url, json=json, headers=headers) as resp: + data = await resp.json(content_type=None) + return FetchResponse( + status=resp.status, + headers=dict(resp.headers), + data=data, + ) + + ctx = MagicMock(name="ExecutionContext") + ctx.fetch = AsyncMock(side_effect=real_fetch) + ctx.auth = {} + return ctx +``` + +### Variant 2: API Key Auth + +For integrations where the action handler adds the API key to headers itself (e.g., Perplexity): + +```python +@pytest.fixture +def live_context(): + if not API_KEY: + pytest.skip("MYINTEGRATION_API_KEY not set — skipping integration tests") + + import aiohttp + + async def real_fetch(url, *, method="GET", json=None, headers=None, **kwargs): + async with aiohttp.ClientSession() as session: + async with session.request(method, url, json=json, headers=headers) as resp: + data = await resp.json() + return FetchResponse(status=resp.status, headers=dict(resp.headers), data=data) + + ctx = MagicMock(name="ExecutionContext") + ctx.fetch = AsyncMock(side_effect=real_fetch) + ctx.auth = {} + return ctx +``` + +### Variant 3: Platform OAuth (token injection) + +For integrations that rely on the SDK's platform auth layer to inject the OAuth token. In tests, bypass the SDK auth by manually adding the `Authorization` header: + +```python +@pytest.fixture +def live_context(): + if not ACCESS_TOKEN: + pytest.skip("MYINTEGRATION_ACCESS_TOKEN not set — skipping integration tests") + + import aiohttp + + async def real_fetch(url, *, method="GET", json=None, headers=None, params=None, **kwargs): + merged_headers = dict(headers or {}) + merged_headers["Authorization"] = f"Bearer {ACCESS_TOKEN}" + async with aiohttp.ClientSession() as session: + async with session.request(method, url, json=json, headers=merged_headers, params=params) as resp: + data = await resp.json(content_type=None) + return FetchResponse( + status=resp.status, + headers=dict(resp.headers), + data=data, + ) + + ctx = MagicMock(name="ExecutionContext") + ctx.fetch = AsyncMock(side_effect=real_fetch) + ctx.auth = { + "auth_type": "PlatformOauth2", + "credentials": {"access_token": ACCESS_TOKEN}, + } + return ctx +``` + +**How to choose**: Check the integration's `config.json` — if `auth.type` is `"platform"`, use Variant 3. If the action handler reads an API key from `context.auth` or env vars and sets headers manually, use Variant 2. If no auth is needed, use Variant 1. + +## The Destructive Marker + +### When to Use + +Tests that **create, update, or delete** real data must be marked `@pytest.mark.destructive`. This prevents accidental data mutation when running integration tests. + +```python +@pytest.mark.destructive +class TestCreateItem: + async def test_creates_item(self, live_context): + result = await myintegration.execute_action( + "create_item", {"name": f"Integration Test {os.getpid()}"}, live_context + ) + data = result.result.data + assert "item" in data + assert data["item"]["id"] is not None +``` + +### Registering the Marker + +The `destructive` marker must be registered in `pyproject.toml`: + +```toml +[tool.pytest.ini_options] +markers = [ + "unit: unit tests (mocked, no network)", + "integration: integration tests (real API calls)", + "destructive: tests that create, update, or delete real data", +] +``` + +### Running Commands + +```bash +# Read-only integration tests only +pytest myintegration/tests/test_myintegration_integration.py -m "integration and not destructive" + +# Destructive tests only +pytest myintegration/tests/test_myintegration_integration.py -m "integration and destructive" + +# All integration tests +pytest myintegration/tests/test_myintegration_integration.py -m integration +``` + +## Test Organization + +### Section Headers by Domain + +Group tests with comment headers matching the integration's domains: + +```python +# ---- Read-Only Tests ---- + +class TestGetItem: + ... + +class TestSearchItems: + ... + +# ---- Destructive Tests (Write Operations) ---- +# These create, update, or delete real data. +# Only run with: pytest -m "integration and destructive" + +@pytest.mark.destructive +class TestCreateItem: + ... +``` + +### One Class per Action (or Logical Group) + +```python +class TestGetContact: + async def test_returns_contact(self, live_context): + ... + + async def test_contact_has_expected_fields(self, live_context): + ... +``` + +### Chained Tests (Dynamic IDs) + +When an action requires an ID from another action's response, chain them within the test: + +```python +class TestGetBitlink: + async def test_fetches_bitlink_details(self, live_context): + # First get a real ID from a list action + list_result = await myintegration.execute_action("list_items", {"size": 1}, live_context) + items = list_result.result.data["items"] + + if not items: + pytest.skip("No items in account to test with") + + item_id = items[0]["id"] + + # Then test the action that needs the ID + result = await myintegration.execute_action("get_item", {"id": item_id}, live_context) + data = result.result.data + assert "item" in data +``` + +### Lifecycle Tests (CRUD Workflows) + +For destructive tests, a lifecycle test that creates, reads, updates, and deletes ensures cleanup: + +```python +@pytest.mark.destructive +class TestItemLifecycle: + """End-to-end workflow: create → read → update → delete.""" + + async def test_full_lifecycle(self, live_context): + # Step 1: Create + create_result = await myintegration.execute_action( + "create_item", {"name": f"Integration test {os.getpid()}"}, live_context + ) + item_id = create_result.result.data["item"]["id"] + assert item_id is not None + + # Step 2: Read + read_result = await myintegration.execute_action("get_item", {"id": item_id}, live_context) + assert read_result.result.data["item"]["id"] == item_id + + # Step 3: Update + update_result = await myintegration.execute_action( + "update_item", {"id": item_id, "name": "Updated name"}, live_context + ) + assert update_result.result.data["success"] is True + + # Step 4: Delete (cleanup) + delete_result = await myintegration.execute_action("delete_item", {"id": item_id}, live_context) + assert delete_result.result.data["success"] is True +``` + +## What to Assert + +Integration tests validate that the integration works against the real API. Focus on: + +1. **Response structure** — expected keys exist in the result data +2. **Non-empty results** — list actions return items, get actions return the object +3. **ID round-trips** — the ID you pass in appears in the response +4. **Limits respected** — passing `limit: 2` returns ≤ 2 items +5. **Error handling** — nonexistent IDs return `ActionError` (if the action handles it) + +Do **not** assert on specific data values from the live API — data changes over time. + +```python +# ✅ Good — structural assertions +assert "contacts" in data +assert data["contact_id"] == TEST_CONTACT_ID +assert len(data["results"]) <= 5 + +# ❌ Bad — brittle value assertions +assert data["contacts"][0]["email"] == "john@acme.com" +assert data["total"] == 42 +``` + +## Coverage Expectations + +| Integration Size | Actions | Target Integration Tests | +|---|---|---| +| Small (1–5 actions) | All | 5–15 | +| Medium (6–15 actions) | All | 15–30 | +| Large (16+ actions) | All read-only + key write flows | 30–50 | + +Every action should have at minimum: +1. One test proving it **works against the real API** +2. One test verifying the **response structure** + +Write actions (create/update/delete) need at least: +1. One `@pytest.mark.destructive` test proving it works +2. Cleanup of created data where possible (lifecycle tests) + +## Workflow + +1. **Read the integration source** — understand each action and its auth mechanism +2. **Check `config.json`** — determine auth type (none / API key / platform OAuth) +3. **Choose the right `live_context` variant** — see the three variants above +4. **Document env vars** in `.env.example` +5. **Write read-only tests first** — these are safe to run repeatedly +6. **Add destructive tests** with `@pytest.mark.destructive` for write actions +7. **Run read-only tests**: + ```bash + pytest /tests/test_*_integration.py -m "integration and not destructive" + ``` +8. **Run destructive tests** (only when you're sure): + ```bash + pytest /tests/test_*_integration.py -m "integration and destructive" + ``` + +## Common Gotchas + +1. **content_type=None in resp.json()**: Some APIs return JSON without a proper `Content-Type` header. Always use `await resp.json(content_type=None)` in the `real_fetch` function to avoid `ContentTypeError`. + +2. **Test isolation**: Each test creates a new `aiohttp.ClientSession`. This is intentional — integration tests should not share session state. + +3. **Rate limiting**: If the API has rate limits, add a small `asyncio.sleep` between tests or reduce the number of tests that hit the same endpoint. + +4. **Dynamic test data**: Never hardcode IDs from the live API. Use environment variables for pre-existing objects, or chain actions (list → get) within the test. + +5. **Destructive test cleanup**: Always clean up created data at the end of lifecycle tests. Use `os.getpid()` in created object names to avoid collisions when running tests in parallel. + +6. **OAuth token expiry**: Platform OAuth tokens expire. Document the refresh process in the integration's README or `.env.example`. + +## Reference Implementations + +Look at these integrations for well-tested examples: +- `hackernews/tests/test_hackernews_integration.py` — public API, no auth, multiple action types +- `perplexity/tests/test_perplexity_integration.py` — API key auth, single action, thorough parameter coverage +- `bitly/tests/test_bitly_integration.py` — platform OAuth, chained tests (list → get), read-only +- `hubspot/tests/test_hubspot_integration.py` — platform OAuth, require_* helpers, destructive marker, lifecycle tests diff --git a/skills/writing-unit-tests/SKILL.md b/skills/writing-unit-tests/SKILL.md new file mode 100644 index 0000000..fee4887 --- /dev/null +++ b/skills/writing-unit-tests/SKILL.md @@ -0,0 +1,443 @@ +--- +name: writing-unit-tests +description: "Writes pytest unit tests for an Autohive integration using the mock_context + FetchResponse pattern. Use when asked to write tests, add test coverage, create unit tests, or test an integration. Covers file structure, test categories, and coverage expectations." +--- + +# Writing Unit Tests for an Integration + +## Prerequisites + +- The integration must be on **SDK 2.0.0** (`autohive-integrations-sdk~=2.0.0` in `requirements.txt`) +- If the integration is still on SDK 1.x, upgrade it first using the `upgrading-sdk-v2` skill + +## File Structure + +### Naming Convention + +Test files live in `/tests/` and follow the pattern `test___unit.py`. + +For small integrations (1–10 actions), a single file is fine: + +``` +myintegration/tests/ +├── __init__.py +└── test_myintegration_unit.py +``` + +For large integrations (10+ actions), split by domain: + +``` +hubspot/tests/ +├── __init__.py +├── test_hubspot_helpers_unit.py +├── test_hubspot_contacts_unit.py +├── test_hubspot_companies_unit.py +├── test_hubspot_deals_unit.py +├── test_hubspot_notes_unit.py +├── test_hubspot_tickets_unit.py +└── test_hubspot_misc_unit.py +``` + +The `_unit.py` suffix is required — CI uses it to discover unit tests. + +### File Header (boilerplate) + +Every test file must start with this exact boilerplate. Replace `myintegration` with the actual integration name and `myintegration.py` with the actual entry point file: + +```python +import os +import sys +import importlib + +_parent = os.path.abspath(os.path.join(os.path.dirname(__file__), "..")) +_deps = os.path.abspath(os.path.join(os.path.dirname(__file__), "../dependencies")) +sys.path.insert(0, _parent) +sys.path.insert(0, _deps) + +import pytest # noqa: E402 +from unittest.mock import AsyncMock, MagicMock # noqa: E402 +from autohive_integrations_sdk import FetchResponse # noqa: E402 +from autohive_integrations_sdk.integration import ResultType # noqa: E402 + +_spec = importlib.util.spec_from_file_location("myintegration_mod", os.path.join(_parent, "myintegration.py")) +_mod = importlib.util.module_from_spec(_spec) +_spec.loader.exec_module(_mod) + +myintegration = _mod.myintegration # the Integration instance +# Also import any helper functions you need to test directly: +# parse_response = _mod.parse_response +# my_helper = _mod.my_helper + +pytestmark = pytest.mark.unit +``` + +Add `from unittest.mock import patch` if you need to patch `asyncio.sleep` or environment variables. + +### mock_context Fixture + +Every test file needs this fixture: + +```python +@pytest.fixture +def mock_context(): + ctx = MagicMock(name="ExecutionContext") + ctx.fetch = AsyncMock(name="fetch") + ctx.auth = {} + return ctx +``` + +If the integration reads credentials from `context.auth`, populate it to match the auth shape in `config.json`. + +**Platform OAuth** (`config.auth.type == "platform"`): The SDK wraps OAuth tokens in a standard envelope: + +```python +ctx.auth = { + "auth_type": "PlatformOauth2", + "credentials": {"access_token": "test_token"}, # nosec B105 +} +``` + +**Custom auth** (`config.auth.type == "custom"`): The SDK validates `context.auth` directly against `config.auth.fields`. Use a flat object matching the field schema: + +```python +# Example: Freshdesk (api_key + domain) +ctx.auth = { + "api_key": "test_api_key", # nosec B105 + "domain": "testcompany", +} + +# Example: Ghost (api_url + content_api_key + admin_api_key) +ctx.auth = { + "api_url": "https://test.ghost.io", + "content_api_key": "test_content_key", # nosec B105 + "admin_api_key": "test_admin_key", # nosec B105 +} +``` + +Using the wrong shape will cause `ResultType.VALIDATION_ERROR` before the action handler runs, making tests false negatives. + +## Test Categories + +Every action should have tests from **all applicable** categories below. Aim for 4–8 tests per action. + +### 1. Happy Path (functional correctness) + +Tests that verify the action works end-to-end with valid inputs and a successful API response. + +```python +class TestGetContact: + @pytest.mark.asyncio + async def test_contact_found(self, mock_context): + mock_context.fetch.return_value = FetchResponse( + status=200, + headers={}, + data={"results": [{"id": "123", "properties": {"email": "test@example.com"}}]}, + ) + + result = await myintegration.execute_action( + "get_contact", {"email": "test@example.com"}, mock_context + ) + + assert result.result.data["contact"]["id"] == "123" +``` + +### 2. Request Verification (are we calling the API correctly?) + +Tests that verify the exact HTTP request being sent — URL, method, headers, payload structure. + +```python + @pytest.mark.asyncio + async def test_request_url_and_method(self, mock_context): + mock_context.fetch.return_value = FetchResponse(status=200, headers={}, data={"results": []}) + + await myintegration.execute_action( + "get_contact", {"email": "test@example.com"}, mock_context + ) + + call_args = mock_context.fetch.call_args + assert call_args.args[0] == "https://api.hubapi.com/crm/v3/objects/contacts/search" + assert call_args.kwargs["method"] == "POST" + + @pytest.mark.asyncio + async def test_request_payload(self, mock_context): + mock_context.fetch.return_value = FetchResponse(status=200, headers={}, data={"results": []}) + + await myintegration.execute_action( + "get_contact", {"email": "test@example.com"}, mock_context + ) + + payload = mock_context.fetch.call_args.kwargs["json"] + assert payload["filterGroups"][0]["filters"][0]["value"] == "test@example.com" + assert payload["limit"] == 1 +``` + +### 3. Error Paths (exception handling) + +Tests that verify exceptions are caught and returned as `ActionError`. + +```python + @pytest.mark.asyncio + async def test_exception_returns_action_error(self, mock_context): + mock_context.fetch.side_effect = Exception("Connection refused") + + result = await myintegration.execute_action( + "get_contact", {"email": "test@example.com"}, mock_context + ) + + assert result.type == ResultType.ACTION_ERROR + assert "Connection refused" in result.result.message +``` + +Not all actions have try/except blocks. Only write error path tests for actions that catch exceptions. Read the implementation first. + +### 4. Edge Cases (boundary conditions) + +Tests for unusual-but-valid inputs, empty results, default values, limit clamping. + +```python + @pytest.mark.asyncio + async def test_contact_not_found(self, mock_context): + mock_context.fetch.return_value = FetchResponse( + status=200, headers={}, data={"results": []} + ) + + result = await myintegration.execute_action( + "get_contact", {"email": "nobody@example.com"}, mock_context + ) + + assert result.type == ResultType.ACTION_ERROR + + @pytest.mark.asyncio + async def test_default_limit(self, mock_context): + mock_context.fetch.return_value = FetchResponse(status=200, headers={}, data={"results": []}) + + await myintegration.execute_action("search_contacts", {"query": "test"}, mock_context) + + payload = mock_context.fetch.call_args.kwargs["json"] + assert payload["limit"] == 100 # verify the default + + @pytest.mark.asyncio + async def test_limit_clamped_to_200(self, mock_context): + mock_context.fetch.return_value = FetchResponse(status=200, headers={}, data={"results": []}) + + await myintegration.execute_action( + "get_notes", {"contact_id": "1", "limit": 150}, mock_context + ) + + payload = mock_context.fetch.call_args.kwargs["json"] + assert payload["limit"] == 150 # within max, passed through +``` + +### 5. Response Shape (contract verification) + +Tests that verify the response data has the expected keys and structure. + +```python + @pytest.mark.asyncio + async def test_response_data_structure(self, mock_context): + mock_context.fetch.return_value = FetchResponse( + status=200, headers={}, data={"id": "123", "properties": {}} + ) + + result = await myintegration.execute_action( + "get_company", {"company_id": "123"}, mock_context + ) + + assert "company" in result.result.data + assert result.result.data["company"]["id"] == "123" +``` + +### 6. Business Logic (domain-specific behavior) + +Tests for integration-specific behavior like timestamp conversion, date parsing, association type IDs, data transformation. + +```python + @pytest.mark.asyncio + async def test_timestamps_converted_to_utc(self, mock_context): + mock_context.fetch.return_value = FetchResponse( + status=200, + headers={}, + data={"results": [{"properties": {"hs_timestamp": "2025-01-15T10:30:00.000Z"}}]}, + ) + + result = await myintegration.execute_action( + "get_notes", {"contact_id": "1"}, mock_context + ) + + assert "UTC" in result.result.data["notes"][0]["properties"]["hs_timestamp"] +``` + +## Mocking Patterns + +### Single fetch call + +```python +mock_context.fetch.return_value = FetchResponse(status=200, headers={}, data={...}) +``` + +### Multiple sequential fetches (e.g., get thread ID then get messages) + +```python +mock_context.fetch.side_effect = [ + FetchResponse(status=200, headers={}, data={"properties": {"thread_id": "t-1"}}), + FetchResponse(status=200, headers={}, data={"results": [{"text": "Hello"}]}), +] +``` + +### Exception from fetch + +```python +mock_context.fetch.side_effect = Exception("HTTP 500: Internal Server Error") +``` + +### DELETE responses (204 No Content) + +```python +mock_context.fetch.return_value = FetchResponse(status=204, headers={}, data=None) +``` + +### Patching asyncio.sleep (for actions with delays) + +```python +from unittest.mock import patch + +@patch("asyncio.sleep", new_callable=AsyncMock) +async def test_paginated_fetch(self, mock_sleep, mock_context): + mock_context.fetch.return_value = FetchResponse(...) + result = await myintegration.execute_action("get_all_items", {...}, mock_context) + # mock_sleep prevents real delay during tests +``` + +### Patching environment variables + +```python +from unittest.mock import patch + +@patch.dict(os.environ, {"MY_API_KEY": "test-key-123"}) # nosec B105 +async def test_api_key_in_header(self, mock_context): + ... +``` + +## Testing Helper Functions + +Pure helper functions (no async, no context) should be tested directly without `mock_context`: + +```python +class TestMyDateParser: + def test_iso_format(self): + result = parse_date("2025-01-15") + assert result.year == 2025 + + def test_none_returns_none(self): + assert parse_date(None) is None + + def test_invalid_raises(self): + with pytest.raises(ValueError, match="Unable to parse"): + parse_date("not-a-date") +``` + +Async helpers like `parse_response` need `@pytest.mark.asyncio`: + +```python +class TestParseResponse: + @pytest.mark.asyncio + async def test_dict_data(self): + response = FetchResponse(status=200, headers={}, data={"key": "value"}) + result = await parse_response(response) + assert result == {"key": "value"} +``` + +## Test Organization + +### One class per action + +```python +class TestGetContact: + # all get_contact tests here + +class TestCreateContact: + # all create_contact tests here +``` + +### Group related tests with comment headers + +```python +# ---- Contact Management ---- + +class TestGetContact: + ... + +class TestCreateContact: + ... + +# ---- Note Management ---- + +class TestCreateNote: + ... +``` + +### Sample test data at module level + +```python +SAMPLE_CONTACT = { + "id": "123", + "properties": {"email": "test@example.com", "firstname": "John"}, +} + +SAMPLE_NOTE_RESPONSE = { + "id": "456", + "properties": {"hs_note_body": "Test note"}, +} +``` + +## Coverage Expectations + +| Integration Size | Actions | Target Tests | Tests/Action | +|---|---|---|---| +| Small | 1–5 | 20–40 | 5–8 | +| Medium | 6–15 | 40–100 | 4–7 | +| Large | 16–50 | 100–300 | 4–6 | + +Every action should have at minimum: +1. One **happy path** test +2. One **request verification** test (URL + method) +3. One **error path** test (if the action has try/except) +4. One **edge case** or **response shape** test + +## Workflow + +1. **Read the integration source** — understand each action's implementation +2. **Identify helper functions** — test pure functions first (easiest) +3. **Create test file(s)** with boilerplate header and `mock_context` fixture +4. **Write tests action by action** — go through each test category +5. **Run tests**: `python -m pytest /tests/test_*_unit.py -v` +6. **Lint and format**: + ```bash + ruff check --fix /tests + ruff format --config ../autohive-integrations-tooling/ruff.toml /tests + ``` +7. **Verify all pass** before committing + +## Common Gotchas + +1. **Actions that make multiple fetches**: Use `side_effect` with a list, not `return_value`. The list must have exactly the right number of `FetchResponse` objects in the right order. + +2. **asyncio.gather in actions**: When an action runs fetches in parallel with `asyncio.gather`, the mock's `side_effect` list still works — each `await context.fetch(...)` pops the next item. + +3. **SDK input validation**: The SDK validates inputs against `config.json` schemas before the action handler runs. If you pass invalid inputs (wrong type, missing required field), the result will have `type == ResultType.VALIDATION_ERROR` and the handler never executes. This means `mock_context.fetch` is never called. + +4. **ActionError vs ActionResult**: Error paths return `ActionError(message=...)` which results in `result.type == ResultType.ACTION_ERROR` and `result.result.message` containing the error. Happy paths return `ActionResult(data=...)` which results in `result.result.data` containing the response. + +5. **Silent exception blocks**: Some actions have `except Exception: continue` or `except Exception: pass` — these are intentional skip-on-failure patterns. Don't write error tests for these; they don't return ActionError. + +6. **The `nosec` comment**: Use `# nosec B105` after test token strings to suppress Bandit false positives on hardcoded credentials in tests. + +7. **Unused variables**: If you call `execute_action` only to verify `mock_context.fetch.call_args`, don't assign the result to a variable — ruff will flag it as unused. Use `await integration.execute_action(...)` without assignment. + +## Reference Implementations + +Look at these integrations for well-tested examples: +- `perplexity/tests/test_perplexity_unit.py` — single-action integration, thorough coverage +- `hackernews/tests/test_hackernews_unit.py` — multi-action with helper function tests +- `bitly/tests/test_bitly_unit.py` — pure function tests + action tests +- `hubspot/tests/test_hubspot_*_unit.py` — large integration split across 7 domain files