Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions INTEGRATION_CHECKLIST.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ The repo's `ruff.toml` enforces rules `E` (pycodestyle errors), `F` (pyflakes),
|------|-----------|-----|
| `__init__.py` | `F401` (unused import) | Import-and-re-export is the expected pattern |
| `tests/context.py` | `F401`, `E402` | Import-after-path-setup is the expected pattern |
| `tests/conftest.py` | `E402` | Import-after-path-setup is the expected pattern (pytest alternative to `context.py`) |

Other files that need intentional "unused" imports must use `# noqa: F401` inline.

Expand Down
37 changes: 32 additions & 5 deletions LOCAL_DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ Run this **first** — it catches structural problems before you waste time on c
- `config.json` has the required fields and valid schema
- `__init__.py` is minimal (only import + `__all__`)
- `requirements.txt` includes `autohive-integrations-sdk`
- `tests/` folder has `__init__.py`, `context.py`, and at least one `test_*.py`
- `tests/` folder has `__init__.py`, `context.py` or `conftest.py`, and at least one `test_*.py`
- Icon is exactly 512x512 pixels
- OAuth scopes are actually used (heuristic)

Expand Down Expand Up @@ -133,19 +133,46 @@ python scripts/check_version_bump.py origin/main $DIRS

Once everything passes, run `validate_integration.py` for a final structure check before pushing.

## Running Tests

Unit tests and integration tests are run separately. See the integrations repo's [CONTRIBUTING.md](https://github.com/autohive-ai/autohive-integrations/blob/master/CONTRIBUTING.md#running-tests) for full details.

### Unit tests (CI + local)

```bash
# Via the tooling runner (installs deps per-integration, runs with coverage)
python scripts/run_tests.py my-integration

# Or directly via pytest (from the integrations repo root)
pytest my-integration/
```

### Integration tests (local only)

Integration tests (`test_*_integration.py`) call real APIs and are **never** run in CI. They must be invoked by passing the file path explicitly:

```bash
pytest my-integration/tests/test_my_integration_integration.py -m integration
```

They are excluded from auto-discovery by two mechanisms:
1. `python_files` in `pyproject.toml` only matches `test_*_unit.py`
2. `addopts` includes `-m unit`, which deselects `@pytest.mark.integration` tests

## What CI Runs on Your PR

The `validate-integration.yml` workflow uses the composite action defined in `action.yml` to run three checks on every PR:
The `validate-integration.yml` workflow uses the composite action defined in `action.yml` to run these checks on every PR:

| Step | Script | What It Does |
|------|--------|-------------|
| 1 | `get_changed_dirs.py` | Detects which integration folders changed |
| 2 | `validate_integration.py` | Structure and config validation |
| 3 | `check_code.py` | Syntax, imports, JSON, lint, format, security, deps, config sync |
| 4 | `check_readme.py` | Checks that the main README.md was updated for new integrations |
| 5 | `check_version_bump.py` | Checks that config.json version was incremented, recommends bump level |
| 4 | `run_tests.py` | Installs each integration's deps, runs `test_*_unit.py` files (unit tests only) |
| 5 | `check_readme.py` | Checks that the main README.md was updated for new integrations |
| 6 | `check_version_bump.py` | Checks that config.json version was incremented, recommends bump level |

If no integration directories changed (only `scripts/`, `tests/`, etc.), steps 2–5 are skipped.
If no integration directories changed (only `scripts/`, `tests/`, etc.), steps 2–6 are skipped.

Results are posted as a sticky PR comment showing ✅ Passed, ⚠️ Passed with warnings, or ❌ Failed for each check.

Expand Down
8 changes: 6 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -198,7 +198,11 @@ python scripts/run_tests.py hackernews bitly notion
python scripts/run_tests.py
```

Integrations without `test_*_unit.py` files are skipped with a warning. The test infrastructure (`pyproject.toml`, `conftest.py`, `requirements-test.txt`) lives in the integrations repo — see its `CONTRIBUTING.md` for how to write and run tests locally.
Integrations without `test_*_unit.py` files are skipped with a warning.

> **Note:** This script only runs unit tests. Integration tests (`test_*_integration.py`) require real API credentials and are run locally by developers — never in CI. See the integrations repo's `CONTRIBUTING.md` for details.

The test infrastructure (`pyproject.toml`, `conftest.py`, `requirements-test.txt`) lives in the integrations repo — see its `CONTRIBUTING.md` for how to write and run tests locally.

## Integration Requirements

Expand All @@ -211,7 +215,7 @@ See `INTEGRATION_CHECKLIST.md` for full details.
- `requirements.txt` - Dependencies (must include `autohive-integrations-sdk`)
- `README.md` - Documentation
- `icon.png` or `icon.svg` - Integration icon (512x512 pixels)
- `tests/` - Test folder with `__init__.py`, `context.py`, and `test_*.py`
- `tests/` - Test folder with `__init__.py`, `context.py` or `conftest.py`, and `test_*.py`

## Integrations

Expand Down
10 changes: 10 additions & 0 deletions scripts/docs/run_tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,16 @@ my-integration/
⚠️ No unit tests to run
```

## Unit Tests Only

This script only runs **unit tests** (`test_*_unit.py`). Integration tests (`test_*_integration.py`) are a separate concern:

- They require real API credentials and must never run in CI.
- They are not auto-discovered by pytest (`python_files` restricts discovery to `test_*_unit.py`).
- Developers run them locally by passing the file path explicitly: `pytest <integration>/tests/test_*_integration.py -m integration`

See the integrations repo's `CONTRIBUTING.md` for full details on running both test types.

## Integration with CI

Called by the composite action in `action.yml`:
Expand Down
2 changes: 1 addition & 1 deletion scripts/docs/validate_integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ Allowed patterns:
|-------|----------|-------------|
| `tests/` directory exists | Error | Test directory required |
| `tests/__init__.py` exists | Error | Test package init |
| `tests/context.py` exists | Error | Test import setup |
| `tests/context.py` or `tests/conftest.py` exists | Error | Test import/fixture setup (either one satisfies this check) |
| `tests/test_*.py` exists | Error | At least one test file |

### 7. Main Python File (`_check_main_python_file`)
Expand Down
12 changes: 5 additions & 7 deletions scripts/validate_integration.py
Original file line number Diff line number Diff line change
Expand Up @@ -367,14 +367,12 @@ def _check_tests_folder(self):
return

# Check required test files
required_test_files = [
('__init__.py', 'Test package init (can be empty)'),
('context.py', 'Test context/import setup'),
]
if not (tests_path / '__init__.py').exists():
self.add_error("Missing tests/__init__.py (Test package init — can be empty)")

for filename, description in required_test_files:
if not (tests_path / filename).exists():
self.add_error(f"Missing tests/{filename} ({description})")
# Accept either context.py (legacy import setup) or conftest.py (pytest fixture setup)
if not (tests_path / 'context.py').exists() and not (tests_path / 'conftest.py').exists():
self.add_error("Missing tests/context.py or tests/conftest.py (test import/fixture setup)")

# Check for at least one test file
test_files = list(tests_path.glob('test_*.py'))
Expand Down
Loading