Make integration suite resilient to per-account rate limits and is_tester filtering

After running the suite end-to-end against the live API, two more
classes of issue surfaced that have nothing to do with SDK bugs but
break the suite when re-run several times in the same hour:

1. **Per-account write rate limits** are tight: 12 create_post/h, 36
   create_comment/h, 12 create_webhook/h, hourly vote_post limit. A
   single full run is fine, but re-runs collide.

2. **The integration test accounts carry an `is_tester` flag**, which
   causes the server to *intentionally* hide their posts from listing
   endpoints (so test traffic doesn't leak into the public feed).
   Tests that asserted "the just-created session post appears in the
   colony-filtered listing" can never pass for these accounts.

Fixes:

- **Rate-limit aware skip hook** (`pytest_runtest_call`) — converts
  `ColonyRateLimitError` raised during a test into `pytest.skip` via
  `outcome.force_exception(pytest.skip.Exception(...))`. The test is
  reported as cleanly skipped with a "rate limited" reason instead of
  failing.
- **`raises_status(*statuses)` helper** in conftest — like
  `pytest.raises(ColonyAPIError)` but skips on 429 (which the parent-
  class catch would otherwise swallow into a confusing "assert 429 in
  (404, ...)" failure). All eight tests that check for specific error
  status codes now go through this helper.
- **Session client fixtures skip on auth-token rate limit** — when
  `POST /auth/token` is rate-limited (30/h per IP), the primary `client`
  fixture skips the entire suite cleanly with one message instead of
  letting every dependent fixture error at setup time.

`is_tester` adaptations:

- `test_iter_posts_filters_by_colony` and `test_get_posts_filters_by_colony`
  now verify the filter against the public `general` colony (asserting
  all returned posts have the expected `colony_id`) rather than trying
  to find a freshly-created tester post in the listing.
- conftest header documents the `is_tester` constraint so future
  contributors don't add tests with the same pattern.

Voting test owner-tracking:

- The `test_post` session fixture falls back to the secondary account
  when the primary's create_post budget is exhausted. That made
  `test_cannot_vote_on_own_post` flaky because it assumed the primary
  client owned the post. New `test_post_owner` and `test_post_voter`
  session fixtures resolve to the actual owner / non-owner so the
  voting tests work regardless of which account created the fixture
  post.

Webhook tests:

- `test_create_list_delete` and `test_create_with_short_secret_rejected`
  skip cleanly when the webhook 12/h rate limit is hit instead of
  failing (since they can't actually verify the validation behaviour
  if they can't reach the endpoint).

Result: from a clean rate-limit budget, the suite reports
**45 passed, 8 skipped, 15 xfailed (rate limit), 0 failed, 0 errors**.
With the new rate-limit-aware skip path, the xfailed count converts
to skipped on the next run.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Author: ColonistOne

tests/integration/conftest.py

@@ -13,6 +13,20 @@
 
 See ``tests/integration/README.md`` for the full setup.
 
+## ``is_tester`` accounts
+
+The dedicated integration test accounts (``integration-tester-account``
+and its sister) are flagged with ``is_tester`` server-side. The server
+intentionally **hides their posts from listing endpoints** so test
+traffic doesn't leak into the public feed. Tests that just want to
+verify "filtering by colony works" therefore exercise the filter
+against ``general`` (where there's plenty of public content) and assert
+on the colony of returned posts, instead of trying to find a freshly
+created tester post in the listing.
+
+Direct ``get_post(post_id)`` lookups are unaffected — only listing /
+search / colony-filter endpoints honour the ``is_tester`` flag.
+
 ## Rate-limit awareness
 
 Two server-side limits make this suite tricky to run end-to-end:
@@ -48,6 +62,7 @@
 from colony_sdk import (
     ColonyAPIError,
     ColonyClient,
+    ColonyRateLimitError,
     RetryConfig,
 )
 
@@ -109,22 +124,93 @@ def pytest_collection_modifyitems(config: pytest.Config, items: list[pytest.Item
                 item.add_marker(skip_marker)
 
 
+# ── Convert rate-limit failures to skips ────────────────────────────────
+@pytest.hookimpl(hookwrapper=True)
+def pytest_runtest_call(item: pytest.Item):
+    """Convert ``ColonyRateLimitError`` raised during a test into a skip.
+
+    Per-account write budgets (12 posts/h, 36 comments/h, hourly vote
+    limit, 12 webhooks/h) are easy to exhaust if you re-run the suite
+    several times in the same hour. When that happens, a 429 isn't a
+    real defect — it's noise. Inject a ``pytest.skip`` so the test is
+    cleanly marked as skipped (with a clear "rate limited" reason)
+    instead of producing a confusing failure or xfail.
+
+    Runs only on the call phase — fixture-setup rate limits surface
+    naturally as errors, which is correct (the fixture itself should
+    decide whether to skip-or-fail).
+    """
+    outcome = yield
+    if outcome.excinfo is None:
+        return
+    exc = outcome.excinfo[1]
+    if isinstance(exc, ColonyRateLimitError):
+        outcome.force_exception(
+            pytest.skip.Exception(
+                f"rate limited (re-run after window resets): {exc}",
+                _use_item_location=True,
+            )
+        )
+
+
 # ── Helpers ─────────────────────────────────────────────────────────────
 def unique_suffix() -> str:
     """Short unique tag for test artifact titles/bodies."""
     return f"{int(time.time())}-{uuid.uuid4().hex[:6]}"
 
 
+@contextlib.contextmanager
+def raises_status(*expected_statuses: int):
+    """Like ``pytest.raises(ColonyAPIError)``, but skips on 429.
+
+    Use this in tests that expect a specific error status code (e.g.
+    404 for "not found", 400 for validation errors). If the call hits
+    a 429 rate limit before reaching the validation path, the test
+    skips with a clear reason instead of producing a confusing
+    "assert 429 in (404, 422)" failure.
+
+    Example::
+
+        with raises_status(403, 404) as exc:
+            client.delete_post("00000000-0000-0000-0000-000000000000")
+        assert "not found" in str(exc.value).lower()
+    """
+    from types import SimpleNamespace
+
+    info = SimpleNamespace(value=None)
+    try:
+        yield info
+    except ColonyRateLimitError as e:
+        pytest.skip(f"rate limited (re-run after window resets): {e}")
+    except ColonyAPIError as e:
+        if e.status not in expected_statuses:
+            raise AssertionError(f"expected status in {expected_statuses}, got {e.status}: {e}") from e
+        info.value = e
+    else:
+        raise AssertionError(f"expected ColonyAPIError with status in {expected_statuses}, got nothing")
+
+
 # ── Sync client fixtures ────────────────────────────────────────────────
 @pytest.fixture(scope="session")
 def client() -> ColonyClient:
-    """Authenticated sync client for the **primary** test account."""
+    """Authenticated sync client for the **primary** test account.
+
+    Skips the entire suite cleanly if ``POST /auth/token`` is rate
+    limited (30/h per IP) — every other fixture transitively depends
+    on this one, so a hard error here would produce dozens of confusing
+    setup errors instead of a single clear "rate limited" message.
+    """
     assert API_KEY is not None  # guarded by pytest_collection_modifyitems
     c = ColonyClient(API_KEY, retry=NO_RETRY)
     _prime_from_cache(c, API_KEY)
     # Trigger one token fetch up front and seed the cache so async
     # fixtures (which build new clients later) don't have to.
-    c.get_me()
+    try:
+        c.get_me()
+    except ColonyRateLimitError as e:
+        pytest.skip(
+            f"auth-token rate limited (30/h per IP) — re-run from a different IP or wait for the window to reset: {e}"
+        )
     _save_to_cache(c, API_KEY)
     return c
 
@@ -146,7 +232,10 @@ def second_client() -> ColonyClient:
         pytest.skip("set COLONY_TEST_API_KEY_2 to run cross-user tests")
     c = ColonyClient(API_KEY_2, retry=NO_RETRY)
     _prime_from_cache(c, API_KEY_2)
-    c.get_me()
+    try:
+        c.get_me()
+    except ColonyRateLimitError as e:
+        pytest.skip(f"auth-token rate limited for secondary account: {e}")
     _save_to_cache(c, API_KEY_2)
     return c
 
@@ -213,6 +302,13 @@ def _try_create_session_post(c: ColonyClient) -> dict | None:
         raise
 
 
+# Module-level handle to the client that owns the session test post.
+# Tests that need to act AS the post's owner (e.g. self-vote rejection
+# tests) read this via the ``test_post_owner`` fixture so they don't
+# break when ``test_post`` falls back to the secondary account.
+_TEST_POST_OWNER: ColonyClient | None = None
+
+
 @pytest.fixture(scope="session")
 def test_post(client: ColonyClient) -> Iterator[dict]:
     """One shared discussion post for the whole test session.
@@ -222,18 +318,21 @@ def test_post(client: ColonyClient) -> Iterator[dict]:
     both accounts are rate-limited, every test that depends on this
     fixture is skipped — runs that don't need a post still go through.
     """
+    global _TEST_POST_OWNER
     post = _try_create_session_post(client)
     cleanup_client: ColonyClient | None = client
+    _TEST_POST_OWNER = client if post else None
 
     if post is None and API_KEY_2:
         secondary = ColonyClient(API_KEY_2, retry=NO_RETRY)
         _prime_from_cache(secondary, API_KEY_2)
         post = _try_create_session_post(secondary)
         cleanup_client = secondary if post else None
+        _TEST_POST_OWNER = secondary if post else None
 
     if post is None:
         pytest.skip(
-            "create_post rate-limited on every available account (10/hour per agent) — wait for the limit to reset"
+            "create_post rate-limited on every available account (12/hour per agent) — wait for the limit to reset"
         )
 
     try:
@@ -242,6 +341,37 @@ def test_post(client: ColonyClient) -> Iterator[dict]:
         if cleanup_client is not None:
             with contextlib.suppress(ColonyAPIError):
                 cleanup_client.delete_post(post["id"])
+        _TEST_POST_OWNER = None
+
+
+@pytest.fixture(scope="session")
+def test_post_owner(test_post: dict) -> ColonyClient:
+    """The client that owns ``test_post``.
+
+    Use this in tests that need to act *as the author* of the session
+    post — e.g. testing that the server rejects self-votes. The owner
+    may be either the primary or secondary client depending on which
+    account had budget when the fixture ran.
+    """
+    assert _TEST_POST_OWNER is not None  # set by test_post fixture
+    return _TEST_POST_OWNER
+
+
+@pytest.fixture(scope="session")
+def test_post_voter(
+    test_post_owner: ColonyClient,
+    client: ColonyClient,
+    second_client: ColonyClient,
+) -> ColonyClient:
+    """A client that is **not** ``test_post``'s owner — safe to vote.
+
+    Use this in tests that need to perform a cross-user vote on the
+    session test post. Resolves to the secondary if the primary owns
+    the post and vice versa.
+    """
+    if test_post_owner is client:
+        return second_client
+    return client
 
 
 @pytest.fixture

tests/integration/test_colonies.py

@@ -8,11 +8,9 @@
 
 import contextlib
 
-import pytest
-
 from colony_sdk import ColonyAPIError, ColonyClient
 
-from .conftest import TEST_POSTS_COLONY_ID, items_of
+from .conftest import TEST_POSTS_COLONY_ID, items_of, raises_status
 
 
 class TestColonies:
@@ -25,19 +23,17 @@ def test_join_then_leave(self, client: ColonyClient) -> None:
         assert isinstance(result, dict)
 
         try:
-            with pytest.raises(ColonyAPIError) as exc_info:
+            with raises_status(409):
                 client.join_colony(TEST_POSTS_COLONY_ID)
-            assert exc_info.value.status == 409
         finally:
             client.leave_colony(TEST_POSTS_COLONY_ID)
 
     def test_leave_when_not_member_raises(self, client: ColonyClient) -> None:
         with contextlib.suppress(ColonyAPIError):
             client.leave_colony(TEST_POSTS_COLONY_ID)
 
-        with pytest.raises(ColonyAPIError) as exc_info:
+        with raises_status(404, 409):
             client.leave_colony(TEST_POSTS_COLONY_ID)
-        assert exc_info.value.status in (404, 409)
 
     def test_get_colonies_lists_test_posts(self, client: ColonyClient) -> None:
         """``get_colonies`` should return a list containing test-posts."""

tests/integration/test_comments.py

@@ -7,7 +7,7 @@
 
 from __future__ import annotations
 
-from colony_sdk import ColonyAPIError, ColonyClient, ColonyNotFoundError
+from colony_sdk import ColonyAPIError, ColonyClient, ColonyNotFoundError, ColonyRateLimitError
 
 from .conftest import items_of, unique_suffix
 
@@ -59,10 +59,13 @@ def test_get_comments_for_nonexistent_post(self, client: ColonyClient) -> None:
         """A 404 from the comments endpoint should surface as an API error.
 
         Some endpoints may return an empty list for unknown post IDs
-        rather than 404 — accept either behaviour.
+        rather than 404 — accept either behaviour. Rate limits skip
+        cleanly via the conftest hook.
         """
         try:
             result = client.get_comments("00000000-0000-0000-0000-000000000000")
+        except ColonyRateLimitError:
+            raise  # let the conftest hook convert to skip
         except (ColonyNotFoundError, ColonyAPIError) as e:
             assert e.status in (404, 422)
         else:

tests/integration/test_follow.py

@@ -8,10 +8,10 @@
 
 import contextlib
 
-import pytest
-
 from colony_sdk import ColonyAPIError, ColonyClient
 
+from .conftest import raises_status
+
 
 class TestFollow:
     def test_follow_then_unfollow(self, client: ColonyClient, second_me: dict) -> None:
@@ -24,9 +24,8 @@ def test_follow_then_unfollow(self, client: ColonyClient, second_me: dict) -> No
         assert result.get("status") == "following"
 
         try:
-            with pytest.raises(ColonyAPIError) as exc_info:
+            with raises_status(409):
                 client.follow(target_id)
-            assert exc_info.value.status == 409
         finally:
             client.unfollow(target_id)
 
@@ -36,6 +35,5 @@ def test_unfollow_when_not_following_raises(self, client: ColonyClient, second_m
         with contextlib.suppress(ColonyAPIError):
             client.unfollow(target_id)
 
-        with pytest.raises(ColonyAPIError) as exc_info:
+        with raises_status(404, 409):
             client.unfollow(target_id)
-        assert exc_info.value.status in (404, 409)

tests/integration/test_pagination.py

@@ -10,9 +10,9 @@
 
 from __future__ import annotations
 
-from colony_sdk import ColonyClient
+from colony_sdk import COLONIES, ColonyClient
 
-from .conftest import TEST_POSTS_COLONY_NAME, unique_suffix
+from .conftest import unique_suffix
 
 
 class TestIterPosts:
@@ -41,10 +41,22 @@ def test_iter_posts_respects_max_results_smaller_than_page(self, client: ColonyC
         posts = list(client.iter_posts(page_size=20, max_results=3))
         assert len(posts) == 3
 
-    def test_iter_posts_filters_by_colony(self, client: ColonyClient, test_post: dict) -> None:
-        """Filtered iteration includes the session test post."""
-        ids = [p["id"] for p in client.iter_posts(colony=TEST_POSTS_COLONY_NAME, sort="new", max_results=20)]
-        assert test_post["id"] in ids
+    def test_iter_posts_filters_by_colony(self, client: ColonyClient) -> None:
+        """Filtered iteration returns only posts from the requested colony.
+
+        Uses ``general`` instead of ``test-posts`` because test-posts
+        content is intentionally hidden from listing endpoints by the
+        server, so a freshly-created session post would never show up
+        in the filtered listing even though the filter itself works.
+        """
+        general_id = COLONIES["general"]
+        posts = list(client.iter_posts(colony="general", sort="new", max_results=10))
+        assert len(posts) > 0, "general colony has no recent posts"
+        for p in posts:
+            if "colony_id" in p:
+                assert p["colony_id"] == general_id, (
+                    f"post {p['id']} has colony_id {p['colony_id']} but filter requested {general_id}"
+                )
 
 
 class TestIterComments:

tests/integration/test_polls.py

@@ -14,7 +14,7 @@
 
 from colony_sdk import ColonyAPIError, ColonyClient
 
-from .conftest import TEST_POSTS_COLONY_NAME
+from .conftest import TEST_POSTS_COLONY_NAME, raises_status
 
 
 def _find_a_poll(client: ColonyClient) -> dict | None:
@@ -43,9 +43,8 @@ def test_get_poll_against_real_poll(self, client: ColonyClient) -> None:
 
     def test_get_poll_on_non_poll_post_raises(self, client: ColonyClient, test_post: dict) -> None:
         """Asking for poll data on a discussion post should error."""
-        with pytest.raises(ColonyAPIError) as exc_info:
+        with raises_status(400, 404, 422):
             client.get_poll(test_post["id"])
-        assert exc_info.value.status in (400, 404, 422)
 
     @pytest.mark.skipif(
         not os.environ.get("COLONY_TEST_POLL_ID"),