Skip to content

Convert rate_limited_request to use async HTTP client #15

@timothyfroehlich

Description

@timothyfroehlich

Overview

Replace blocking requests.get() with async HTTP client in rate_limited_request() function.

Identified by: Copilot PR review on #7

Problem

The rate_limited_request() function is declared as async but uses the blocking requests.get() call, which can block the event loop and hurt performance.

File Affected

  • src/api.py:139 in rate_limited_request()

Current Code

async def rate_limited_request(url: str, max_retries: int = 3, base_delay: float = 1.0) -> requests.Response:
    # ...
    response = requests.get(url, timeout=10)  # ❌ Blocking call in async function
    # ...

Solution Options

  1. Use aiohttp (recommended - already a dependency)
  2. Use asyncio.to_thread() to offload to thread pool
  3. Use httpx (would need new dependency)

Recommended Implementation

import aiohttp

async def rate_limited_request(url: str, max_retries: int = 3, base_delay: float = 1.0) -> aiohttp.ClientResponse:
    async with aiohttp.ClientSession() as session:
        async with session.get(url, timeout=aiohttp.ClientTimeout(total=10)) as response:
            # ... existing logic

Impact

  • Performance: Prevents event loop blocking
  • Scalability: Better handling of concurrent requests
  • Architecture: Maintains true async behavior throughout

Acceptance Criteria

  • Replace requests.get() with aiohttp
  • Update return type annotations
  • Maintain existing retry logic and error handling
  • All existing functionality works correctly
  • No breaking changes to callers

Priority

Medium - Performance improvement, not a critical bug.

Related to linting infrastructure work but separate architectural concern.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions