Skip to content

feat: add async wrapper as extension module#4

Open
jgwesterlund wants to merge 10 commits intomainfrom
feat/async-wrapper
Open

feat: add async wrapper as extension module#4
jgwesterlund wants to merge 10 commits intomainfrom
feat/async-wrapper

Conversation

@jgwesterlund
Copy link
Owner

Pull Request

What does this change do?

Adds AsyncSDSavior in sdsavior.aio as a separate extension module wrapping SDSavior for asyncio applications. Uses a dedicated single-thread executor and asyncio.Lock for thread safety. Implements chunked async iteration to avoid list materialization.

Why is this needed?

Enables using SDSavior in async applications without blocking the event loop. Per reviewer feedback, this is a separate extension module (not integrated into core __init__.py) — users import explicitly from sdsavior.aio.

Key design decisions addressing reviewer bugs:

  • Thread safety: Uses ThreadPoolExecutor(max_workers=1) by default (not run_in_executor with the default pool which runs parallel threads)
  • No list materialization: iter_records is an AsyncIterator that yields records in chunks of 256, avoiding memory spikes for large rings

Type of change

  • Feature

How was this tested?

8 new tests in tests/test_async.py:

  • Basic append and iterate
  • Context manager
  • Export to JSONL
  • Flush
  • Concurrent appends are serialized (thread-safety verification)
  • Custom executor
  • Reopen persistence
  • from_seq filtering

All 44 tests pass, 98% coverage, ruff clean, mypy clean.

Checklist

  • I tested this locally
  • I didn't reformat the entire repo
  • I wrote sane commit messages
  • I read the contributing guide
  • This probably won't summon new bugs

Notes for reviewers

AsyncSDSavior is NOT added to __init__.py.__all__ — it's opt-in via from sdsavior.aio import AsyncSDSavior.

Added pytest-asyncio as a dev dependency and asyncio_mode = "auto" to pyproject.toml.

Docs updated: usage.md, api.md.

jgwesterlund and others added 10 commits March 4, 2026 12:55
Merge pull request #1 from jgwesterlund/fix/ring-open-recovery-safety
Automatically generated by python-semantic-release
Automatically generated by python-semantic-release
lets hope, cause im tired....
Add AsyncSDSavior in sdsavior.aio with dedicated single-thread executor
for thread safety. Implements async iteration that yields records in
chunks to avoid list materialization. Import explicitly from sdsavior.aio.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants