Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 47 additions & 14 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,15 +1,21 @@
name: CI
name: SD Savior Pipeline

on:
push:
branches: ["**"]
pull_request:
push:
branches:
- main

permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
validate:
name: Validate (lint, types, tests)
runs-on: ubuntu-latest
strategy:
fail-fast: false
Expand All @@ -24,6 +30,7 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: pip

- name: Install dependencies
run: |
Expand All @@ -36,14 +43,14 @@ jobs:
- name: Type check
run: mypy src

- name: Tests with coverage gate
- name: Tests
run: pytest -q

semver:
name: Release
name: Release (semantic-release)
runs-on: ubuntu-latest
needs: [validate]
if: github.ref == 'refs/heads/main'
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
permissions:
contents: write
env:
Expand All @@ -52,6 +59,7 @@ jobs:
released: ${{ steps.release.outputs.released }}
tag: ${{ steps.release.outputs.tag }}
commit_sha: ${{ steps.release.outputs.commit_sha }}

steps:
- name: Checkout (full history for tags)
uses: actions/checkout@v4
Expand All @@ -68,12 +76,14 @@ jobs:
git_committer_email: "41898282+github-actions[bot]@users.noreply.github.com"

docs-build:
name: Build docs
needs: [validate]
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
needs: validate
runs-on: ubuntu-latest
permissions:
pages: write
id-token: write

steps:
- name: Checkout
uses: actions/checkout@v4
Expand All @@ -82,6 +92,7 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: "3.11"
cache: pip

- name: Install dependencies
run: |
Expand All @@ -100,29 +111,51 @@ jobs:
path: site

docs-deploy:
name: Deploy docs
needs: [docs-build]
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
needs: docs-build
runs-on: ubuntu-latest
permissions:
pages: write
id-token: write
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}

steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4

pypi-publish:
needs: semver
name: Upload release to PyPI
needs: [semver]
if: github.event_name == 'push' && github.ref == 'refs/heads/main' && needs.semver.outputs.released == 'true'
runs-on: ubuntu-latest
environment:
name: pypi
url: https://pypi.org/p/sdsavior
permissions:
id-token: write

steps:
- name: Publish package distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
- name: Checkout release tag
uses: actions/checkout@v4
with:
fetch-depth: 0
ref: ${{ needs.semver.outputs.tag }}

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
cache: pip

- name: Build tooling
run: python -m pip install -U build twine

- name: Build package
run: python -m build

- name: Upload package to PyPI
run: python -m twine upload dist/*
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }}
26 changes: 26 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,32 @@

<!-- version list -->

## v1.0.2 (2026-03-04)

### Bug Fixes

- **ci**: Pypy with twine
([`f4bd2c2`](https://github.com/well-it-wasnt-me/SDSavior/commit/f4bd2c2833b42478fb15be6ac2b63a7a17ee7250))

### Chores

- Line to long
([`fd9e506`](https://github.com/well-it-wasnt-me/SDSavior/commit/fd9e5069ff4e5c79ca96b8e2ab320de1d3c36bd9))

### Testing

- **coverage**: Extended test coverage
([`3f0b76a`](https://github.com/well-it-wasnt-me/SDSavior/commit/3f0b76a922c79593ce877fbf91df6e27dd98bff5))


## v1.0.1 (2026-03-04)

### Bug Fixes

- Harden open/recovery edge cases
([`f55a6f2`](https://github.com/well-it-wasnt-me/SDSavior/commit/f55a6f2a742f43790f0f8ad866a2d12b88963cc8))


## v1.0.0 (2026-02-28)

- Initial Release
Expand Down
29 changes: 29 additions & 0 deletions docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,32 @@ Dataclass storing persisted pointer state:
```python
from sdsavior import SDSavior, MetaState
```

## `AsyncSDSavior`

Async wrapper around `SDSavior`, imported from `sdsavior.aio`.

```python
from sdsavior.aio import AsyncSDSavior
```

### Constructor

`AsyncSDSavior(data_path, meta_path, capacity_bytes, *, executor=None, **kwargs)`

- `executor`: optional `ThreadPoolExecutor`. Defaults to a dedicated single-thread executor for thread safety.
- `**kwargs`: passed through to `SDSavior`.

### Async Lifecycle

- `await open()` / `await close()`
- Async context manager: `async with AsyncSDSavior(...) as rb:`

### Async Data Operations

- `await append(obj) -> int`: append and return sequence number.
- `async for seq, ts_ns, obj in rb.iter_records(from_seq=None)`: async iteration yielding records in chunks (avoids full list materialization).
- `async for seq, ts_ns, obj in rb.from_seq(seq)`: convenience filter by sequence.
- `await export_jsonl(out_path, from_seq=None)`: export to JSONL.
- `await flush()`: flush pending coalesced records.
- `rb.ring`: access the underlying `SDSavior` instance.
31 changes: 31 additions & 0 deletions docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,3 +49,34 @@ with SDSavior("data.ring", "data.meta", 8 * 1024 * 1024) as rb:
- `recover_scan_limit_bytes=None` (default): scan up to capacity during recovery.

Use `fsync_data=True` when stronger durability is required and throughput tradeoffs are acceptable.

## Async Usage

For asyncio applications, use the async wrapper:

```python
from sdsavior.aio import AsyncSDSavior

async with AsyncSDSavior("data.ring", "data.meta", 8 * 1024 * 1024) as rb:
await rb.append({"event": "boot"})

async for seq, ts_ns, obj in rb.iter_records():
print(seq, obj)

# Filter by sequence
async for seq, ts_ns, obj in rb.from_seq(100):
print(seq, obj)

# Export
await rb.export_jsonl("out/export.jsonl")
```

`AsyncSDSavior` uses a dedicated single-thread executor to safely wrap the non-thread-safe `SDSavior`. A custom executor can be provided:

```python
from concurrent.futures import ThreadPoolExecutor

executor = ThreadPoolExecutor(max_workers=1)
async with AsyncSDSavior("data.ring", "data.meta", 8 * 1024 * 1024, executor=executor) as rb:
await rb.append({"event": "boot"})
```
4 changes: 3 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "hatchling.build"

[project]
name = "sdsavior"
version = "0.1.0"
version = "1.0.1"
description = "Crash-recoverable memory-mapped ring buffer for JSON records (SD-card friendly-ish)"
readme = "README.md"
requires-python = ">=3.11"
Expand Down Expand Up @@ -51,3 +51,5 @@ select = ["E", "F", "I", "UP", "B"]
[tool.pytest.ini_options]
addopts = "--cov=src/sdsavior --cov-report=term-missing --cov-fail-under=90"
testpaths = ["tests"]
pythonpath = ["src"]
asyncio_mode = "auto"
Loading
Loading