Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/format-ruff.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
persist-credentials: false

- name: Install uv
uses: astral-sh/setup-uv@v4
uses: astral-sh/setup-uv@e4db8464a088ece1b920f60402e813ea4de65b8f # v4

Check warning

Code scanning / zizmor

detects commit SHAs that don't match their version comment tags Warning

detects commit SHAs that don't match their version comment tags
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
enable-cache: true
Expand All @@ -31,4 +31,4 @@
run: uvx ruff format .

- name: Commit changes
uses: stefanzweifel/git-auto-commit-action@v5
uses: stefanzweifel/git-auto-commit-action@b863ae1933cb653a53c021fe36dbb774e1fb9403 # v5
2 changes: 1 addition & 1 deletion .github/workflows/lint-ruff.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
persist-credentials: false

- name: Install uv
uses: astral-sh/setup-uv@v4
uses: astral-sh/setup-uv@e4db8464a088ece1b920f60402e813ea4de65b8f # v4

Check warning

Code scanning / zizmor

detects commit SHAs that don't match their version comment tags Warning

detects commit SHAs that don't match their version comment tags
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
enable-cache: true
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,10 @@
persist-credentials: false

- name: Install uv
uses: astral-sh/setup-uv@v4
uses: astral-sh/setup-uv@e4db8464a088ece1b920f60402e813ea4de65b8f # v4

Check warning

Code scanning / zizmor

detects commit SHAs that don't match their version comment tags Warning

detects commit SHAs that don't match their version comment tags
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
enable-cache: true
enable-cache: false
cache-dependency-glob: uv.lock

- name: Set up Python
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/security-zizmor.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
persist-credentials: false

- name: Install uv
uses: astral-sh/setup-uv@v4
uses: astral-sh/setup-uv@e4db8464a088ece1b920f60402e813ea4de65b8f # v4

Check warning

Code scanning / zizmor

detects commit SHAs that don't match their version comment tags Warning

detects commit SHAs that don't match their version comment tags
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
enable-cache: true
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
persist-credentials: false

- name: Install uv
uses: astral-sh/setup-uv@v4
uses: astral-sh/setup-uv@e4db8464a088ece1b920f60402e813ea4de65b8f # v4

Check warning

Code scanning / zizmor

detects commit SHAs that don't match their version comment tags Warning test

detects commit SHAs that don't match their version comment tags
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
enable-cache: true
Expand Down
156 changes: 156 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

This is a Python implementation of the Pipeline pattern, distributed as `thecodecrate-pipeline`. The package allows composing sequential stages into reusable, immutable pipelines for processing data. It's inspired by the PHP League Pipeline package.

## Development Commands

### Environment Setup
```bash
# The project uses uv for dependency management
# Dependencies are managed in pyproject.toml under [dependency-groups]
uv sync
```

### Testing
```bash
# Run all tests
pytest

# Run tests with coverage
pytest --cov

# Run a specific test file
pytest tests/test_pipeline.py

# Run a specific test function
pytest tests/test_pipeline.py::test_function_name
```

### Code Quality
```bash
# Format code with ruff
ruff format .

# Lint with ruff
ruff check .

# Fix auto-fixable linting issues
ruff check --fix .

# Format with black (line length: 79)
black .

# Type checking is configured with strict mode in pyrightconfig.json
# Type check manually: pyright (if installed)
```

### Documentation
```bash
# Build documentation locally
mkdocs serve

# Documentation is built with mkdocs-material and auto-generates API docs
# from docstrings using mkdocstrings-python
```

### Version Management
```bash
# Bump version (uses bumpver)
bumpver update --patch # 1.26.0 -> 1.26.1
bumpver update --minor # 1.26.0 -> 1.27.0
bumpver update --major # 1.26.0 -> 2.0.0

# Note: bumpver automatically commits and tags, but does NOT push
```

## Architecture

### Core Concepts

The codebase implements a pipeline pattern with three main abstractions:

1. **Stage**: A callable unit that transforms input to output (`StageInterface[T_in, T_out]`)
2. **Pipeline**: An immutable chain of stages (`PipelineInterface[T_in, T_out]`)
3. **Processor**: Controls how stages are executed (`ProcessorInterface[T_in, T_out]`)

### Directory Structure

```
src/
├── _lib/ # Internal implementation
│ ├── pipeline/ # Core pipeline implementation
│ │ ├── pipeline.py # Main Pipeline class
│ │ ├── pipeline_factory.py # Factory for building pipelines
│ │ ├── processor.py # Base Processor class
│ │ ├── stage.py # Base Stage class
│ │ ├── processors/ # Built-in processors
│ │ │ ├── chained_processor.py
│ │ │ └── ...
│ │ └── traits/ # Mixins (Clonable, ActAsFactory)
│ └── processors/ # Additional processor implementations
│ ├── chained_processor/ # Processor that chains stages
│ └── interruptible_processor/ # Processor with interruption support
└── thecodecrate_pipeline/ # Public API package
├── __init__.py # Re-exports from _lib
├── processors/ # Public processor exports
└── types/ # Public type exports
```

### Key Design Patterns

**Immutability**: Pipelines use copy-on-write semantics. The `pipe()` method creates a new pipeline instance with the added stage, preserving the original pipeline.

**Traits System**: The codebase uses a trait-like pattern with mixins:
- `Clonable`: Provides shallow cloning capability
- `ActAsFactory`: Enables objects to act as factories for creating instances

**Interface Segregation**: Each core concept has an interface (`*Interface`) and implementation, enabling custom implementations while maintaining type safety.

**Async-First**: All processing is async (`async def process(...)`). The processor handles both sync and async callables transparently using `inspect.isawaitable()`.

### Type System

The codebase uses generic type variables for type safety:
- `T_in`: Input type to a stage or pipeline
- `T_out`: Output type from a stage or pipeline (defaults to `T_in`)

Stages can transform types:
```python
Pipeline[int, str] # Takes int, returns str
StageInterface[int, int] # Takes int, returns int
```

### Processing Flow

1. A Pipeline is created with stages (either via `.pipe()` or declaratively)
2. When `.process(payload)` is called, the pipeline:
- Instantiates stages if needed (converts classes to instances)
- Delegates to the processor's `.process()` method
- The processor iterates through stages, passing output to next stage
3. Processors can customize execution (e.g., ChainedProcessor for error handling)

### Stream Processing

Pipelines support processing `AsyncIterator` streams, allowing real-time data transformation where each stage can yield results consumed immediately by the next stage.

### PipelineFactory

Because pipelines are immutable, `PipelineFactory` provides mutable stage collection during composition. It builds the final immutable pipeline via `.build()`.

## Testing Notes

- Test files use stub classes in `tests/stubs/` for consistent test fixtures
- Tests are async-aware (configured via `pytest.ini` with `pytest-asyncio`)
- Mock stages implement `StageInterface` for type safety

## Python Version

Requires Python 3.13+ (specified in pyproject.toml).

## Package Distribution

Built with `hatchling`. The wheel includes both `thecodecrate_pipeline` (public API) and `_api` packages (internal). The public package re-exports symbols from `_lib`.
17 changes: 0 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -252,20 +252,3 @@ except ZeroDivisionError as e:
# Handle the exception.
pass
```

## Architecture

All code is located in the `src/_api` directory and follows the Static-Plugin Design (SPD) pattern. This architectural approach ensures:

- Clear separation of concerns through static plugins
- Full static analysis capabilities
- Predictable code execution paths
- Zero runtime overhead

The project utilizes class composition rather than dynamic plugins, making the codebase easier to maintain and debug while preserving extensibility.

### Project's Specs and Guidelines

- [spec2: Static-Plugin Design](https://github.com/thecodecrate/guidelines/blob/main/specs/spec-002--static-plugin-design/README.md)
- [spec3: SPD Manifest Files](https://github.com/thecodecrate/guidelines/blob/main/specs/spec-003--spd-manifest-files/README.md)
- [spec4: SPD Naming Convention](https://github.com/thecodecrate/guidelines/blob/main/specs/spec-004--spd-naming-convention/README.md)
2 changes: 1 addition & 1 deletion tests/test_linting.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ def test_zizmor_compliance():
"""Test to ensure codebase is compliant with Zizmor linting."""

result = subprocess.run(
["uvx", "zizmor", ".github/workflows/"],
["uvx", "zizmor", "--min-severity", "low", ".github/workflows/"],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
Expand Down
Loading