Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .flake8
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[flake8]
max_line_length = 88
extend_ignore = E203, E501, W503
# ^ Black-compatible
# E203, E501 and W503 have edge cases handled by black
exclude =
build
dist
.eggs
docs/conf.py
23 changes: 14 additions & 9 deletions .github/agents/copilot.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,25 +19,30 @@ The main client class is `FlexMeasuresClient` in `src/flexmeasures_client/client
## Running Tests

```bash
pip install -e ".[testing]"
python3 -m pytest tests/test_client.py -q
uv sync --group test
uv run poe test
```

## Linting

Always fix linting before pushing. Run individually if `pre-commit` is unavailable:
Always fix linting before pushing. Run via pre-commit (preferred):

```bash
pip install black isort flake8
black src/ tests/
isort src/ tests/
flake8 src/ tests/
uv run poe lint
```

Or via pre-commit (preferred):
Or run individually using `uv run`:

```bash
pip install pre-commit && pre-commit run --all-files
uv run black src/ tests/
uv run isort src/ tests/
uv run flake8 src/ tests/
```

To install pre-commit as a standalone tool:

```bash
uv tool install pre-commit && pre-commit run --all-files
```

## Coding Patterns
Expand Down
50 changes: 38 additions & 12 deletions .github/agents/test-specialist.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,45 +55,71 @@ When writing tests for this project, follow these patterns:

## Code Quality and Linting

Before finalizing tests, always apply the project's code quality checks:
Before finalizing tests, always apply the project's code quality checks.

### Poe Tasks
The project uses [poethepoet](https://poethepoet.natn.io/) for common tasks. Prefer these over running tools directly:

```bash
uv run poe lint # Run all pre-commit hooks on all files
uv run poe type-check # Run mypy on files with type hints
uv run poe test # Run the full test suite
uv run poe test-no-s2 # Run tests excluding S2
uv run poe test-s2 # Run S2 tests only
```

### Running Pre-commit Hooks
The project uses `.pre-commit-config.yaml` to enforce code quality standards. Always run pre-commit hooks before committing:
The project uses `.pre-commit-config.yaml` to enforce code quality standards. `pre-commit` is included in the `dev` dependency group, so no separate installation is needed:

```bash
# Install pre-commit (if not already installed)
pip install pre-commit
# If you have not installed pre-commit already:
uv tool install pre-commit

# Run all pre-commit hooks on all files
pre-commit run --all-files
uv run pre-commit run --all-files

# Or via the poe task
uv run poe lint
```

### Pre-commit Hooks in This Project
The following hooks are configured:
- **trailing-whitespace**: Removes trailing whitespace
- **end-of-file-fixer**: Ensures files end with a newline
- **check-added-large-files**: Prevents committing large files
- **check-ast**: Validates Python syntax
- **check-json**: Validates JSON files
- **check-merge-conflict**: Detects merge conflict markers
- **check-xml**: Validates XML files
- **check-yaml**: Validates YAML files
- **debug-statements**: Detects debug statements
- **end-of-file-fixer**: Ensures files end with a newline
- **requirements-txt-fixer**: Sorts requirements files
- **mixed-line-ending**: Normalises line endings
- **isort**: Sorts Python imports
- **black**: Formats Python code (line length, style)
- **flake8**: Checks Python code style and quality
- **mypy**: Performs static type checking

### Type Checking
Mypy is run separately from pre-commit via the poe task:

```bash
uv run poe type-check
```

When mypy reports errors:
- Add type hints where needed
- Use `# type: ignore` comments sparingly for known issues

### Fixing Linting Issues
When pre-commit hooks fail:
1. Review the output to understand what failed
2. Many hooks auto-fix issues (black, isort, end-of-file-fixer) - re-run to verify
2. Many hooks auto-fix issues (black, isort, end-of-file-fixer) re-run to verify
3. For manual fixes (flake8 errors):
- Address unused imports, undefined names, line too long, etc.
- Run pre-commit again to verify fixes
4. For mypy type errors:
- Add type hints where needed
- Use `# type: ignore` comments sparingly for known issues

### Best Practices
- Run pre-commit hooks frequently during development
- Run `uv run poe lint` frequently during development
- Fix linting issues before requesting code review
- Keep test code clean and well-formatted like production code
- Ensure all hooks pass before pushing changes
144 changes: 46 additions & 98 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,37 +22,29 @@ concurrency:
cancel-in-progress: true

jobs:
prepare:
check:
runs-on: ubuntu-latest
outputs:
wheel-distribution: ${{ steps.wheel-distribution.outputs.path }}
name: Check (on Python 3.12)
steps:
- uses: actions/checkout@v3
with: { fetch-depth: 0 } # deep clone for setuptools-scm
- uses: actions/setup-python@v4
id: setup-python
with: { python-version: "3.11" }
- name: Run static analysis and format checkers
run: pipx run pre-commit run --all-files --show-diff-on-failure
- name: Build package distribution files
run: >-
pipx run --python '${{ steps.setup-python.outputs.python-path }}'
tox -e clean,build
- name: Record the path of wheel distribution
id: wheel-distribution
run: echo "path=$(ls dist/*.whl)" >> $GITHUB_OUTPUT
- name: Store the distribution files for use in other stages
# `tests` and `publish` will use the same pre-built distributions,
# so we make sure to release the exact same package that was tested
uses: actions/upload-artifact@v4
- uses: actions/checkout@v6
- name: "Set up Python"
uses: actions/setup-python@v6
with:
name: python-distribution-files
path: dist/
retention-days: 1
python-version-file: ".python-version"
- name: Install uv
uses: astral-sh/setup-uv@v7
with:
enable-cache: true
cache-dependency-glob: '.pre-commit-config.yaml'
version: "0.10.9"
- name: Install dev dependencies
run: uv sync --only-group dev --frozen
- uses: tox-dev/action-pre-commit-uv@v1

test:
needs: prepare
needs: check
strategy:
fail-fast: false
matrix:
python:
- "3.10"
Expand All @@ -61,33 +53,24 @@ jobs:
platform:
- ubuntu-latest
- macos-latest
# - windows-latest
runs-on: ${{ matrix.platform }}
name: "Test (${{ matrix.platform }} on Python ${{ matrix.python }})"
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
id: setup-python
- uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v7
with:
enable-cache: true
version: "0.10.9"
python-version: ${{ matrix.python }}
- name: Retrieve pre-built distribution files
uses: actions/download-artifact@v4
with:
name: python-distribution-files
path: dist/
- name: Install dependencies
run: uv sync --frozen --group test
- name: Run tests
run: >-
pipx run --python '${{ steps.setup-python.outputs.python-path }}'
tox --installpkg '${{ needs.prepare.outputs.wheel-distribution }}'
-- -rFEx --durations 10 --color yes # pytest args
- name: Run tests (s2)
run: >-
pipx run --python '${{ steps.setup-python.outputs.python-path }}'
tox -e s2 --installpkg '${{ needs.prepare.outputs.wheel-distribution }}'
-- -rFEx --durations 10 --color yes # pytest args
run: uv run pytest tests --ignore=tests/s2 -rFEx --durations 10 --color yes --cov
- name: Show coverage report
run: pipx run coverage report
run: uv run coverage report
- name: Generate coverage report
run: pipx run coverage lcov -o coverage.lcov
run: uv run coverage lcov -o coverage.lcov
- name: Upload partial coverage report
uses: coverallsapp/github-action@v2
with:
Expand All @@ -97,8 +80,25 @@ jobs:
flag-name: ${{ matrix.platform }} - py${{ matrix.python }}
parallel: true

test-s2:
needs: check
runs-on: ubuntu-latest
name: "Test S2 (on Python 3.12)"
steps:
- uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v7
with:
enable-cache: true
version: "0.10.9"
python-version-file: ".python-version"
- name: Install dependencies
run: uv sync --frozen --extra s2 --group test
- name: Run S2 tests
run: uv run pytest tests/s2 -rFEx --durations 10 --color yes

finalize:
needs: test
needs: [ test, test-s2]
runs-on: ubuntu-latest
steps:
- name: Coveralls Finished
Expand All @@ -108,55 +108,3 @@ jobs:
parallel-finished: true
github-token: ${{ secrets.GITHUB_TOKEN }}
- run: echo "Finished checks"

publish:
needs: finalize
if: ${{ github.event_name == 'push' && contains(github.ref, 'refs/tags/v') }}
runs-on: ubuntu-latest
environment:
name: release
url: https://pypi.org/project/flexmeasures-client/
permissions:
id-token: write
steps:
- uses: actions/checkout@v3
with: {fetch-depth: 0} # deep clone for setuptools-scm
- uses: actions/setup-python@v4
with: { python-version: "3.11" }
- name: Retrieve pre-built distribution files
uses: actions/download-artifact@v4
with:
name: python-distribution-files
path: dist/
- name: Publish Package
uses: pypa/gh-action-pypi-publish@release/v1
# run: pipx run tox -e publish
- name: Publish release on GitHub
uses: softprops/action-gh-release@v2
with:
generate_release_notes: true
tag_name: ${{ inputs.custom_version || github.ref_name }}
env:
GITHUB_TOKEN: ${{ secrets.GH_RELEASE_PAT }}

# test-publish:
# needs: finalize
# if: ${{ github.event_name == 'push' && contains(github.ref, 'refs/tags/test') }}
# runs-on: ubuntu-latest
# environment:
# name: testpypi
# url: https://test.pypi.org/project/flexmeasures-client/
# permissions:
# id-token: write
# steps:
# - uses: actions/checkout@v3
# - uses: actions/setup-python@v4
# with: {python-version: "3.11"}
# - name: Retrieve pre-built distribution files
# uses: actions/download-artifact@v3
# with: {name: python-distribution-files, path: dist/}
# - name: Publish Package
# uses: pypa/gh-action-pypi-publish@release/v1
# with:
# repository-url: https://test.pypi.org/legacy/
# # run: pipx run tox -e publish
69 changes: 69 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
name: release

on:
push:
tags:
- 'v[0-9]+\.[0-9]+\.[0-9]+'
- 'v[0-9]+\.[0-9]+\.[0-9]+\.dev[0-9]+'

permissions:
contents: write

jobs:
release-build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Install uv
uses: astral-sh/setup-uv@v7
with:
version: "0.10.9"
python-version-file: ".python-version"
enable-cache: true

- name: Build release distributions
run: uv build

- name: Upload distributions
uses: actions/upload-artifact@v4
with:
name: release-dists
path: dist/

pypi-publish:
runs-on: ubuntu-latest
needs: release-build
permissions:
# IMPORTANT: this permission is mandatory for trusted publishing
id-token: write

# Dedicated environments with protections for publishing are strongly recommended.
# For more information, see: https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment#deployment-protection-rules
environment:
name: pypi
url: https://pypi.org/project/flexmeasures-client/

steps:
- name: Retrieve release distributions
uses: actions/download-artifact@v4
with:
name: release-dists
path: dist/

- name: Publish release distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
packages-dir: dist/

github-release:
runs-on: ubuntu-latest
needs: pypi-publish
steps:
- name: Publish release on GitHub
uses: softprops/action-gh-release@v2
with:
generate_release_notes: true
tag_name: ${{ inputs.custom_version || github.ref_name }}
env:
GITHUB_TOKEN: ${{ secrets.GH_RELEASE_PAT }}
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,6 @@ MANIFEST
# Per-project virtualenvs
.venv*/
.conda*/
.python-version

venv/*
log
Expand Down
3 changes: 0 additions & 3 deletions .isort.cfg

This file was deleted.

Loading
Loading