Feature 584 enhance code linting application test

.bumpversion.cfg

@@ -1,14 +1,14 @@
 [bumpversion]
 current_version = 0.16.0
 parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)((?P<release>(a|na))+(?P<build>\d+))?
-serialize = 
+serialize =
 	{major}.{minor}.{patch}{release}{build}
 	{major}.{minor}.{patch}
 
 [bumpversion:part:release]
 first_value = a
 optional_value = na
-values = 
+values =
 	a
 	na
 

.github/ISSUE_TEMPLATE/issue_template_bug.md

@@ -10,7 +10,7 @@ assignees: ''
 ## Description of the issue
 
 Describe the problem in as much detail as possible.
-Focus on the expected and current behavior. 
+Focus on the expected and current behavior.
 If necessary, create a screenshot and insert below.
 
 ## Steps to Reproduce
@@ -23,9 +23,9 @@ If necessary, create a screenshot and insert below.
 Describe possible ideas for solution and evaluate advantages and disadvantages.
 
 ## Context and Environment
-* Version used: 
-* Operating system: 
-* Environment setup and (python) version: 
+* Version used:
+* Operating system:
+* Environment setup and (python) version:
 
 ## Workflow checklist
 - [ ] I have checked [the documentation](https://open-mastr.readthedocs.io/en/latest/) and confirmed this issue is not already addressed there.

.github/ISSUE_TEMPLATE/issue_template_feature.md

@@ -9,7 +9,7 @@ assignees: ''
 
 ## Description of the issue
 
-Describe the problem in as much detail as possible. 
+Describe the problem in as much detail as possible.
 
 ## Ideas of solution
 

.github/ISSUE_TEMPLATE/issue_template_release.md

@@ -15,7 +15,7 @@ assignees: ''
 ## Preparation
 - [ ] 🐙 Create a `Draft GitHub Release` with the release version number `vx.x.x` as title
 - [ ] Merge all open Pull Requests to `develop`
-- [ ] Run tests locally with `pytest` and apply linting with `pre-commit run -a` 
+- [ ] Run tests locally with `pytest` and apply linting with `pre-commit run -a`
 ## Create a `release` branch
 - [ ] Checkout `develop` and branch with `git checkout -b release-vx.x.x`
 - [ ] Update version for test release with `bump2version --current-version current_version> --new-version <new_version> patch`

.github/ISSUE_TEMPLATE/issue_template_user_kudos.md

@@ -16,10 +16,10 @@ Please, insert your information below - fill out at minimum affiliation :purple_
 
 :pencil2: **Spaces** and the following special characters are allowed: @ ? ! | . , : ; - _ [ / ( ) \ ] § $ % & = + < >
 
-family-names: 
-given-names: 
-alias: 
-affiliation: 
-orcid: 
+family-names:
+given-names:
+alias:
+affiliation:
+orcid:
 
 Thank you!

.github/workflows/extend_user_cff.yml

@@ -8,7 +8,7 @@ on:
 permissions:
   contents: write
   pull-requests: write
-  
+
 jobs:
   create-pr-to-add-new-user:
     if: contains(join(github.event.issue.labels.*.name, ','), 'user')
@@ -67,10 +67,10 @@ jobs:
             orcid: ${{env.ORCID}}
         EOF
 
-        
+
     - name: Print updated USERS.cff
       run: cat USERS.cff
-      
+
     - name: Create Pull Request
       uses: peter-evans/create-pull-request@v6
       with:
@@ -82,7 +82,7 @@ jobs:
           Add new user to USERS.cff
         body: |
           This pull request updates the USERS.cff file with new user information extracted from issue #${{ github.event.issue.number }}
-          
+
           Closes #${{ github.event.issue.number }}
 
           Many thanks @${{ github.actor }}!

.github/workflows/pre-commit.yml

@@ -0,0 +1,32 @@
+name: Pre-commit Checks
+
+on:
+  pull_request:
+    branches:
+      - production
+      - develop
+  push:
+    branches:
+      - production
+      - develop
+
+jobs:
+  pre-commit:
+    runs-on: ubuntu-latest
+    name: Run pre-commit hooks
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.11'
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install pre-commit
+
+      - name: Run pre-commit
+        run: pre-commit run --all-files --show-diff-on-failure

.github/workflows/pypi-publish.yml

@@ -35,4 +35,4 @@ jobs:
       if: startsWith(github.ref, 'refs/tags')
       uses: pypa/gh-action-pypi-publish@release/v1
       with:
-        password: ${{ secrets.PYPI }}
+        password: ${{ secrets.PYPI }}

.github/workflows/test-pypi-publish.yml

@@ -36,4 +36,4 @@ jobs:
       with:
         password: ${{ secrets.PYPI_TEST }}
         repository-url: https://test.pypi.org/legacy/
-        verbose: true
+        verbose: true

.pre-commit-config.yaml

@@ -1,6 +1,43 @@
+# Global excludes - apply to all hooks
+exclude: ^(build/|dist/|.*\.egg-info/|\.tox/|\.pytest_cache/|\.mypy_cache/|__pycache__/|postprocessing/|open_mastr/soap_api/|scripts/)
+
 repos:
-  - repo: https://github.com/psf/black
-    rev: 22.6.0
+  # Base pre-commit hooks for common issues
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v6.0.0
     hooks:
-      - id: black
-        language_version: python3.11
+      - id: trailing-whitespace
+        exclude: \.ipynb$
+      - id: end-of-file-fixer
+        exclude: \.ipynb$
+      - id: check-yaml
+      - id: check-added-large-files
+        args: ['--maxkb=1000']
+      - id: check-json
+      - id: check-toml
+      - id: check-merge-conflict
+      - id: debug-statements
+      - id: check-case-conflict
+      - id: check-docstring-first
+      - id: mixed-line-ending
+        args: ['--fix=lf']
+
+  # Ruff - fast, comprehensive linter and formatter (replaces flake8, isort, pylint, pyupgrade, black, pydocstyle)
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: v0.14.9
+    hooks:
+      - id: ruff
+        args: [--fix, --exit-non-zero-on-fix]
+      - id: ruff-format
+
+  # mypy - static type checking
+  - repo: https://github.com/pre-commit/mirrors-mypy
+    rev: v1.19.1
+    hooks:
+      - id: mypy
+        additional_dependencies:
+          - types-requests
+          - types-PyYAML
+          - types-tqdm
+        args: [--ignore-missing-imports, --show-error-codes]
+        exclude: ^(tests/|scripts/|docs/|postprocessing/)

.readthedocs.yml

@@ -12,4 +12,4 @@ mkdocs:
 
 python:
   install:
-  - requirements: docs/requirements.txt
+  - requirements: docs/requirements.txt

CHANGELOG.md

@@ -1,7 +1,7 @@
 # Changelog
 
 All notable changes to this project will be documented in this file.
-For each version important additions, changes and removals are listed here. 
+For each version important additions, changes and removals are listed here.
 
 The format is inspired from [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
 and the versioning aims to respect [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
@@ -18,6 +18,8 @@ and the versioning aims to respect [Semantic Versioning](http://semver.org/spec/
 ### Added
 - Add partial bulk download
   [#652](https://github.com/OpenEnergyPlatform/open-MaStR/pull/652)
+- Modernize linting setup and add pre-commit checks in CI
+  [#671](https://github.com/OpenEnergyPlatform/open-MaStR/pull/671)
 ### Changed
 - Updates the system_catalog dict with missing Einheittyp values
   [#653](https://github.com/OpenEnergyPlatform/open-MaStR/pull/653)
@@ -101,7 +103,7 @@ and the versioning aims to respect [Semantic Versioning](http://semver.org/spec/
 - Fixed missing call to gen_url in case first bulk download fails as xml file for today is not yet available [#534](https://github.com/OpenEnergyPlatform/open-MaStR/pull/534)
 - Repair links in the documentation page [#536](https://github.com/OpenEnergyPlatform/open-MaStR/pull/536)
 
-## [v0.14.3] Fix Pypi Release - 2024-04-24 
+## [v0.14.3] Fix Pypi Release - 2024-04-24
 ### Added
 - Add new table `changed_dso_assignment` [#510](https://github.com/OpenEnergyPlatform/open-MaStR/pull/510)
 - Add deprecation warning for `MaStRMirror` and `MaStRDownload` [#492](https://github.com/OpenEnergyPlatform/open-MaStR/pull/492)
@@ -265,27 +267,27 @@ Additionally, a new datasource was identified and can be used: the xml bulk down
 The API was updated to the newest version and the data model was adapted.
 
 ### Added
-- The class :class:`open_mastr.mastr.Matr` 
-  was introduced as the entrypoint for users, the API download was included in this entrypoint 
+- The class :class:`open_mastr.mastr.Matr`
+  was introduced as the entrypoint for users, the API download was included in this entrypoint
   [#203](https://github.com/OpenEnergyPlatform/open-MaStR/issues/203)
 - A method for downloading and parsing the xml dump from the MaStR website
-  was implemented 
+  was implemented
   [#202](https://github.com/OpenEnergyPlatform/open-MaStR/issues/202)
-- New data classes and attributes were introduced to orm.py 
+- New data classes and attributes were introduced to orm.py
 [#209](https://github.com/OpenEnergyPlatform/open-MaStR/issues/209)
 - The documentation page was updated
-- Unit tests were created 
+- Unit tests were created
 [#207](https://github.com/OpenEnergyPlatform/open-MaStR/issues/207)
-- A CI pipeline was introduced 
+- A CI pipeline was introduced
 [#208](https://github.com/OpenEnergyPlatform/open-MaStR/issues/208)
-- The metadata was updated 
+- The metadata was updated
 [#219](https://github.com/OpenEnergyPlatform/open-MaStR/issues/219)
 
 
 ## [v0.10.0] Unreleased - Refactoring - 2020-10-08
-A complete refactoring took place! Downloading data was entirely changed; introducing layers of code and removing 
-duplicated code while more of less following DRY. 
-Moreover, post-processing was changed to be more accessible and easier to execute. For example, docker now helps to 
+A complete refactoring took place! Downloading data was entirely changed; introducing layers of code and removing
+duplicated code while more of less following DRY.
+Moreover, post-processing was changed to be more accessible and easier to execute. For example, docker now helps to
 spin up a database container.
 The documention on RTD was extended, update and improved to be more helpful for new users.
 Read more about the details:
@@ -295,24 +297,24 @@ Read more about the details:
 - added documentation for ReadTheDocs
 - improved parallel download
 - merged all stale branches
-- The class :class:`open_mastr.soap_api.mirror.MaStRMirror` 
-  was introduced for mirroring MaStR data with latest updates 
+- The class :class:`open_mastr.soap_api.mirror.MaStRMirror`
+  was introduced for mirroring MaStR data with latest updates
   [#149](https://github.com/OpenEnergyPlatform/open-MaStR/issues/149)
 - Introduce project home `~/.open-MaStR/config/` [#120](https://github.com/OpenEnergyPlatform/open-MaStR/issues/120)
 - Documentation of post-processing [#117](https://github.com/OpenEnergyPlatform/open-MaStR/issues/117)
 - Updated documentation of downloading data
   [#124](https://github.com/OpenEnergyPlatform/open-MaStR/issues/124) which is harmonized with the other parts of docs
   and with GitHubs README [#135](https://github.com/OpenEnergyPlatform/open-MaStR/issues/135)
-- Local execution of post-processing now possible, optionally in dockered database 
+- Local execution of post-processing now possible, optionally in dockered database
   [#116](https://github.com/OpenEnergyPlatform/open-MaStR/issues/116)
 - Post-processing adapted to CSV data from :class:`open_mastr.soap_api.mirror.MaStRMirror`
   [#172](https://github.com/OpenEnergyPlatform/open-MaStR/issues/172)
 - Tests for changed download code are added [#131](https://github.com/OpenEnergyPlatform/open-MaStR/issues/131)
-- Metadata added for raw data as frictionless data package 
+- Metadata added for raw data as frictionless data package
   [#160](https://github.com/OpenEnergyPlatform/open-MaStR/issues/160)
 - Suffix columns instead of deferring in database CSV export
   [#157](https://github.com/OpenEnergyPlatform/open-MaStR/issues/157)
-- Code examples added for :class:`open_mastr.soap_api.mirror.MaStRMirror` explaining basic use of 
+- Code examples added for :class:`open_mastr.soap_api.mirror.MaStRMirror` explaining basic use of
   mirroring database [#164](https://github.com/OpenEnergyPlatform/open-MaStR/issues/164)
 - CSV file reader for MaStR raw data added
   [#181](https://github.com/OpenEnergyPlatform/open-MaStR/issues/181)
@@ -329,27 +331,27 @@ Read more about the details:
 - The CHANGELOG is now included in the documentation
 
 ### Changed
-- Download of raw data has entirely been refactored. A 
-  [python wrapper](https://open-mastr.readthedocs.io/en/latest/download.html#mastr-api-wrapper) for querying 
+- Download of raw data has entirely been refactored. A
+  [python wrapper](https://open-mastr.readthedocs.io/en/latest/download.html#mastr-api-wrapper) for querying
   the MaStR API was introduced
   [#83](https://github.com/OpenEnergyPlatform/open-MaStR/issues/83)
-- Based on that, for bulk data download, 
-  [MaStRDownload](https://open-mastr.readthedocs.io/en/latest/download.html#bulk-download) provides handy query 
+- Based on that, for bulk data download,
+  [MaStRDownload](https://open-mastr.readthedocs.io/en/latest/download.html#bulk-download) provides handy query
   functions for power unit data
   [#86](https://github.com/OpenEnergyPlatform/open-MaStR/issues/86). See also
   [#128](https://github.com/OpenEnergyPlatform/open-MaStR/issues/128)
 - configuration through config filen in `~/.open-MaStR/config/` with less hard-coded parameters in source files
-  [#120](https://github.com/OpenEnergyPlatform/open-MaStR/issues/120), 
+  [#120](https://github.com/OpenEnergyPlatform/open-MaStR/issues/120),
   [#112](https://github.com/OpenEnergyPlatform/open-MaStR/issues/112)
 - move code into one package named `open_mastr` [#123](https://github.com/OpenEnergyPlatform/open-MaStR/issues/123)
 - Switch to GitHub Actions for CI instead of Travis [#143](https://github.com/OpenEnergyPlatform/open-MaStR/issues/143)
-- Fixed unexpected line breaks during CSV export that corrupted data 
+- Fixed unexpected line breaks during CSV export that corrupted data
   [#170](https://github.com/OpenEnergyPlatform/open-MaStR/issues/170)
-- Filtering of duplicates in MaStR data (see 
+- Filtering of duplicates in MaStR data (see
   `MaStR help <https://www.marktstammdatenregister.de/MaStRHilfe/subpages/statistik.html>`_) got changed to filter
   units by leading three characters and select only directly entered data
   [#180](https://github.com/OpenEnergyPlatform/open-MaStR/issues/180)
-- Generalize CSV reading function 
+- Generalize CSV reading function
   [#188](https://github.com/OpenEnergyPlatform/open-MaStR/issues/188)
 
 ### Removed
@@ -363,7 +365,7 @@ Read more about the details:
 - tests
 - setup.py file
 - added update function (based on latest timestamp in powerunits csv)
-- added wind functions 
+- added wind functions
   * only download power units for wind to avoid massive download
   * changed : process units wind ("one-click solution")
 - added loop to retry failed power unit downloads, currently one retry

CITATION.cff

@@ -37,11 +37,11 @@ authors:
   - family-names: 'Krämer'
     given-names: "Kevin"
     alias: "pt-kkraemer"
-    affiliation: "ProjectTogether gGmbH"  
+    affiliation: "ProjectTogether gGmbH"
 title: "open-MaStR"
 type: software
 license: AGPL-3.0
 version: 0.16.0
-doi: 
+doi:
 date-released: 2025-11-26
 url: "https://github.com/OpenEnergyPlatform/open-MaStR/"