Skip to content

OpenSTEF Meta V0.1#771

Open
Lars800 wants to merge 114 commits intorelease/v4.0.0from
research/HybridForecaster2.0
Open

OpenSTEF Meta V0.1#771
Lars800 wants to merge 114 commits intorelease/v4.0.0from
research/HybridForecaster2.0

Conversation

@Lars800
Copy link
Collaborator

@Lars800 Lars800 commented Nov 28, 2025

Introducting OpenSTEF-Meta, a one-stop-shop for all things meta learning.

This sub-package introduces four common meta learning algorithms:

Residual Forecaster

  • 2 stage Forecasting Model
  • Primary model is fitted as usual
  • Secondary model fitted on residuals

Stacking Forecaster

  • Multiple Base Forecasters
  • Single Regressor is fitted on Base Predictions

Learned Weights Forecaster

  • Multiple Base Forecasters
  • Classification model learns optimal model weights

Rules Forecaster

  • Multiple Base Forecasters
  • Pre-defined rules on how to combine base methods.

This is an initial implementation

  • The methods have been tested on the Liander 2024 Huggingface dataset.
  • The results are accurate and improve over existing LGBM, XGBoost and GBLinear models
  • The code can still be further optimized for efficiency

Lars800 and others added 30 commits November 7, 2025 15:48
@Lars800
Added Lightgbm, LightGBM Linear Trees and Hybrid Stacking Forecasters
f69b8d2
@Lars800
Fixed small issues
6fcd632
@Lars800
Ruff compliance
7523987
@Lars800
fixed quality checks
c680aa1
@Lars800
Fixed last issues, Signed-off-by: Lars van Someren <lars.vansomeren@s…
9c1e3d3 
…ia-partners.com>
@Lars800
fixed comments
4394895
@Lars800
Refactor LightGBM to LGBM
5745212
@Lars800
Update LGBM and LGBMLinear defaults, fixed comments
a2538b6
@Lars800
Merge branch 'release/v4.0.0' into Openstef4.0.0_Additional_forecasters
1788759
@Lars800
Fixed comments
3d54604
@Lars800
Added SkopsModelSerializer
34fc3e5
@Lars800
Fixed issues
bad4c44
@Lars800
Gitignore optimization and dev sandbox
99c9bc5
@Lars800
Added MultiQuantileAdapter Class
4027de7
@Lars800
small fix
064a92d
@Lars800
Hybrid V2
ed83b3a
@Lars800
Small fix
bfa2e2f
@Lars800
Squashed commit of the following:
ce2172a 
commit 37089b8
Author: Egor Dmitriev <egor.dmitriev@alliander.com>
Date:   Mon Nov 17 15:29:59 2025 +0100

    fix(#728): Fixed parallelism stability issues, and gblinear feature pipeline. (#752)

    * fix(STEF-2475): Added loky as default option for parallelism since fork causes instabilities for xgboost results.

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    * fix(STEF-2475): Added better support for flatliners and predicting when data is sparse.

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    * fix(STEF-2475): Feature handing improvements for gblinear. Like imputation, nan dropping, and checking if features are available.

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    * fix(#728): Added checks on metrics to gracefully handle empty data. Added flatline filtering during evalution.

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    * fix(#728): Updated xgboost to skip scaling on empty prediction.

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    * fix(STEF-2475): Added parallelism parameters.

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    ---------

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

commit a85a3f7
Author: Egor Dmitriev <egor.dmitriev@alliander.com>
Date:   Fri Nov 14 14:31:34 2025 +0100

    fix(STEF-2475): Fixed rolling aggregate adder by adding forward filling and stating support for only one horizon. (#750)

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

commit 4f0c664
Author: Egor Dmitriev <egor.dmitriev@alliander.com>
Date:   Thu Nov 13 16:54:15 2025 +0100

    feature: Disabled data cutoff by default to be consistent with openstef 3.  And other minor improvements. (#748)

commit 493126e
Author: Egor Dmitriev <egor.dmitriev@alliander.com>
Date:   Thu Nov 13 16:12:35 2025 +0100

    fix(STEF-2475) fix and refactor backtesting iction in context of backtestforecasting config for clarity. Added more colors. Fixed data split function to handle 0.0 splits. (#747)

    * fix: Fixed data collation during backtesting. Renamed horizon to prediction in context of backtestforecasting config for clarity. Added more colors. Fixed data split function to handle 0.0 splits.

    * fix: Formatting.

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    * fix: Formatting.

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    ---------

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

commit 6b1da44
Author: Egor Dmitriev <egor.dmitriev@alliander.com>
Date:   Thu Nov 13 16:05:32 2025 +0100

    feature: forecaster hyperparams and eval metrics (#746)

    * feature(#729) Removed to_state and from_state methods in favor of builtin python state saving functions.

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    * feature(#729): Fixed issue where generic transform pipeline could not be serialized.

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    * feature(#729): Added more state saving tests

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    * feature(#729): Added more state saving tests

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    * feature(#729): Added more state saving tests

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    * feature: standardized objective function. Added custom evaluation functions for forecasters.

    * fix: Formatting.

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    ---------

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
@Lars800
set silence
8be453a
@Lars800
Merge branch 'release/v4.0.0' into research/v4.1.0_Additional_Forecas…
8c5743b 
…ters
@Lars800
small fix
ea90239
@Lars800
Fix final learner
93baa03
@Lars800
fixed lgbm efficiency
4f8ea8f
@Lars800
updated lgbm linear params
b4bdbdc
@Lars800
Fixed type and quality issues
ea1f5f7
@Lars800
First Version Sample Weighting Approach
22688e0 
Signed-off-by: Lars van Someren <lvsom1@gmail.com>
@Lars800
MetaForecasterClass
9b971d3 
Signed-off-by: Lars van Someren <lvsom1@gmail.com>
@Lars800 @egordm
Research/v4.1.0 additional forecasters (#765)
f476523 
* Added Lightgbm, LightGBM Linear Trees and Hybrid Stacking Forecasters

* Fixed small issues

* Ruff compliance

* fixed quality checks

* Fixed last issues, Signed-off-by: Lars van Someren <lars.vansomeren@sia-partners.com>

* fixed comments

* Refactor LightGBM to LGBM

* Update LGBM and LGBMLinear defaults, fixed comments

* Fixed comments

* Added SkopsModelSerializer

* Fixed issues

* Gitignore optimization and dev sandbox

* Added MultiQuantileAdapter Class

* small fix

* Hybrid V2

* Small fix

* Squashed commit of the following:

commit 37089b8
Author: Egor Dmitriev <egor.dmitriev@alliander.com>
Date:   Mon Nov 17 15:29:59 2025 +0100

    fix(#728): Fixed parallelism stability issues, and gblinear feature pipeline. (#752)

    * fix(STEF-2475): Added loky as default option for parallelism since fork causes instabilities for xgboost results.

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    * fix(STEF-2475): Added better support for flatliners and predicting when data is sparse.

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    * fix(STEF-2475): Feature handing improvements for gblinear. Like imputation, nan dropping, and checking if features are available.

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    * fix(#728): Added checks on metrics to gracefully handle empty data. Added flatline filtering during evalution.

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    * fix(#728): Updated xgboost to skip scaling on empty prediction.

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    * fix(STEF-2475): Added parallelism parameters.

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    ---------

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

commit a85a3f7
Author: Egor Dmitriev <egor.dmitriev@alliander.com>
Date:   Fri Nov 14 14:31:34 2025 +0100

    fix(STEF-2475): Fixed rolling aggregate adder by adding forward filling and stating support for only one horizon. (#750)

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

commit 4f0c664
Author: Egor Dmitriev <egor.dmitriev@alliander.com>
Date:   Thu Nov 13 16:54:15 2025 +0100

    feature: Disabled data cutoff by default to be consistent with openstef 3.  And other minor improvements. (#748)

commit 493126e
Author: Egor Dmitriev <egor.dmitriev@alliander.com>
Date:   Thu Nov 13 16:12:35 2025 +0100

    fix(STEF-2475) fix and refactor backtesting iction in context of backtestforecasting config for clarity. Added more colors. Fixed data split function to handle 0.0 splits. (#747)

    * fix: Fixed data collation during backtesting. Renamed horizon to prediction in context of backtestforecasting config for clarity. Added more colors. Fixed data split function to handle 0.0 splits.

    * fix: Formatting.

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    * fix: Formatting.

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    ---------

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

commit 6b1da44
Author: Egor Dmitriev <egor.dmitriev@alliander.com>
Date:   Thu Nov 13 16:05:32 2025 +0100

    feature: forecaster hyperparams and eval metrics (#746)

    * feature(#729) Removed to_state and from_state methods in favor of builtin python state saving functions.

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    * feature(#729): Fixed issue where generic transform pipeline could not be serialized.

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    * feature(#729): Added more state saving tests

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    * feature(#729): Added more state saving tests

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    * feature(#729): Added more state saving tests

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    * feature: standardized objective function. Added custom evaluation functions for forecasters.

    * fix: Formatting.

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

    ---------

    Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

* set silence

* small fix

* Fix final learner

* fixed lgbm efficiency

* updated lgbm linear params

* Fixed type and quality issues

* remove depricated files

Signed-off-by: Lars van Someren <lvsom1@gmail.com>

* change: Fixed dependencies to align more with the current release.

Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

* change: Style fixes.

Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>

---------

Signed-off-by: Lars van Someren <lvsom1@gmail.com>
Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
Co-authored-by: Egor Dmitriev <egor.dmitriev@alliander.com>
@Lars800
Merge remote-tracking branch 'origin/research/v4.1.0' into research/H…
5a54c4f 
…ybridForecaster2.0
@Lars800
fix merge issue
72b1ca7 
Signed-off-by: Lars van Someren <lvsom1@gmail.com>
@MvLieshout
Cleaning up
f3bd623 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout
Move skops code to separate branch
38ff7f4 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout
Move rules combiner code to separate branch
17f2528 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout
Cleaning up
ba8d99f 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout
Cleaning up
51e8490 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout
Improve hyperparam naming
f571ef8 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout
Remove openstef-meta dependency in openstef-models
2c102dc 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout
Separating openstef-models and openstef-meta WIP
1b5b027 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout
Move residual model to separate branch
5d8cf7d 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout
Merge branch 'release/v4.0.0' into research/HybridForecaster2.0
faf7722 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout
Rename regression tests to integration
2b16bd6 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout
Move forecaster back for now
e0f373b 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout
Use sample weight config in models
0529bf8 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout
Add importances to median forecaster
d2ee035 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout
Rename for consistency
bf18c16 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout
Move base mlflow callback to separate file
704fc2e 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout
Clean up docstring
f778c2a 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout
Add explainability to combiner models
eefe21a 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout
Make combiners explainable
5d6a288 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout
Delete empty folder
fe63349 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout
Use protocols
8c32d2f 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout
Rename test file
ae1d266 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout
Reset pinball_losses changes
698d61d 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout
Fix linting issues
a6fa901 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout MvLieshout requested a review from Copilot February 19, 2026 14:26
Copilot AI reviewed Feb 19, 2026
View reviewed changes
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 59 out of 67 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@MvLieshout
Add base models to tags MLflow
a5e3d31 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout
Improve what is stored in MLFlow
2a92cfa 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
@MvLieshout
Formatting
1d9ff5e 
Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@egordm egordm egordm requested changes

@MvLieshout MvLieshout MvLieshout left review comments

Copilot code review Copilot Copilot left review comments

Requested changes must be addressed to merge this pull request.

Assignees

@egordm egordm

@MvLieshout MvLieshout

Labels

feature New feature or request OpenSTEF 4.0 Work for OpenSTEF 4.0 OpenSTEF-Meta

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Lars800 @MvLieshout @egordm

Comments