Lift what you can: Green Online Learning with Heterogeneous Ensembles

Heterogeneous Online Ensemble (HEROS)

• HEROS is a framework to address the challenges of managing multiple models in an online ensemble and sustainable training
• Concept: For every training step, HEROS chooses a subset of models from a pool of models initialized with diverse hyperparameter choices under resource constraints to train.
• We define a Markov decision process (MDP) using Gymnasium library.
• Introduction of various policies (online_pool_gym/policy)

Example Code

from capymoa.drift.detectors import ADWIN
from capymoa.datasets import Electricity
from capymoa.evaluation import ClassificationEvaluator
from online_pool_gym.pool_gym import PoolGym
from online_pool_gym.policy import ZetaPolicy
from experiments.init.init_pool import init_ht_pool

data_stream = Electricity()
policy = ZetaPolicy(num_models_to_train=6, zeta=0.01, epsilon=0.1)
cm_evaluator = ClassificationEvaluator(schema=data_stream.get_schema())
model = PoolGym(
    pool=init_ht_pool(schema=data_stream.get_schema(),
                      sizes=[{'max_byte_size': '2KB', 'confidence': 0.001, 'grace_period': 200},
                             {'max_byte_size': '4KB', 'confidence': 0.001, 'grace_period': 200}],
                      seeds=[1, 2]
                      ),
    policy=policy,
    evaluate_all_models=True,
    detectors=[ADWIN(delta=1e-5) for _ in range(4)],
    num_instances_to_train_all_models=100)

while data_stream.has_more_instances():
    instance = data_stream.next_instance()
    y_pred = model.predict(instance)
    cm_evaluator.update(instance.y_index, y_pred)
    model.train(instance)
print(f"Accuracy: {cm_evaluator.accuracy()}")

Reproduce paper results

To reproduce the results from the paper "Lift what you can", run the following command in your terminal:

bash ./paper_results.sh