Skip to content

Commit 943c4c3

Browse files
batched metric was not aggregated properly (#1067)
Co-authored-by: Francesco Bertolotti <francesco.bertolotti@igenius.ai>
1 parent 66ce47e commit 943c4c3

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

src/lighteval/metrics/__init__.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -45,8 +45,8 @@ def apply_metric(responses: list[ModelResponse], docs: list[Doc], metrics: list[
4545
output = {}
4646

4747
# Add batched metric results for this sample
48-
for metric_outputs in batched_outputs:
49-
output.update(metric_outputs[i])
48+
for metric, metric_outputs in zip(batched_metrics, batched_outputs):
49+
output.update({metric.metric_name: metric_outputs[metric.metric_name][i]})
5050

5151
# Add non-batched metric results for this sample
5252
for metric in non_batched_metrics:

0 commit comments

Comments
 (0)