Measure the energy efficiency of large language model inference.
llenergymeasure is a Python framework for measuring the energy consumption, throughput, and computational cost (FLOPs) of LLM inference across different deployment configurations. It helps researchers compare the energy efficiency of different models, precisions, and inference engines — reproducibly and at publication quality.
- Multi-backend inference — PyTorch (local), vLLM (Docker), TensorRT-LLM (planned)
- GPU energy measurement — NVML (default), Zeus, or CodeCarbon
- Study / sweep system — define parameter grids, run Cartesian product experiments automatically
- Docker isolation — per-experiment containers with full GPU passthrough for vLLM
- Reproducibility — fixed seeds, cycle ordering, thermal management, effective config recorded
- Built-in datasets — AI Energy Score benchmark prompts included
pip install "llenergymeasure[pytorch]"Run your first measurement:
llem run --model gpt2 --backend pytorchSee Installation for system requirements, Docker setup, and available extras. See Getting Started to run and interpret your first experiment.
| Guide | Description |
|---|---|
| Installation | System requirements, pip install, Docker setup path |
| Getting Started | First experiment, PyTorch and Docker tracks |
| Docker Setup | NVIDIA Container Toolkit walkthrough for vLLM |
| Backend Configuration | PyTorch vs vLLM, parameter support matrix |
| Study & Experiment Configuration | YAML reference, sweeps, config schema |
| CLI Reference | llem run and llem config flags and options |
| Energy Measurement | NVML, Zeus, CodeCarbon backends, measurement mechanics |
| Measurement Methodology | Warmup, baseline, thermal management, reproducibility |
| Troubleshooting | Common issues, invalid combinations, getting help |
| Guide | Description |
|---|---|
| What We Measure | Plain-language explanation of energy, throughput, and FLOPs |
| Interpreting Results | How to read llenergymeasure output |
| Getting Started (Policy Maker) | Minimal path to running a measurement |
| Comparison with Other Benchmarks | MLPerf, AI Energy Score, CodeCarbon, Zeus context |
Contributions welcome. See the development install instructions to set up a local environment.