HDP: Heatwave Diagnostics Package

The Heatwave Diagnostics Package (HDP) is an open-source Python project that equips researchers with computationally-efficient tools to quantify heatwave metrics across multiple parameters for daily, gridded data produced by climate model large ensembles.

The HDP offers functions that leverage Xarray, Dask, and Numba to take full advantage of the available hardware capabilities. These functions have been optimized to both run quickly in serial execution and scale effectively in parallel and distributed computing systems. In addition to computing heatwave datasets, the HDP also contains several plotting functions that generate summary figures for quickly evaluating changes in heatwave patterns spatially, temporally, and across the heatwave parameter space. The user can choose to use the resulting matplotlib figures as base templates for creating their own custom figures or opt to create a standardized deck of figures that broadly summarizes the different metric trends. All graphical plots can then be stored in a Jupyter notebook for easy viewing and consolidated storage.

Why create the HDP?

Existing tools used to quantify heatwave metrics (such as ehfheatwaves, heatwave3, nctoolkit) were not designed to sample large sections of the heatwave parameter space. Many of these tools struggle to handle the computational burden of analyzing terabyte-scale datasets and do not offer a complete workflow for generating heatwave diagnostics from daily, gridded climate model output. The HDP expands upon this work to empower the user to conduct parameter-sampling analysis and reduce the computational burden of calculating heatwave metrics from increasingly large model output.

Documentation

To learn more about the HDP and how to use it, check out the full ReadTheDocs documentation at https://hdp.readthedocs.io/en/latest/user.html#.

Quick-Start

The code block below showcases an example HDP workflow using generated sample data:

from hdp.graphics.notebook import create_notebook
import hdp.utils
import hdp.measure
import hdp.threshold
import hdp.metric
import numpy as np

output_dir = "."

sample_control_temp = hdp.utils.generate_test_control_dataarray(add_noise=True)
sample_warming_temp = hdp.utils.generate_test_warming_dataarray(add_noise=True)

baseline_measures = hdp.measure.format_standard_measures(
    temp_datasets=[sample_control_temp]
)
test_measures = hdp.measure.format_standard_measures(
    temp_datasets=[sample_warming_temp]
)

percentiles = np.arange(0.9, 1.0, 0.01)

thresholds_dataset = hdp.threshold.compute_thresholds(
    baseline_measures,
    percentiles
)

definitions = [[3,0,0], [3,1,1], [4,0,0], [4,1,1], [5,0,0], [5,1,1]]

metrics_dataset = hdp.metric.compute_group_metrics(test_measures, thresholds_dataset, definitions, include_threshold=True)
metrics_dataset.to_netcdf(f"{output_dir}/sample_hw_metrics.nc", mode='w')

figure_notebook = create_notebook(metrics_dataset)
figure_notebook.save_notebook(f"{output_dir}/sample_hw_summary_figures.ipynb")

sample_control_temp = sample_control_temp.to_dataset()
sample_control_temp.attrs["description"] = "Mock control temperature dataset generated by HDP for unit testing."
sample_control_temp.to_netcdf(f"{output_dir}/sample_control_temp.nc", mode='w')

sample_warming_temp = sample_warming_temp.to_dataset()
sample_warming_temp.attrs["description"] = "Mock temperature dataset with warming trend generated by HDP for unit testing."
sample_warming_temp.to_netcdf(f"{output_dir}/sample_warming_temp.nc", mode='w')

Unit Tests

The testing suite can be run by cloning the repository, building the docker image, and then running the container:

git clone git@github.com:AgentOxygen/HDP.git
cd HDP
docker build --rm -t hdp .
docker run -v .:/project -it hdp

Tests are also run automatically in the GitHub Actions to verify that commits pushed to the repository do not break existing functionality.

All tests are written using pytest and are described in the hdp/tests/ directory.

Contributing

Please report any bugs, ask questions, and make suggestions through the GitHub Issues form of this repository.

Check out the Developer's Guide for information on how to setup a development environment and make changes to the HDP.

Acknowledgements

I would like to acknowledge the following people for their contributions to this project:

1. Dr. Geeta Persad for her guidance, mentorship, and encouragement throughout the development of this package.
2. Dr. Jane Baldwin for sharing her initial heatwave analysis code that inspired the HDP and providing her expertise on the science of extreme heat.
3. Dr. Tammas Loughran for developing ehfheatwaves which served as a comparable project and informed the software design of the HDP.
4. Dr. Ifeanyi Nduka for his design input and expertise in quantifying extreme heat.
5. (Soon to be Dr.) Sebastian Utama for helping me debug my code and brainstorm ideas.

Citation

If our software package helps you with your research, please consider citing it:

• Cummins, C., & Persad, G. (2025). HDP: Heatwave Diagnostics Package [Software]. Available from https://github.com/AgentOxygen/HDP.

In BibTeX:

@Manual{         cummins2025hdp,
 title         = {{HDP}: Heatwave Diagnostics Package (Software)},
 author        = {Cameron Cummins and Geeta Persad},
 year          = {2025},
 url           = {https://github.com/AgentOxygen/HDP}
}

This software citation is provided while the JOSS paper remains under review. The citations above follow guidelines described in this article by the Software Sustainability Institute.