The up-to-date documentation regarding usage and features of CausalBench can be found at https://docs.causalbench.org.
Registration at CausalBench website is required in order to utilize the CausalBench package.
pip install causalbench-asu
CausalBench is a flexible, fair, and easy-to-use evaluation platform designed to advance research in causal learning. It facilitates scientific collaboration by providing a suite of tools for novel algorithms, datasets, and metrics. Our mission is to promote scientific objectivity, reproducibility, fairness, and awareness of bias in causal learning research. CausalBench serves as a comprehensive benchmarking resource, impacting a broad range of scientific and engineering disciplines.
- Transparent and Fair Evaluation: Ensures unbiased and transparent benchmarking processes.
- Facilitation of Scientific Collaboration: Encourages the sharing and development of novel algorithms, datasets, and metrics.
- Scientific Objectivity: Promotes objective assessment and comparison of causal learning methods.
- Reproducibility: Supports the reproducibility of research results, a cornerstone of scientific integrity.
- Awareness of Bias: Highlights and addresses biases in causal learning research.
- Benchmarking Data: A repository of diverse datasets specifically curated for evaluating causal learning algorithms.
- Algorithm Evaluation: Tools and frameworks for testing and comparing the performance of causal learning algorithms.
- Model Benchmarking: Standards and protocols for assessing the efficacy of different causal models.
- Metric Evaluation: A collection of metrics tailored for comprehensive evaluation of causal learning techniques.
CausalBench meets the needs of various scientific and engineering disciplines by providing essential resources and standards for evaluating causal learning methods. This platform helps researchers to:
- Collaborate and share advancements in causal learning.
- Ensure their work meets high standards of scientific rigor and fairness.
- Access a centralized repository of resources for benchmarking and evaluation.
To start using CausalBench, follow these steps:
- Installation: Instructions for installing CausalBench can be found here.
- Documentation: Comprehensive documentation for CausalBench, including CausalBench terms and tutorials, is available here.
CausalBench is an open-source project and welcomes contributions from the community. We plan to announce the contribution guideline soon.
CausalBench is licensed under the Apache License.
For questions, feedback, or further information, please contact us at support@causalbench.org.
This work is supported by NSF grant 2311716, "CausalBench: A Cyberinfrastructure for Causal-Learning Benchmarking for Efficacy, Reproducibility, and Scientific Collaboration".
CausalBench is structured to support different machine learning tasks and dataset types. With user contribution, the supported context will be expanded, currently (as of 8/12/25), these models and tasks are provided.
| Dataset | File | Description |
|---|---|---|
| Abalone | data, static graph | |
| Adult | data, static graph | |
| Sachs | data, static graph | |
| NetSim | data, static graph | Brain FMRI scan - 28 simulations - Each has different DGPs, num of nodes (5, 50), num of observations (50 to 5000), 1400 datasets in total |
| Time series simulated | data, temporal graph | |
| Telecom | data, temporal graph |
| Model | Task |
|---|---|
| PC | Static |
| GES | Static |
| VAR-LiNGAM | Temporal |
| PCMCIplus | Temporal |
| Metric | Task |
|---|---|
| Accuracy | Static |
| F1 | Static |
| Precision | Static |
| Recall | Static |
| SHD | Static |
| Accuracy | Temporal |
| F1 | Temporal |
| Precision | Temporal |
| Recall | Temporal |
| SHD | Temporal |