@@ -4,22 +4,69 @@ This document outlines the benchmarking process for comparing the performance of
44
55> About tests performance can be found at: [ ![ CodSpeed Badge] ( https://img.shields.io/endpoint?url=https://codspeed.io/badge.json )] ( https://codspeed.io/cpp-linter/cpp-linter-hooks )
66
7- ## Running the Benchmark
7+ ## Run benchmark separately
88
99``` bash
10- python3 testing/benchmark_hooks.py
10+ rm -rf examples || true
11+ git clone --depth 1 --quiet https://github.com/gouravthakur39/beginners-C-program-examples.git examples
12+
13+ pre-commit clean
14+ hyperfine --warmup 1 -r 10 ' pre-commit run --config testing/test-cpp-linter-hooks.yaml --all-files'
15+
16+ rm -rf examples || true
17+ git clone --depth 1 --quiet https://github.com/gouravthakur39/beginners-C-program-examples.git examples
18+ pre-commit clean
19+
20+ hyperfine --warmup 1 -r 10 ' pre-commit run --config testing/test-mirrors-clang-format.yaml --all-files'
21+ ```
22+
23+ Results:
24+
25+ ``` bash
26+ # Updated on 2025-09-19
27+
28+ Cleaned /home/sxp/.cache/pre-commit.
29+ Benchmark 1: pre-commit run --config testing/test-cpp-linter-hooks.yaml --all-files
30+ Time (mean ± σ): 150.2 ms ± 1.8 ms [User: 121.7 ms, System: 29.2 ms]
31+ Range (min … max): 148.3 ms … 153.9 ms 10 runs
32+
33+ Cleaned /home/sxp/.cache/pre-commit.
34+ Benchmark 1: pre-commit run --config testing/test-mirrors-clang-format.yaml --all-files
35+ Time (mean ± σ): 122.6 ms ± 1.9 ms [User: 98.0 ms, System: 24.7 ms]
36+ Range (min … max): 120.3 ms … 125.5 ms 10 runs
1137```
1238
13- ## Results
39+ ### Run benchmark comparison
40+
41+ Compare the results of both commands.
1442
1543``` bash
16- # Updated on 2025-09-02
17- Benchmark Results:
44+ rm -rf examples || true
45+ git clone --depth 1 --quiet https://github.com/gouravthakur39/beginners-C-program-examples.git examples
46+
47+ hyperfine -i --warmup 1 -r 20 ' pre-commit run --config ../testing/test-cpp-linter-hooks.yaml --all-files' ' pre-commit run --config ../testing/test-mirrors-clang-format.yaml --all-files'
48+ ```
1849
19- Hook | Avg (s) | Std (s) | Min (s) | Max (s) | Runs
20- ---------------------+------------------+------------------+------------------+------------------+-----------------
21- cpp-linter-hooks | 12.473 | 1.738 | 11.334 | 15.514 | 5
22- mirrors-clang-format | 4.960 | 0.229 | 4.645 | 5.284 | 5
50+ Results:
51+
52+ ``` bash
53+ # Updated on 2025-09-19
54+ Benchmark 1: pre-commit run --config ../testing/test-cpp-linter-hooks.yaml --all-files
55+ Time (mean ± σ): 84.1 ms ± 3.2 ms [User: 73.5 ms, System: 10.2 ms]
56+ Range (min … max): 79.7 ms … 95.2 ms 20 runs
2357
24- Results saved to testing/benchmark_results.txt
58+ Warning: Ignoring non-zero exit code.
59+
60+ Benchmark 2: pre-commit run --config ../testing/test-mirrors-clang-format.yaml --all-files
61+ Time (mean ± σ): 85.0 ms ± 3.0 ms [User: 71.8 ms, System: 13.3 ms]
62+ Range (min … max): 81.0 ms … 91.0 ms 20 runs
63+
64+ Warning: Ignoring non-zero exit code.
65+
66+ Summary
67+ ' pre-commit run --config ../testing/test-cpp-linter-hooks.yaml --all-files' ran
68+ 1.01 ± 0.05 times faster than ' pre-commit run --config ../testing/test-mirrors-clang-format.yaml --all-files'
2569```
70+
71+ > [ !NOTE]
72+ > The results may vary based on the system and environment where the benchmarks are run.
0 commit comments