chore: bump dev-deps, toolchain, disable benchmarks

[Kamirus]

• Bump dev-dependencies (test 2.1.2, bench 2.0.1) and toolchain (wasmtime 44.0.0).
• Gitignore mops.lock.
• Disable broken benchmark workflow (kept as manual workflow_dispatch).

[github-actions]

Benchmark Results

WARNING: dfx is deprecated, use icp-cli https://cli.internetcomputer.org. LLM skills can be found at https://skills.internetcomputer.org/llms.txt
WARNING: dfx is deprecated, use icp-cli https://cli.internetcomputer.org. LLM skills can be found at https://skills.internetcomputer.org/llms.txt
WARNING: dfx is deprecated, use icp-cli https://cli.internetcomputer.org. LLM skills can be found at https://skills.internetcomputer.org/llms.txt
WARNING: dfx is deprecated, use icp-cli https://cli.internetcomputer.org. LLM skills can be found at https://skills.internetcomputer.org/llms.txt
WARNING: dfx is deprecated, use icp-cli https://cli.internetcomputer.org. LLM skills can be found at https://skills.internetcomputer.org/llms.txt
WARNING: dfx is deprecated, use icp-cli https://cli.internetcomputer.org. LLM skills can be found at https://skills.internetcomputer.org/llms.txt
WARNING: dfx is deprecated, use icp-cli https://cli.internetcomputer.org. LLM skills can be found at https://skills.internetcomputer.org/llms.txt
WARNING: dfx is deprecated, use icp-cli https://cli.internetcomputer.org. LLM skills can be found at https://skills.internetcomputer.org/llms.txt
WARNING: dfx is deprecated, use icp-cli https://cli.internetcomputer.org. LLM skills can be found at https://skills.internetcomputer.org/llms.txt
WARNING: dfx is deprecated, use icp-cli https://cli.internetcomputer.org. LLM skills can be found at https://skills.internetcomputer.org/llms.txt

bench/ArrayBuilding.bench.mo $({\color{green}-6.10\%})$

Large known-size array building

Compares performance of different data structures for building arrays of known size.

Instructions: ${\color{green}-6.04\%}$
Heap: ${\color{gray}0\%}$
Stable Memory: ${\color{gray}0\%}$
Garbage Collection: ${\color{green}-0.05\%}$

Instructions

	1000	100000	1000000
List	542_124 $({\color{green}-1.11\%})$	47_994_218 $({\color{green}-0.68\%})$	475_069_964 $({\color{green}-0.65\%})$
Buffer	341_984 $({\color{green}-0.01\%})$	33_903_378 $({\color{green}-0.00\%})$	339_003_575 $({\color{green}-0.00\%})$
pure/List	209_110 $({\color{green}-30.79\%})$	20_703_488 $({\color{green}-31.00\%})$	207_055_893 $({\color{green}-30.99\%})$
VarArray ?T	172_496 $({\color{green}-4.45\%})$	17_002_890 $({\color{green}-4.49\%})$	170_003_087 $({\color{green}-4.49\%})$
VarArray T	160_791 $({\color{green}-0.01\%})$	15_803_185 $({\color{green}-0.00\%})$	158_003_382 $({\color{green}-0.00\%})$
Array (baseline)	42_669 $({\color{green}-0.06\%})$	4_003_063 $({\color{green}-0.00\%})$	40_003_260 $({\color{green}-0.00\%})$

Heap

	1000	100000	1000000
List	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$
Buffer	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$
pure/List	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$
VarArray ?T	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$
VarArray T	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$
Array (baseline)	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$

Garbage Collection

	1000	100000	1000000
List	9.96 KiB $({\color{green}-0.97\%})$	797.46 KiB $({\color{green}-0.01\%})$	7.67 MiB $({\color{green}-0.00\%})$
Buffer	8.71 KiB $({\color{gray}0\%})$	782.15 KiB $({\color{gray}0\%})$	7.63 MiB $({\color{gray}0\%})$
pure/List	19.95 KiB $({\color{gray}0\%})$	1.91 MiB $({\color{gray}0\%})$	19.07 MiB $({\color{gray}0\%})$
VarArray ?T	8.24 KiB $({\color{gray}0\%})$	781.68 KiB $({\color{gray}0\%})$	7.63 MiB $({\color{gray}0\%})$
VarArray T	8.23 KiB $({\color{gray}0\%})$	781.67 KiB $({\color{gray}0\%})$	7.63 MiB $({\color{gray}0\%})$
Array (baseline)	4.3 KiB $({\color{gray}0\%})$	391.02 KiB $({\color{gray}0\%})$	3.82 MiB $({\color{gray}0\%})$

No previous results found "/home/runner/work/motoko-core/motoko-core/.bench/Base64.bench.json"

bench/Base64.bench.mo $({\color{gray}0\%})$

Base64

Compare zero bytes vs mixed bytes encoding to Base64

Instructions: ${\color{gray}0\%}$
Heap: ${\color{gray}0\%}$
Stable Memory: ${\color{gray}0\%}$
Garbage Collection: ${\color{gray}0\%}$

Instructions

	zero bytes	mixed bytes
1	927	927
10	3_179	3_179
100	22_364	22_364
1000	214_214	214_214
10000	2_132_755	2_132_755

Heap

	zero bytes	mixed bytes
1	272 B	272 B
10	272 B	272 B
100	272 B	272 B
1000	272 B	272 B
10000	272 B	272 B

Garbage Collection

	zero bytes	mixed bytes
1	344 B	344 B
10	496 B	496 B
100	1.77 KiB	1.77 KiB
1000	14.66 KiB	14.66 KiB
10000	143.57 KiB	143.57 KiB

bench/FromIters.bench.mo $({\color{green}-18.12\%})$

Benchmarking the fromIter functions

Columns describe the number of elements in the input iter.

Instructions: ${\color{green}-18.12\%}$
Heap: ${\color{gray}0\%}$
Stable Memory: ${\color{gray}0\%}$
Garbage Collection: ${\color{gray}0\%}$

Instructions

	100	10_000	100_000
Array.fromIter	44_209 $({\color{green}-9.34\%})$	4_252_070 $({\color{green}-9.76\%})$	42_503_138 $({\color{green}-9.77\%})$
List.fromIter	24_963 $({\color{green}-21.25\%})$	2_381_606 $({\color{green}-22.21\%})$	23_803_617 $({\color{green}-22.22\%})$
List.fromIter . Iter.reverse	39_158 $({\color{green}-22.15\%})$	3_712_583 $({\color{green}-23.18\%})$	37_105_496 $({\color{green}-23.19\%})$

Heap

	100	10_000	100_000
Array.fromIter	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$
List.fromIter	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$
List.fromIter . Iter.reverse	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$

Garbage Collection

	100	10_000	100_000
Array.fromIter	2.76 KiB $({\color{gray}0\%})$	234.79 KiB $({\color{gray}0\%})$	2.29 MiB $({\color{gray}0\%})$
List.fromIter	3.51 KiB $({\color{gray}0\%})$	312.88 KiB $({\color{gray}0\%})$	3.05 MiB $({\color{gray}0\%})$
List.fromIter . Iter.reverse	5.11 KiB $({\color{gray}0\%})$	469.17 KiB $({\color{gray}0\%})$	4.58 MiB $({\color{gray}0\%})$

bench/ListBufferNewArray.bench.mo $({\color{green}-9.33\%})$

List vs. Buffer for creating known-size arrays

Performance comparison between List and Buffer for creating a new array.

Instructions: ${\color{green}-5.14\%}$
Heap: ${\color{gray}0\%}$
Stable Memory: ${\color{gray}0\%}$
Garbage Collection: ${\color{green}-4.19\%}$

Instructions

	0 (baseline)	1	5	10	100 (for loop)
List	1_431 $({\color{green}-7.50\%})$	2_751 $({\color{green}-5.66\%})$	8_731 $({\color{green}-3.48\%})$	13_525 $({\color{green}-3.03\%})$	73_225 $({\color{green}-1.80\%})$
pure/List	1_311 $({\color{red}+5.13\%})$	1_328 $({\color{green}-1.99\%})$	2_039 $({\color{green}-16.40\%})$	2_935 $({\color{green}-22.78\%})$	22_627 $({\color{green}-29.00\%})$
Buffer	2_185 $({\color{red}+3.11\%})$	2_337 $({\color{red}+2.91\%})$	3_584 $({\color{red}+1.88\%})$	5_150 $({\color{red}+1.28\%})$	36_703 $({\color{red}+0.17\%})$

Heap

	0 (baseline)	1	5	10	100 (for loop)
List	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$
pure/List	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$
Buffer	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$

Garbage Collection

	0 (baseline)	1	5	10	100 (for loop)
List	476 B $({\color{green}-17.36\%})$	516 B $({\color{green}-16.23\%})$	676 B $({\color{green}-12.89\%})$	784 B $({\color{green}-11.31\%})$	1.84 KiB $({\color{green}-5.05\%})$
pure/List	360 B $({\color{gray}0\%})$	380 B $({\color{gray}0\%})$	460 B $({\color{gray}0\%})$	560 B $({\color{gray}0\%})$	2.3 KiB $({\color{gray}0\%})$
Buffer	856 B $({\color{gray}0\%})$	864 B $({\color{gray}0\%})$	896 B $({\color{gray}0\%})$	936 B $({\color{gray}0\%})$	1.62 KiB $({\color{gray}0\%})$

bench/PriorityQueues.bench.mo $({\color{green}-3.52\%})$

Different priority queue implementations

_Compare the performance of the following priority queue implementations:

• PriorityQueue: Binary heap implementation over List.
• PriorityQueueSet: Wrapper over Set<(T, Nat)>._

Instructions: ${\color{green}-3.54\%}$
Heap: ${\color{gray}0\%}$
Stable Memory: ${\color{gray}0\%}$
Garbage Collection: ${\color{red}+0.02\%}$

Instructions

	A) PriorityQueue	B) PriorityQueueSet
1.) 100000 operations (push:pop = 1:1)	568_912_967 $({\color{green}-4.79\%})$	512_819_534 $({\color{green}-1.90\%})$
2.) 100000 operations (push:pop = 2:1)	707_495_313 $({\color{green}-4.77\%})$	794_441_406 $({\color{green}-1.88\%})$
3.) 100000 operations (push:pop = 10:1)	336_409_528 $({\color{green}-6.01\%})$	861_119_552 $({\color{green}-1.38\%})$
4.) 100000 operations (only push)	176_982_741 $({\color{green}-8.02\%})$	876_145_673 $({\color{green}-1.20\%})$
5.) 50000 pushes, then 50000 pops	745_226_483 $({\color{green}-4.04\%})$	942_202_387 $({\color{green}-2.04\%})$
6.) 50000 pushes, then 25000 "pop;push"es	504_254_116 $({\color{green}-4.76\%})$	906_644_478 $({\color{green}-1.68\%})$

Heap

	A) PriorityQueue	B) PriorityQueueSet
1.) 100000 operations (push:pop = 1:1)	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$
2.) 100000 operations (push:pop = 2:1)	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$
3.) 100000 operations (push:pop = 10:1)	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$
4.) 100000 operations (only push)	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$
5.) 50000 pushes, then 50000 pops	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$
6.) 50000 pushes, then 25000 "pop;push"es	272 B $({\color{gray}0\%})$	272 B $({\color{gray}0\%})$

Garbage Collection

	A) PriorityQueue	B) PriorityQueueSet
1.) 100000 operations (push:pop = 1:1)	15.07 MiB $({\color{red}+0.24\%})$	17.43 MiB $({\color{gray}0\%})$
2.) 100000 operations (push:pop = 2:1)	19.73 MiB $({\color{gray}0\%})$	19.32 MiB $({\color{gray}0\%})$
3.) 100000 operations (push:pop = 10:1)	8.67 MiB $({\color{gray}0\%})$	12.64 MiB $({\color{gray}0\%})$
4.) 100000 operations (only push)	3.87 MiB $({\color{gray}0\%})$	9.96 MiB $({\color{gray}0\%})$
5.) 50000 pushes, then 50000 pops	22.03 MiB $({\color{red}+0.02\%})$	26.2 MiB $({\color{gray}0\%})$
6.) 50000 pushes, then 25000 "pop;push"es	14.22 MiB $({\color{gray}0\%})$	18.44 MiB $({\color{gray}0\%})$

bench/PureListStackSafety.bench.mo $({\color{green}-34.14\%})$

List Stack safety

Check stack-safety of the following pure/List-related functions.

Instructions: ${\color{green}-34.14\%}$
Heap: ${\color{gray}0\%}$
Stable Memory: ${\color{gray}0\%}$
Garbage Collection: ${\color{gray}0\%}$

Instructions

pure/List.split	15_502_639 $({\color{green}-36.99\%})$
pure/List.all	5_701_079 $({\color{green}-27.84\%})$
pure/List.any	5_801_455 $({\color{green}-27.49\%})$
pure/List.map	14_003_792 $({\color{green}-39.39\%})$
pure/List.filter	12_204_214 $({\color{green}-42.17\%})$
pure/List.filterMap	17_804_766 $({\color{green}-35.03\%})$
pure/List.partition	12_205_020 $({\color{green}-42.71\%})$
pure/List.join	21_905_353 $({\color{green}-33.83\%})$
pure/List.flatten	13_605_695 $({\color{green}-45.15\%})$
pure/List.take	15_505_779 $({\color{green}-36.98\%})$
pure/List.drop	7_804_212 $({\color{green}-21.20\%})$
pure/List.foldRight	12_205_838 $({\color{green}-36.11\%})$
pure/List.merge	20_408_449 $({\color{green}-35.84\%})$
pure/List.chunks	35_910_420 $({\color{green}-30.29\%})$
pure/Queue	112_662_773 $({\color{green}-21.03\%})$

Heap

pure/List.split	272 B $({\color{gray}0\%})$
pure/List.all	272 B $({\color{gray}0\%})$
pure/List.any	272 B $({\color{gray}0\%})$
pure/List.map	272 B $({\color{gray}0\%})$
pure/List.filter	272 B $({\color{gray}0\%})$
pure/List.filterMap	272 B $({\color{gray}0\%})$
pure/List.partition	272 B $({\color{gray}0\%})$
pure/List.join	272 B $({\color{gray}0\%})$
pure/List.flatten	272 B $({\color{gray}0\%})$
pure/List.take	272 B $({\color{gray}0\%})$
pure/List.drop	272 B $({\color{gray}0\%})$
pure/List.foldRight	272 B $({\color{gray}0\%})$
pure/List.merge	272 B $({\color{gray}0\%})$
pure/List.chunks	272 B $({\color{gray}0\%})$
pure/Queue	272 B $({\color{gray}0\%})$

Garbage Collection

pure/List.split	3.05 MiB $({\color{gray}0\%})$
pure/List.all	328 B $({\color{gray}0\%})$
pure/List.any	328 B $({\color{gray}0\%})$
pure/List.map	3.05 MiB $({\color{gray}0\%})$
pure/List.filter	3.05 MiB $({\color{gray}0\%})$
pure/List.filterMap	3.05 MiB $({\color{gray}0\%})$
pure/List.partition	3.05 MiB $({\color{gray}0\%})$
pure/List.join	3.05 MiB $({\color{gray}0\%})$
pure/List.flatten	3.05 MiB $({\color{gray}0\%})$
pure/List.take	3.05 MiB $({\color{gray}0\%})$
pure/List.drop	328 B $({\color{gray}0\%})$
pure/List.foldRight	1.53 MiB $({\color{gray}0\%})$
pure/List.merge	4.58 MiB $({\color{gray}0\%})$
pure/List.chunks	7.63 MiB $({\color{gray}0\%})$
pure/Queue	18.31 MiB $({\color{gray}0\%})$

bench/Queues.bench.mo $({\color{green}-10.78\%})$

Different queue implementations

Compare the performance of the following queue implementations:

• pure/Queue: The default immutable double-ended queue implementation.
  • Pros: Good amortized performance, meaning that the average cost of operations is low O(1).
  • Cons: In worst case, an operation can take O(size) time rebuilding the queue as demonstrated in the Pop front 2 elements scenario.
• pure/RealTimeQueue
  • Pros: Every operation is guaranteed to take at most O(1) time and space.
  • Cons: Poor amortized performance: Instruction cost is on average 3x for pop and 8x for push compared to pure/Queue.
• mutable Queue
  • Pros: Also O(1) guarantees with a lower constant factor than pure/RealTimeQueue. Amortized performance is comparable to pure/Queue.
  • Cons: It is mutable and cannot be used in shared types (not shareable).

Instructions: ${\color{green}-10.78\%}$
Heap: ${\color{gray}0\%}$
Stable Memory: ${\color{gray}0\%}$
Garbage Collection: ${\color{gray}0\%}$

Instructions

	pure/Queue	pure/RealTimeQueue	mutable Queue
Initialize with 2 elements	2_846 $({\color{green}-7.96\%})$	2_368 $({\color{red}+2.78\%})$	3_001 $({\color{green}-1.28\%})$
Push 500 elements	79_279 $({\color{green}-12.60\%})$	678_272 $({\color{green}-8.86\%})$	187_811 $({\color{green}-14.35\%})$
Pop front 2 elements	52_879 $({\color{green}-39.20\%})$	4_385 $({\color{green}-1.37\%})$	3_830 $({\color{green}-0.44\%})$
Pop 150 front&back	71_206 $({\color{green}-22.68\%})$	269_482 $({\color{green}-11.62\%})$	109_955 $({\color{green}-11.74\%})$

Heap

	pure/Queue	pure/RealTimeQueue	mutable Queue
Initialize with 2 elements	324 B $({\color{gray}0\%})$	300 B $({\color{gray}0\%})$	352 B $({\color{gray}0\%})$
Push 500 elements	8.08 KiB $({\color{gray}0\%})$	8.17 KiB $({\color{gray}0\%})$	19.8 KiB $({\color{gray}0\%})$
Pop front 2 elements	240 B $({\color{gray}0\%})$	240 B $({\color{gray}0\%})$	192 B $({\color{gray}0\%})$
Pop 150 front&back	-4.42 KiB $({\color{gray}0\%})$	-492 B $({\color{gray}0\%})$	-11.45 KiB $({\color{gray}0\%})$

Garbage Collection

	pure/Queue	pure/RealTimeQueue	mutable Queue
Initialize with 2 elements	508 B $({\color{gray}0\%})$	444 B $({\color{gray}0\%})$	456 B $({\color{gray}0\%})$
Push 500 elements	10.1 KiB $({\color{gray}0\%})$	137.84 KiB $({\color{gray}0\%})$	344 B $({\color{gray}0\%})$
Pop front 2 elements	12.19 KiB $({\color{gray}0\%})$	528 B $({\color{gray}0\%})$	424 B $({\color{gray}0\%})$
Pop 150 front&back	15.61 KiB $({\color{gray}0\%})$	49.66 KiB $({\color{gray}0\%})$	12.1 KiB $({\color{gray}0\%})$

No previous results found "/home/runner/work/motoko-core/motoko-core/.bench/Sort.bench.json"

bench/Sort.bench.mo $({\color{gray}0\%})$

Sort

VarArray.sortInPlace profiling

Instructions: ${\color{gray}0\%}$
Heap: ${\color{gray}0\%}$
Stable Memory: ${\color{gray}0\%}$
Garbage Collection: ${\color{gray}0\%}$

Instructions

	100	1000	10000	12000	100000	1000000
old-sort	205_451	2_681_815	35_810_320	43_067_826	442_387_549	5_046_582_599
new-sort	72_460	1_123_218	16_084_225	19_445_600	201_432_925	2_423_397_103

Heap

	100	1000	10000	12000	100000	1000000
old-sort	272 B	272 B	272 B	272 B	272 B	308 B
new-sort	272 B	272 B	272 B	272 B	272 B	308 B

Garbage Collection

	100	1000	10000	12000	100000	1000000
old-sort	736 B	4.23 KiB	39.39 KiB	47.2 KiB	390.95 KiB	3.82 MiB
new-sort	536 B	2.28 KiB	19.86 KiB	23.77 KiB	195.64 KiB	1.91 MiB

Note: Renamed benchmarks cannot be compared. Refer to the current baseline for manual comparison.

[ggreif]

Okay!