Benchmarks for hypervolume and dominance computation

[MLopez-Ibanez]

Hi,

I mentioned Xopt here: https://multi-objective.github.io/moocore/python/#benchmarks

I hope I have reflected fairly what the performance of Xopt is when computing hypervolume and dominance. Please let me know if my understanding is incorrect.

If there are other multi-objective functionality that would be interesting to benchmark (see the API: https://multi-objective.github.io/moocore/python/reference/functions.html), please let me know.

[nikitakuklev]

Thanks for letting us know. As you mention, we merely reuse methods from other libraries since HV calculation is not done except for visualization. For BO, we rely on DominatedPartitioning, a more complex but necessary procedure. I did a quick benchmark (#323) on our end and it (generally) confirms your conclusions.

[MLopez-Ibanez]

Thanks. Could you include moocore in your benchmark? The easiest is probably to use: https://multi-objective.github.io/moocore/python/reference/generated/moocore.hypervolume.html

Is there are reason to not use the timeit package for benchmarking? (I may take some ideas from your benchmarking setup if that is OK).

[nikitakuklev]

Sorry, to be clear that benchmark is just for us to decide which to use from currently available dependencies (it is hard to change deps since Xopt is used on isolated industrial networks). Technically even pymoo is not a dependency yet, which is why we ended up using botorch.

I used custom helper to collect times and results - results are cross-checked to ensure methods are correctly implemented.

[nikitakuklev]

Closing as resolved

[MLopez-Ibanez]

Just a heads-up that pymoo has started to use moocore, so you may start seeing differences in your benchmarks: anyoptimization/pymoo#732