The old implementation of COBYLA is buggy and challenging to maintain. Switch to the PRIMA implementation?

[zaikunzhang]

Dear csnumrics maintainers,

This is Dr. Zaikun Zhang from the Hong Kong Polytechnic University. Together with Professor N.I.M. Gould, I am responsible for maintaining the derivative-free optimization solvers of late Professor M.J.D. Powell.

Thank you for making COBYLA and BOBYQA available in csnumrics. I note that the current version is based on the original Fortran 77 implementation, the latter being not maintained anymore.

Although Powell's Fortran 77 implementation is truly a masterpiece, it contains many bugs, most of which are due to the language itself. For example, see Section 4.4 of our recent paper and the GitHub issues / requests listed at the end of this message (not all of them concern csnumrics).

To avoid the problems originating from the Fortran 77 code, I suggest you use the PRIMA implementation of Powell's solvers. PRIMA provides the reference implementation for Powell's renowned derivative-free optimization methods, namely COBYLA, UOBYQA, NEWUOA, BOBYQA, and LINCOA. The "P" in the name stands for Powell, and "RIMA" is an acronym for "Reference Implementation with Modernization and Amelioration".

PRIMA provides the modern implementation of the solvers in Fortran 2008. It fixes bugs in the original Fortran 77 code. In addition, it introduces improvements that boost the performance in terms of the number of function evaluations, which is the standard measure of computational costs in derivative-free optimization.

See the GitHub repo of PRIMA for more information. I will be glad to provide assistance if help is needed.

Thanks.

---

Issues and requests raised due to bugs in the Fortran 77 implementation (and translation) of Powell's solvers

• The Fortran 77 solvers may get stuck in infinite loops.

  • optimize: COBYLA hangs / infinite loop #8998

  • BUG: Scipy.optimize / COBYLA hangs on some CPUs #15527

  • COBYLA freezes (though maxeval and maxtime are given) #370

  • COBYLA hangs #118

  • NEWUOA_BOUND stuck in infinite loop inside MMA #117

  • Cobyla freezes in 0T1.16rc1 #1651

  • Optimization freezes #25

  • BOBYQA gets stuck in infinite loop. #7

  • Algorithm turns into infinite loop and never finishes #3

• The Fortran 77 solvers may crash with segmentation faults
  due to uninitialized variables that are used as indices.

  • Fix all uninitialized variable warnings #134

  • BOBYQA uninitialised variables in rare cases #133

  • Use of uninitialized variable in BOBYQA altmov #36

• Fortran 77 COBYLA may not return the best point that is evaluated; sometimes, the returned point can have a
  large constraint violation even though the starting point is feasible.

  • nlopt COBYLA optimizer gives unexpected output #182

  • Last Result Returned Not Optimized Result #110

  • COBYLA returns last evaluated function which might not be minimum #57

  • Successful termination when constraints violated #1

[anders9ustafsson]

Sorry for the late response, and many thanks for informing about the new PRIMA implementation. Are there any benchmarks that already today confirms that the PRIMA implementations are more reliable and faster than the original implementations?

It would be quite an undertaking to re-implement the C# code based on PRIMA, and I personally do not have the time or commitment necessary to do that right now. But if anyone else is interested in pursuing this, please feel free to share your efforts.

[zaikunzhang]

Are there any benchmarks that already today confirms that the PRIMA implementations are more reliable and faster than the original implementations?

Yes. See the benchmarking results in the README of PRIMA. Thanks.