Add L-BFGS optimizer from pyensmallen

[gauravmanmode]

This PR addresses issue #551.
pyensmallen provides python wrappers over the ensmallen C++ library.
This PR adds wrappers for L-BFGS optimizer in optimagic through pyensmallen.

[codecov]

Codecov Report

Attention: Patch coverage is 81.53846% with 12 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/optimagic/optimizers/pyensmallen_optimizers.py	78.43%	11 Missing ⚠️
src/optimagic/config.py	80.00%	1 Missing ⚠️

Files with missing lines	Coverage Δ
src/optimagic/algorithms.py	85.98% <100.00%> (+0.04%)	⬆️
src/optimagic/config.py	70.12% <80.00%> (+0.68%)	⬆️
src/optimagic/optimizers/pyensmallen_optimizers.py	78.43% <78.43%> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[janosg]

Hi @gauravmanmode, thanks for your PR. I made a few comments but this is definitely going into the right direction.

[janosg]

Hi @gauravmanmode, @timmens and I quickly looked into the failures on GitHub actions and found the following:

• Since pyensmallen will be an optional dependency, you have to put all imports into if statements. See fides as an example.
• Most of the failures are expected due to known limitations of the pyensmallen wheels (see e.g. here. Here the only thing we can do for now is skip the tests on non-linux systems and for Python >= 3.13; For all tests you are writing you can use pytests skipif decorator to enable this.
• On some systems the failure is already at the environment creation. @timmens will post a workaround for this.

[timmens]

Hey!

After a quick discussion with Janos we came to the conclusion that it is easiest if you solve the environment problem by

1. Removing the pyensmallen library from the environment.yml file (since it cannot be installed on Windows this would be problematic in any case). After running the pre-commit hooks this should also remove the library from the testing environments.
2. In the GitHub Actions specification file (main.yml) you have to split the "run pytest" step into two steps: one should run only on Python 3.13 (adjust the name accordingly) and the other one should run on Python < 3.13. For the latter one, install pyensmallen via pip after activating the optimagic environment and before running pytest. To skip a step conditional on the Python version, check out the if: ... line in the "Upload coverage report" step.

If you have any questions or need help, feel free to ask; @janosg will be happy to help!

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[gauravmanmode]

I have done the above said changes.

This is the alignment of tuning parameters for:

ensmallen_lbfgs

Old Name	Proposed Name	from optimizer in optimagic
numBasis	limited_memory_storage_length	scipy
maxIterations	stopping_maxiter	scipy
armijoConstant	armijo_constant	needs review
wolfe	wolfe_condition	needs review
minGradientNorm	convergence_gtol_abs	scipy
factr	convergence_ftol_rel	scipy
maxLineSearchTrials	max_line_search_trials	needs review
minStep	min_step_for_line_search	needs review
maxStep	max_step_for_line_search	scipy

I have attached a jupyter notebook which i used for testing and exploring the library.
link

[janosg]

I have done the above said changes.

This is the alignment of tuning parameters for:

ensmallen_lbfgs

Old Name Proposed Name from optimizer in optimagic
numBasis limited_memory_storage_length scipy
maxIterations stopping_maxiter scipy
armijoConstant armijo_constant needs review
wolfe wolfe_condition needs review
minGradientNorm convergence_gtol_abs scipy
factr convergence_ftol_rel scipy
maxLineSearchTrials max_line_search_trials needs review
minStep min_step_for_line_search needs review
maxStep max_step_for_line_search scipy
I have attached a jupyter notebook which i used for testing and exploring the library. link

Hi @gauravmanmode, thanks for the table. The names are really good. The only thing I would change is max_line_search_trials -> max_line_search_steps because wa already use that name in scipy_lbfgsb. (Unless I'm misunderstanding something and the options are conceptually different).

[gauravmanmode]

hi,
both max_line_search_steps and max_line_search_trials are the same.
also, i should be able to get a more detailed OptimizationReport from pyensmallen in a few days.

[janosg]

Is there any update on the more detailed OptimizationReport?

[gauravmanmode]

I have created a PR here on pyensmallen repo 2 weeks ago but the maintainer seems inactive so this has been pending.