Skip to content

Migrate nevergrad optimizers to new documentation style #632

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 18 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
839 changes: 284 additions & 555 deletions docs/source/algorithms.md

Large diffs are not rendered by default.

157 changes: 76 additions & 81 deletions docs/source/how_to/how_to_start_parameters.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,125 +14,120 @@ advantages and drawbacks of each of them.
Again, we use the simple `sphere` function you know from other tutorials as an example.

```{eval-rst}
.. tabbed:: Array

A frequent choice of ``params`` is a one-dimensional numpy array. This is
because one-dimensional numpy arrays are all that is supported by most optimizer
libraries.
.. tab-set::
.. tab-item:: Array

In our opinion, it is rarely a good choice to represent parameters as flat numpy arrays
and then access individual parameters or sclices by positions. The only exception
are simple optimization problems with very-fast-to-evaluate criterion functions where
any overhead must be avoided.
A frequent choice of ``params`` is a one-dimensional numpy array. This is
because one-dimensional numpy arrays are all that is supported by most optimizer
libraries.

If you still want to use one-dimensional numpy arrays, here is how:
In our opinion, it is rarely a good choice to represent parameters as flat numpy arrays
and then access individual parameters or sclices by positions. The only exception
are simple optimization problems with very-fast-to-evaluate criterion functions where
any overhead must be avoided.

.. code-block:: python
If you still want to use one-dimensional numpy arrays, here is how:

import optimagic as om
.. code-block:: python

import optimagic as om

def sphere(params):
return params @ params

def sphere(params):
return params @ params

om.minimize(
fun=sphere,
params=np.arange(3),
algorithm="scipy_lbfgsb",
)

```
om.minimize(
fun=sphere,
params=np.arange(3),
algorithm="scipy_lbfgsb",
)

```{eval-rst}
.. tabbed:: DataFrame
.. tab-item:: DataFrame

Originally, pandas DataFrames were the mandatory format for ``params`` in optimagic.
They are still highly recommended and have a few special features. For example,
they allow to bundle information on start parameters and bounds together into one
data structure.
Originally, pandas DataFrames were the mandatory format for ``params`` in optimagic.
They are still highly recommended and have a few special features. For example,
they allow to bundle information on start parameters and bounds together into one
data structure.

Let's look at an example where we do that:
Let's look at an example where we do that:

.. code-block:: python
.. code-block:: python

def sphere(params):
return (params["value"] ** 2).sum()
def sphere(params):
return (params["value"] ** 2).sum()


params = pd.DataFrame(
data={"value": [1, 2, 3], "lower_bound": [-np.inf, 1.5, 0]},
index=["a", "b", "c"],
)
params = pd.DataFrame(
data={"value": [1, 2, 3], "lower_bound": [-np.inf, 1.5, 0]},
index=["a", "b", "c"],
)

om.minimize(
fun=sphere,
params=params,
algorithm="scipy_lbfgsb",
)
om.minimize(
fun=sphere,
params=params,
algorithm="scipy_lbfgsb",
)

DataFrames have many advantages:
DataFrames have many advantages:

- It is easy to select single parameters or groups of parameters or work with
the entire parameter vector. Especially, if you use a well designed MultiIndex.
- It is very easy to produce publication quality LaTeX tables from them.
- If you have nested models, you can easily update the parameter vector of a larger
model with the values from a smaller one (e.g. to get good start parameters).
- You can bundle information on bounds and values in one place.
- It is easy to compare two params vectors for equality.
- It is easy to select single parameters or groups of parameters or work with
the entire parameter vector. Especially, if you use a well designed MultiIndex.
- It is very easy to produce publication quality LaTeX tables from them.
- If you have nested models, you can easily update the parameter vector of a larger
model with the values from a smaller one (e.g. to get good start parameters).
- You can bundle information on bounds and values in one place.
- It is easy to compare two params vectors for equality.


If you are sure you won't have bounds on your parameter, you can also use a
pandas.Series instead of a pandas.DataFrame.
If you are sure you won't have bounds on your parameter, you can also use a
pandas.Series instead of a pandas.DataFrame.

A drawback of DataFrames is that they are not JAX compatible. Another one is that
they are a bit slower than numpy arrays.
A drawback of DataFrames is that they are not JAX compatible. Another one is that
they are a bit slower than numpy arrays.


```
.. tab-item:: Dict

```{eval-rst}
.. tabbed:: Dict
``params`` can also be a (nested) dictionary containing all of the above and more.

``params`` can also be a (nested) dictionary containing all of the above and more.
.. code-block:: python

.. code-block:: python
def sphere(params):
return params["a"] ** 2 + params["b"] ** 2 + (params["c"] ** 2).sum()

def sphere(params):
return params["a"] ** 2 + params["b"] ** 2 + (params["c"] ** 2).sum()

res = om.minimize(
fun=sphere,
params={"a": 0, "b": 1, "c": pd.Series([2, 3, 4])},
algorithm="scipy_neldermead",
)

res = om.minimize(
fun=sphere,
params={"a": 0, "b": 1, "c": pd.Series([2, 3, 4])},
algorithm="scipy_neldermead",
)
Dictionarys of arrays are ideal if you want to do vectorized computations with
groups of parameters. They are also a good choice if you calculate derivatives
with JAX.

Dictionarys of arrays are ideal if you want to do vectorized computations with
groups of parameters. They are also a good choice if you calculate derivatives
with JAX.
While optimagic won't stop you, don't go too far! Having parameters in very deeply
nested dictionaries makes it hard to visualize results and/or even to compare two
estimation results.

While optimagic won't stop you, don't go too far! Having parameters in very deeply
nested dictionaries makes it hard to visualize results and/or even to compare two
estimation results.

```
.. tab-item:: Scalar

```{eval-rst}
.. tabbed:: Scalar
If you have a one-dimensional optimization problem, the natural way to represent
your params is a float:

If you have a one-dimensional optimization problem, the natural way to represent
your params is a float:
.. code-block:: python

.. code-block:: python
def sphere(params):
return params**2

def sphere(params):
return params**2

om.minimize(
fun=sphere,
params=3,
algorithm="scipy_lbfgsb",
)

om.minimize(
fun=sphere,
params=3,
algorithm="scipy_lbfgsb",
)
```
5 changes: 3 additions & 2 deletions docs/source/refs.bib
Original file line number Diff line number Diff line change
Expand Up @@ -964,8 +964,8 @@ @inproceedings{tbpsaimpl
year = {2016},
month = {09},
pages = {},
title = {Evolution under Strong Noise: A Self-Adaptive Evolution Strategy Can Reach the Lower Performance Bound - the pcCMSA-ES},
volume = {9921},
title = {Evolution under Strong Noise: A Self-Adaptive Evolution Strategy Can Reach the Lower Performance Bound - the pcCMSA-ES},
booktitle = {Parallel Problem Solving from Nature -- PPSN XIII},volume = {9921},
isbn = {9783319458229},
doi = {10.1007/978-3-319-45823-6_3}
}
Expand Down Expand Up @@ -1037,6 +1037,7 @@ @book{emnaimpl
pages = {},
title = {Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation},
isbn = {9781461356042},
publisher = {Springer},
journal = {Genetic algorithms and evolutionary computation ; 2},
doi = {10.1007/978-1-4615-1539-5}
}
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -380,6 +380,7 @@ module = [
"pdbp",
"iminuit",
"nevergrad",
"nevergrad.optimization.base.ConfiguredOptimizer",
"yaml",
]
ignore_missing_imports = true
7 changes: 5 additions & 2 deletions src/optimagic/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,11 @@ def _is_installed(module_name: str) -> bool:
IS_NUMBA_INSTALLED = _is_installed("numba")
IS_IMINUIT_INSTALLED = _is_installed("iminuit")
IS_NEVERGRAD_INSTALLED = _is_installed("nevergrad")
IS_BAYESOPT_INSTALLED = _is_installed("bayes_opt")

IS_BAYESOPTIM_INSTALLED = _is_installed("bayes-optim")
IS_BAYESOPT_INSTALLED_AND_VERSION_NEWER_THAN_2 = (
_is_installed("bayes_opt")
and importlib.metadata.version("bayesian_optimization") > "2.0.0"
)

# ======================================================================================
# Check if pandas version is newer or equal to version 2.1.0
Expand Down
6 changes: 3 additions & 3 deletions src/optimagic/optimizers/bayesian_optimizer.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
from scipy.optimize import NonlinearConstraint

from optimagic import mark
from optimagic.config import IS_BAYESOPT_INSTALLED
from optimagic.config import IS_BAYESOPT_INSTALLED_AND_VERSION_NEWER_THAN_2
from optimagic.exceptions import NotInstalledError
from optimagic.optimization.algo_options import N_RESTARTS
from optimagic.optimization.algorithm import Algorithm, InternalOptimizeResult
Expand All @@ -35,7 +35,7 @@
@mark.minimizer(
name="bayes_opt",
solver_type=AggregationLevel.SCALAR,
is_available=IS_BAYESOPT_INSTALLED,
is_available=IS_BAYESOPT_INSTALLED_AND_VERSION_NEWER_THAN_2,
is_global=True,
needs_jac=False,
needs_hess=False,
Expand Down Expand Up @@ -72,7 +72,7 @@ class BayesOpt(Algorithm):
def _solve_internal_problem(
self, problem: InternalOptimizationProblem, x0: NDArray[np.float64]
) -> InternalOptimizeResult:
if not IS_BAYESOPT_INSTALLED:
if not IS_BAYESOPT_INSTALLED_AND_VERSION_NEWER_THAN_2:
raise NotInstalledError(
"To use the 'bayes_opt' optimizer you need to install bayes_opt. "
"Use 'pip install bayesian-optimization'. "
Expand Down
Loading
Loading