Skip to content

Commit 572797e

Browse files
antoinebou12antoinebou12
authored
Update solvers.md
1 parent 360cd15 commit 572797e

File tree

1 file changed

+69
-17
lines changed

1 file changed

+69
-17
lines changed

‎docs/details/solvers.md‎

Lines changed: 69 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,10 @@
11
## Linear Solver
22

3-
PolyFEM offers several linear solver options based on compilation options. For more information, see [PolySolve](../polysolve.md) a stand-alone linear solver wrapper library used by PolyFEM.
3+
PolyFEM offers several linear solver options based on compilation options. For more information, see [PolySolve](../polysolve.md), a stand-alone linear solver wrapper library used by PolyFEM.
44

5-
Options: `AMGCL`, `Eigen::BiCGSTAB`, `Eigen::CholmodSupernodalLLT`, `Eigen::ConjugateGradient`, `Eigen::DGMRES`, `Eigen::GMRES`, `Eigen::LeastSquaresConjugateGradient`, `Eigen::MINRES`, `Eigen::PardisoLDLT`, `Eigen::PardisoLU`, `Eigen::SimplicialLDLT`, `Eigen::SparseLU`, `Eigen::UmfPackLU`, `Hypre`, `Pardiso`
5+
**Options:** `AMGCL`, `Eigen::BiCGSTAB`, `Eigen::CholmodSupernodalLLT`, `Eigen::ConjugateGradient`, `Eigen::DGMRES`, `Eigen::GMRES`, `Eigen::LeastSquaresConjugateGradient`, `Eigen::MINRES`, `Eigen::PardisoLDLT`, `Eigen::PardisoLU`, `Eigen::SimplicialLDLT`, `Eigen::SparseLU`, `Eigen::UmfPackLU`, `Hypre`, `Pardiso`
66

7+
---
78

89
## Nonlinear Solver
910

@@ -14,8 +15,7 @@ The settings for the solver are stored inside the field `"nonlinear"`. General s
1415
* `"grad_norm"` (default: `1e-8`): convergence tolerance on the norm ($L^2$) of the gradient
1516
* `"nl_iterations"` (default: `1000`): maximum number of iterations to spend solving
1617
* `"use_grad_norm"` (default: `true`): whether to use the gradient norm or update direction norm for convergence checks
17-
* When optimizing a function it is natural to check for a zero (up to tolerance) gradient as this signifies an extremum. However, we also implement the convergence criteria used by [Li et al. [2020]](https://ipc-sim.github.io/). Where instead of the gradient's norm the update direction's $L^\infty$-norm is used. This provides two benefits: (1) it expresses the convergence criteria in the units of the variable (e.g., distance for elasticity) which (2) avoids applying small updates that lead to a marginal change in the variables. Note: this criterion has been well tested for nonlinear elasticity, but your mileage may vary for other formulations.
18-
<!-- * As a sanity check, there is always a check that $\|\nabla f\| < 10^{-2}$ to make sure the solver does not converge with a large gradient. -->
18+
* When optimizing a function, it is natural to check for a zero (up to tolerance) gradient as this signifies an extremum. However, we also implement the convergence criteria used by [Li et al., 2020](https://ipc-sim.github.io/). Instead of the gradient's norm, the update direction's $L^\infty$-norm is used. This provides two benefits: (1) it expresses the convergence criteria in the units of the variable (e.g., distance for elasticity) and (2) avoids applying small updates that lead to marginal changes in the variables. **Note:** This criterion has been well tested for nonlinear elasticity, but your mileage may vary for other formulations.
1919

2020
### Newton's Method
2121

@@ -25,30 +25,82 @@ A (projected) Newton's method with line search.
2525

2626
A quasi-Newton method, L-BFGS requires more iterations than the full Newton's method but avoids expensive linear solves.
2727

28-
Reference: https://en.wikipedia.org/wiki/Limited-memory_BFGS<br>
29-
Acknowledgments: The L-BFGS solver is implemented using the [LBFGS++](https://github.com/yixuan/LBFGSpp) library.
28+
**Reference:** [Limited-memory BFGS](https://en.wikipedia.org/wiki/Limited-memory_BFGS)
29+
**Acknowledgments:** The L-BFGS solver is implemented using the [LBFGS++](https://github.com/yixuan/LBFGSpp) library.
30+
31+
---
3032

3133
## Line Search
3234

33-
!!! todo
34-
Describe line-search and its purpose.
35+
Line search is a numerical optimization technique used to determine the appropriate step size along a given search direction in iterative optimization algorithms. The primary goal is to ensure sufficient decrease in the objective function, improving convergence speed and stability. By adaptively selecting a step size, line search helps to navigate the optimization landscape efficiently while avoiding overshooting or undershooting.
36+
37+
---
3538

3639
### Backtracking
3740

38-
!!! todo
39-
Describe this method and its parameters.
41+
Backtracking line search is an iterative method that starts with a large initial step size and reduces it iteratively until a sufficient decrease condition, often based on the robust Armijo criterion, is satisfied.
4042

41-
### Armijo
43+
**Parameters:**
4244

43-
!!! todo
44-
Describe this method and its parameters.
45+
- **Initial Step Size:** Starting value for the step size, typically set to `1`.
46+
- **Reduction Factor:** A constant in the range `(0, 1)` that determines how much the step size is reduced in each iteration. Typical values are `0.5` or `0.8`.
47+
- **Armijo Parameter:** A small positive constant, e.g., `1e-4`, which defines the sufficient decrease condition for the step size.
4548

46-
#### Armijo (alt)
49+
**Algorithm:**
50+
51+
1. Set the initial step size.
52+
2. Evaluate the objective function at the current step size.
53+
3. Check if the robust Armijo condition is satisfied:
54+
\[
55+
f(x + \alpha p) \leq f(x) + c \alpha \nabla f(x)^T p
56+
\]
57+
where \( \alpha \) is the step size, \( p \) is the search direction, and \( c \) is the Armijo parameter.
58+
4. If the condition is not met, reduce the step size by multiplying it with the reduction factor and repeat.
59+
5. Stop when the condition is satisfied or the step size becomes too small.
60+
61+
---
62+
63+
### Robust Armijo
64+
65+
The robust Armijo method improves the traditional Armijo criterion by incorporating additional tolerance checks to ensure meaningful convergence. It is particularly useful in preventing premature convergence when the objective function changes negligibly.
66+
67+
**Parameters:**
68+
69+
- **Armijo Parameter (c):** A small positive constant, typically `1e-4`, which sets the strictness of the sufficient decrease condition.
70+
- **Delta Relative Tolerance:** A small positive constant defining the threshold for acceptable relative changes in the objective function.
71+
72+
**Mathematical Formulation:**
73+
74+
The standard robust Armijo condition is:
75+
\[
76+
f(x + \alpha p) \leq f(x) + c \alpha \nabla f(x)^T p
77+
\]
78+
79+
If this condition is not met, the relative tolerance criterion is checked:
80+
\[
81+
\frac{|f(x + \alpha p) - f(x)|}{|f(x)|} \leq \text{tolerance}
82+
\]
83+
84+
If both conditions fail, the step size is reduced further.
85+
86+
**Purpose:**
87+
88+
The robust Armijo method ensures optimization steps are meaningful by combining sufficient decrease with relative tolerance checks. This avoids poor solutions caused by steps that barely alter the objective function.
89+
90+
**References:**
91+
92+
- Longva et al., 2023: Introduced numerical robustness improvements for the Armijo line search, making it suitable for more challenging optimization problems.
93+
- [Wikipedia: Line Search](https://en.wikipedia.org/wiki/Line_search)
94+
95+
---
4796

48-
!!! todo
49-
Describe this method and its parameters.
5097

5198
### More-Thuente
5299

53100
!!! todo
54-
Describe this method and its parameters.
101+
Describe this method and its parameters.
102+
103+
104+
**References:**
105+
106+
- Jorge Nocedal and Stephen J. Wright, *Numerical Optimization*, Springer, 2006.

0 commit comments

Comments
 (0)