NLCG Wolfe produces NaN's in `BlkDiag_Cmplx` for BLOCK2 Example

[MiCurry]

Running the BLOCK2 example with NLCG with the Wolfe line search option produces NaN's in the BlkDiag_Cmplx:

 ERROR: NaN in BlkDiag_Cmplx for i=           1  and j=       23898
 ERROR: NaN in BlkDiag_Cmplx for i=           1  and j=       23901
 ERROR: NaN in BlkDiag_Cmplx for i=           1  and j=       23904
...

This happens during the 15h iteration of the NLCG iteration and appears to happen when a Strong Wolfe condition is not satisfied:

!======Strong Wolfe Condition NOT satisfied!======
  !========Now sectioning with brute force========
 alpha_l=   0.0000000000000000      alpha_r=   2269.5866147572719     
 SQRT of negative value in CUBIC INTERP!
  using default value to bracket...
  CUBICLS: f=3.766214E+00 m2=7.519497E-01 rms=   1.736164 lambda=1.000000E+00 alpha=0.000000E+00

The Wolfe and Wolfe2 line searches of LBFGS do not produce NaN's in this problem, though they still find that a Strong Wolfe Condition is not satisfied around this part of their iteration.

Not sure if this is just a bug, or if the BLOCK2 case is not good for the NLCG with Wolfe. But posting here for future reference/work. Perhaps this always occurs when the Strong Wolfe condition is not met? I am slowly looking for other examples.

[dong-hao]

Thanks @MiCurry for reporting the problem! The BLOCK2 has a tricky dataset that was calculated using the QMR FWD solver with a loose residual criteria. So it is common to see the BiCG FWD solver struggling to fit the data. This has some influences on the inversions, too, if my memory serves (i.e. inversion tries to move towards a point in the model space where the observed data cannot be fitted).

Still, even if the line search fails, the inversion should exit (or continue) in a graceful manner instead of throwing NaNs. I will take a look at that.

[MiCurry]

From Hao's email:

Today I was able to reproduce the problem on my laptop (with the BLOCK2 case using 4 processors). It seems the problem is due to a line that (kind of arbitrarily) determines the line search length using the left and right boundary, when the both the quadratic and cubic fitting failed.

alpha = sqrt(alpha_lalpha_r)
write(,'(a10,es12.5)') 'alpha=',alpha
write(ioLog,'(a10,es12.5)') 'alpha=',alpha
to determine a rough test line search step, which is the (geometric) middle point of the left and right bound of the line search. However, the alpha_l should be zero if not updated by the previous Quadratic or Cubic line search (which is a rare case). So the alpha will become 0 (!) and cause the issue.

I guess this is not reported before as the lineSearchWolfe was never wired in either the NLCG (cubic is the default) and LBFGS (lineSearchWolfe2 is the default).

[MiCurry]

Thanks @MiCurry for reporting the problem! The BLOCK2 has a tricky dataset that was calculated using the QMR FWD solver with a loose residual criteria. So it is common to see the BiCG FWD solver struggling to fit the data. This has some influences on the inversions, too, if my memory serves (i.e. inversion tries to move towards a point in the model space where the observed data cannot be fitted).

Still, even if the line search fails, the inversion should exit (or continue) in a graceful manner instead of throwing NaNs. I will take a look at that.

Ah that makes sense! I really am not too familiar with these examples so good to have a better understanding of them. I guess it's good we have an example that hits the non-strong-Wolfe condition edge case!