too small a value error in "min(exp(B*alpha))" while running ISMA fit

[data25]

I am trying the toy fitting problem (Figure 3) from the paper "Data fitting with geometric programming compatible function". I did manage to get the 'MA' and 'SMA' fit, however while trying the 'ISMA' fit i am constantly encountering the "Fitted constraint with too small values" error in the parameters value after running levenberg-marquardt algorithm. I did check the implementation a lot of times and i ended up getting the same error either in the parameter B( with too small value) or in the parameter A (with too large value). Is there a work around to this problem? Please note that I tried initializing the parameters using both 'MA' and 'SMA'. The alpha_init value was set to 1. And the lambda to be 0.2.

[1ozturkbe]

This is likely because you have too many posynomial terms. What is the dimension of your data, and the parameters of your fit? Also, you can try disabling the error in the source code and see what you get.

[1ozturkbe]

(you can find the directory of gpfit by calling import gpfit; gpfit.__path__.)

[data25]

i just have a 1D data with 50 points. x.shape = (50,1), y.shape=(50,1) and K=3 affine planes. With that i am supposed to get the constraint "1 = sum(B_k*x**A_k)." I did try bypassing the constraint fit (cs = FitCS(fitdata)) and also tried disabling the errors, but still faced the same issue. Also all i need is just to fit the convex function to the existing data for which i need a converged values of A and B.

The above picture shows the implementation of all the convex function classes.

[data25]

Oh yes i did. Facing same issues

[1ozturkbe]

I'm trying to replicate now. I am currently using gpfit with no issues on similar examples, so this is odd.

[1ozturkbe]

Alright, I replicated with the following MWE (please provide this for future issues):

import numpy as np
from gpfit.fit import fit
np.random.seed(314)
x = np.arange(-2,2,0.01)
a = -6*x + 6
b = 1/2*x
c = 1/5*x**5 + 1/2*x
y = np.max([a,b,c], axis=0)
K=3
cstrt, rms = fit(x, y, K, "ISMA")

The issues can be caused by random seed. I put 314 and issues were fixed. Please confirm that this works for you.

[data25]

the value of a is -6*x-6. Even with random seed i am encountering the "RuntimeWarning: invalid value encountered in double_scalars
abs(RMStraj[itr] - RMStraj[itr-2]) <" in levenberg_marquardt.py. And the plot looks like above

[1ozturkbe]

Here's the deal. I ran the LM algorithm with high verbosity for a shorter maxtime, and here are the results.

## -- End pasted text --

                    First-Order                        Norm of

Iter        Residual        optimality            Lambda            step        Jwarp

    0         0.513567             75.1

    1              inf             75.1                0.02               230.6        3.17e+06

    2              inf             75.1                 0.2               51.39        3.17e+06

    3              inf             75.1                   2               19.99        3.17e+06

c:\users\berk\dropbox (mit)\mit graduate school\code\gpfit\gpfit\levenberg_marquardt.py:112: RuntimeWarning: invalid value encountered in double_scalars
  abs(RMStraj[itr] - RMStraj[itr-2]) <
    4         0.511031             31.3                  20               2.786        3.17e+06

    5         0.508551             29.6                  20               4.319        3.42e+06

    6              inf             29.6                   2                8.12        3.933e+06

    7         0.506592               28                  20              0.8293        3.933e+06

    8         0.504837             26.6                  20              0.8376        4.582e+06

    9         0.494917             19.2                   2               3.416        5.262e+06

   10              inf             19.2                 0.2               21.68        7.78e+06

   11         0.487916             15.4                   2                8.11        7.78e+06

   12         0.482042             13.4                   2               4.453        3.844e+07

   13              inf             13.4                 0.2               54.59        7.21e+07

   14         0.476498             14.3                   2               23.94        7.21e+07

   15         0.471157             14.7                   2               10.64        6.618e+08

   16              inf             14.7                 0.2               227.3        1.266e+09

   17         0.465943             14.7                   2               102.9        1.266e+09

   18         0.460868             14.6                   2               22.93        4.715e+10

   19              inf             14.6                 0.2                1913        6.755e+10

   20         0.455908             14.4                   2               705.4        6.755e+10

   21         0.451069             14.2                   2               52.03        3.704e+13

   22              inf             14.2                 0.2           3.991e+04        1.988e+13

   23         0.446337             13.9                   2                8187        1.988e+13

   24         0.441714             13.7                   2               166.2        3.744e+17

   25              inf             13.7                 0.2           3.557e+06        1.272e+17

   26         0.437194             13.5                   2           5.656e+05        1.272e+17

   27         0.432772             13.2                   2                 300        6.785e+24

   28              inf             13.2                 0.2            1.38e+10        3.008e+24

   29         0.428445               13                   2           4.394e+09        3.008e+24

   30         0.424206             12.8                   2               131.8        2.879e+40

   31         0.388818             10.7                 0.2              0.8375        1.574e+40

   32         0.463318             10.7                0.02               6.135        1.027e+40

   33         0.359146             9.24                 0.2              0.3963        1.027e+40

   34           0.3343             8.08                 0.2              0.4035        2.152e+40

   35         0.269662             12.8                0.02               4.806        1.794e+40

   36              inf             12.8               0.002                3.69        1.647e+39

Reached maxtime (0.15 seconds)
Final RMS: 0.269662263366964
ISMA fit from params
1 = (1/w**2.27559e-10) * (u_1)**-1.21462e-09
    + (0.990747/w**0.770619) * (u_1)**0.364169
    + (9.9639e-06/w**1.6515) * (u_1)**10.6377

It is clearly converging towards an optimum...

But for some reason it ends up getting derailed.

[1ozturkbe]

Fyi, the occasional infs in the residual give the warnings. I didn't write the algorithm so I don't know exactly how the residual becomes inf and how the method navigates these solutions, but I know it involves a trust region reduction for the next iteration. But it is clear that as the solution is further refined, the singularity of the coefficients get worse. At maxtime of around 0.25-0.3s you actually recover the results from the paper. This is the best I can do to help.

[data25]

Actually i did do the same thing just now. Printed out the residuals as a sanity check. I got similar results like you posted. It is strange that ISMA works perfectly well for a log-space transformed input and gives out a perfect GP compatible fit function. However, it is just for this problem, the residuals go wild. Thanks a lot for helping out though.

[1ozturkbe]

No problem! I will keep this issue open since this is undesired behavior, and one that should probably be addressed. I am 95% certain it is because of the kink in this specific function.