Problem with Gram matrix not being positive semi-definite

[kimbente]

Dear all,

Thanks a lot for making the code available!

I have been running into some issues with signature_kernel.compute_Gram() as the output is not always positive semi-definite and thus I run into problems with Cholesky decomposition in the following step.

Error message:
_LinAlgError: linalg.cholesky: The factorization could not be completed because the input is not positive-definite (the leading minor of order 65 is not positive-definite).

The following is an isolated, reproducible example on simulated data, however I can only reproduce negative eigenvalues occasionally.

# Specify the static kernel (for linear kernel use sigkernel.LinearKernel())
static_kernel = sigkernel.RBFKernel(sigma = 0.5)

# Specify dyadic order for PDE solver (int > 0, default 0, the higher the more accurate but slower)
dyadic_order = 1

# Initialize the corresponding signature kernel
signature_kernel = sigkernel.SigKernel(static_kernel, dyadic_order)

batch, len_x, dim = 5, 10, 2
X = torch.rand((batch, len_x, dim), dtype = torch.float64, device = 'cuda')

G_x_x = signature_kernel.compute_Gram(X, X, sym = True, max_batch = 100)

print((G_x_x == G_x_x.mT).all())
> tensor(False, device='cuda:0')

# reproducible on (larger) real data:
print((torch.linalg.eigvals(G_x_x).to(torch.float64) >= 0).all())
> tensor(False, device='cuda:0')

Am I doing something wrong and can you help out?

Other questions: Can signature_kernel.compute_Gram() be used on irregularly spaced time series where I have the corresponding vector t for the time series as an auxiliary input?

Thanks!!

[crispitagorico]

Hi Kim, Thanks for your question. The PDE solver provides an approximation to the signature kernel. If the computational grid is too course, the PDE solver might not have converged to a reasonable accuracy. In that case, it is conceivable that you will get a Gram matrix that is not positive semi-definite. You should try to increase dyadic_order to higher (integer) values or rescaling down your paths to accelarate the convergence. I hope this helps.