%-th leading minor of the array is not positive definite

Questions
1

Hi all,

I am trying to configure PyMC3 Polynomial kernel with the following hyperpriors;

with pm.Model() as self.model:

      l = pm.Gamma("l", alpha=2, beta=1)
      offset = pm.Gamma("offset", alpha=2, beta=1)
      nu = pm.HalfCauchy("nu", beta=1)
      d = pm.HalfNormal("d", sd=5)

      cov = nu ** 2 * pm.gp.cov.Polynomial(X.shape[1], l, d, offset)

      self.gp = pm.gp.Marginal(cov_func=cov)

      sigma = pm.HalfCauchy("sigma", beta=1)
      y_ = self.gp.marginal_likelihood("y", X=X, y=Y, noise=sigma)

      self.map_trace = [pm.find_MAP()]

However, I’m getting Cholesky decomposition failed due to %-th leading minor of the array is not positive definite error as follows when sampling using this code snippet;

with model:
  f_pred = gp.conditional('f_pred', X_test)
  pred_samples = pm.sample_posterior_predictive(map_trace, vars=[f_pred], samples=2000)
  y_pred, uncer = pred_samples['f_pred'].mean(axis=0), pred_samples['f_pred'].std(axis=0)

Error:


LinAlgError                               Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/theano/compile/function_module.py in __call__(self, *args, **kwargs)
    902             outputs =\
--> 903                 self.fn() if output_subset is None else\
    904                 self.fn(output_subset=output_subset)

24 frames
LinAlgError: 7-th leading minor of the array is not positive definite

During handling of the above exception, another exception occurred:

LinAlgError                               Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/scipy/linalg/decomp_cholesky.py in _cholesky(a, lower, overwrite_a, clean, check_finite)
     38     if info > 0:
     39         raise LinAlgError("%d-th leading minor of the array is not positive "
---> 40                           "definite" % info)
     41     if info < 0:
     42         raise ValueError('LAPACK reported an illegal value in {}-th argument'

LinAlgError: 7-th leading minor of the array is not positive definite
Apply node that caused the error: Cholesky{lower=True, destructive=False, on_error='raise'}(Elemwise{Composite{((sqr(i0) * i1) + i2 + i3)}}[(0, 0)].0)
Toposort index: 11
Inputs types: [TensorType(float64, matrix)]
Inputs shapes: [(40, 40)]
Inputs strides: [(320, 8)]
Inputs values: ['not shown']
Outputs clients: [[Solve{A_structure='lower_triangular', lower=False, overwrite_A=False, overwrite_b=False}(Cholesky{lower=True, destructive=False, on_error='raise'}.0, TensorConstant{[  69.79  ..   472.83]}), Solve{A_structure='lower_triangular', lower=False, overwrite_A=False, overwrite_b=False}(Cholesky{lower=True, destructive=False, on_error='raise'}.0, Elemwise{Composite{(sqr(i0) * i1)}}[(0, 0)].0)]]

I checked the data, and there are no duplicates or NaN values. Infact, the same dataset works fine on Matern kernel.

Can someone help to properly configure this kernel?

Thanks.

2

This error happens when your covariance matrix doesn’t appear numerically to be positive definite. I would check carefully the range of the priors on your hyperparameters. Some values may lead to difficult to cholesky decompose the covariance matrix. I don’t know for sure, but values very close to zero on d from the HalfNormal prior may be causing this. I would start by hard-coding in reasonable values for the hyperparameters, and see if the problem persists. Then try swapping them out with informative priors until you find the source of the issue.

2 Likes
3

What @bwengals said, is probably the best way to proceed. An alternative way woud be to add a small WhiteNoise to your kernel. Note that this is not the best way to proceed. If you add too much noise, your model will end up just inferring noise and nothing meaningful.

2 Likes
4

Hello guys,

Thanks for the suggestions. I will implement these changes and update with the results.

Thanks again :slight_smile:

5

I was able to fix this for the moment by hardcoding hyperparameters as @bwengals suggested. However, I still get errors if I try to train with a different dataset. For the moment this works for me. Thanks.

6

Hi,
I am having the same issue in some cases. Is there a solution other than hardcoding the hyperparameters?