Cannot convert <class 'numpy.ndarray'> to TensorType'

Questions
1

Sorry if this is a simple question, but I don’t seem to be able to figure it out.
I provide a minimal working example of a code, which creates a multivariate normal distribution with a covariance that depends on random variables.

import pymc3 as pm
import numpy as np

with pm.Model() as model:
n1 = pm.Gamma(‘n1’, alpha=2, beta=1 / 1.3, transform=None)
n2 = pm.Gamma(‘n2’, alpha=2, beta=1 / 1.3, transform=None)
D2 = pm.Gamma(‘D2’, alpha=2, beta=1 / 1.3, transform=None)

mu = [0,0]
cov = np.array([[n1, n2], [n2, D2]])
pm.MvNormal('test', mu=mu, cov=cov, observed = [1, 10])

However, the last line fails with the following error.

WARNING (theano.tensor.blas): Using NumPy C-API based implementation for BLAS functions.
Traceback (most recent call last):
File “C:\Python37\lib\site-packages\theano\tensor\type.py”, line 269, in dtype_specs
}[self.dtype]
KeyError: ‘object’

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “C:\Python37\lib\site-packages\theano\tensor\basic.py”, line 246, in constant
ttype = TensorType(dtype=x_.dtype, broadcastable=bcastable)
File “C:\Python37\lib\site-packages\theano\tensor\type.py”, line 51, in init
self.dtype_specs() # error checking is done there
File “C:\Python37\lib\site-packages\theano\tensor\type.py”, line 272, in dtype_specs
% (self.class.name, self.dtype))
TypeError: Unsupported dtype for TensorType: object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “C:\Python37\lib\site-packages\theano\tensor\basic.py”, line 194, in as_tensor_variable
return constant(x, name=name, ndim=ndim)
File “C:\Python37\lib\site-packages\theano\tensor\basic.py”, line 266, in constant
raise TypeError(“Could not convert %s to TensorType” % x, type(x))
TypeError: (‘Could not convert [[n1 n2]\n [D1 D2]] to TensorType’, <class ‘numpy.ndarray’>)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “C:\Program Files (x86)\JetBrains\PyCharm Community Edition 2019.1.2\helpers\pydev\pydevd.py”, line 1758, in
main()
File “C:\Program Files (x86)\JetBrains\PyCharm Community Edition 2019.1.2\helpers\pydev\pydevd.py”, line 1752, in main
globals = debugger.run(setup[‘file’], None, None, is_module)
File “C:\Program Files (x86)\JetBrains\PyCharm Community Edition 2019.1.2\helpers\pydev\pydevd.py”, line 1147, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File “C:\Program Files (x86)\JetBrains\PyCharm Community Edition 2019.1.2\helpers\pydev_pydev_imps_pydev_execfile.py”, line 18, in execfile
exec(compile(contents+“\n”, file, ‘exec’), glob, loc)
File “D:/temp.py”, line 43, in
pm.MvNormal(‘test’, mu=mu, cov=cov, observed = [1, 10])
File “C:\Python37\lib\site-packages\pymc3\distributions\distribution.py”, line 46, in new
dist = cls.dist(*args, **kwargs)
File “C:\Python37\lib\site-packages\pymc3\distributions\distribution.py”, line 57, in dist
dist.init(*args, **kwargs)
File “C:\Python37\lib\site-packages\pymc3\distributions\multivariate.py”, line 222, in init
super().init(mu=mu, cov=cov, tau=tau, chol=chol, lower=lower, *args, **kwargs)
File “C:\Python37\lib\site-packages\pymc3\distributions\multivariate.py”, line 59, in init
cov = tt.as_tensor_variable(cov)
File “C:\Python37\lib\site-packages\theano\tensor\basic.py”, line 200, in as_tensor_variable
raise AsTensorError(“Cannot convert %s to TensorType” % str_x, type(x))
theano.tensor.var.AsTensorError: (‘Cannot convert [[n1 n2]\n [D1 D2]] to TensorType’, <class ‘numpy.ndarray’>)

I was following an example given here
https://docs.pymc.io/api/distributions/multivariate.html#pymc3.distributions.multivariate.MvNormal
but I must be doing something wrong. Can someone advise?

It seems like theano is unable to convert cov from a numpy array to a tensor, but in it seems to be the case in the example above as well.

2

Hi!
I think the problem comes from the fact that you’re mixing numpy arrays and theano tensors in your covariance.

In the example you’re refering to, notice that, when the covariance matrix is known, it is a genuine numpy array, with floats in it, not theano tensors: np.array([[1., 0.5], [0.5, 2]]).

If you’re trying to estimate your covariance matrix through MCMC, then you need to define your cov as a tensor and not as an array, as explained later in the example. Usually, for computational reasons, you’ll use the Cholesky decomposition: chol_packed = pm.LKJCholeskyCov('chol_packed', n=3, eta=2, sd_dist=sd_dist).

Hope this helps :vulcan_salute:

3

Hi!

Thanks for the suggestions!

Could you point to the exact spot in the example, where cov [is defined] as a tensor and not as an array, as explained later in the example? Let us assume I would like to use the covariance and not the Cholesky decomposition for the moment.

4

I guess you suggest I replace

cov = np.array([[n1, n2], [n2, D2]])

with

cov = tt.stacklists([[n1, n2], [n2, D2]])

This seems to solve the problem, thank you. I just get a warning now when I try to estimate MAP which I don’t understand:

C:\Python37\lib\site-packages\theano\tensor\basic.py:6611: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use arr[tuple(seq)] instead of arr[seq]. In the future this will be interpreted as an array index, arr[np.array(seq)], which will result either in an error or a different result.
result[diagonal_slice] = x
C:\Python37\lib\site-packages\theano\tensor\basic.py:6611: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use arr[tuple(seq)] instead of arr[seq]. In the future this will be interpreted as an array index, arr[np.array(seq)], which will result either in an error or a different result.
result[diagonal_slice] = x
C:\Python37\lib\site-packages\theano\tensor\basic.py:6611: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use arr[tuple(seq)] instead of arr[seq]. In the future this will be interpreted as an array index, arr[np.array(seq)], which will result either in an error or a different result.
result[diagonal_slice] = x
C:\Python37\lib\site-packages\theano\tensor\basic.py:6611: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use arr[tuple(seq)] instead of arr[seq]. In the future this will be interpreted as an array index, arr[np.array(seq)], which will result either in an error or a different result.
result[diagonal_slice] = x

Do you know what it referst to?

5

I did point to the spot of the example: chol_packed = pm.LKJCholeskyCov('chol_packed', n=3, eta=2, sd_dist=sd_dist).

What you’re doing now is working because you’re using theano operators to define your cov matrix. In other words, when your cov matrix is unknown and must be estimated through MCMC, you need to put a prior distribution over it. Usually it’s done in terms of a distribution of correlations matrices (LKJCorr) or a distribution of Cholesky factors (LKJCholeskyCov, which works best computationally).

I’m not sure how your parametrization will fit, as it’s quite unusual, but if it makes sense scientifically and in your use-case, you’re right to try!

The warning is theano related and you can safely ignore it for now. Also, note that using MAP is discouraged by the PyMC team – unless you know what you’re doing.

1 Like
6

Hi Alex,

Thanks for your reply.

You were right: the problem was due to that I had to supply theano tensors and not numpy arrays of theano variables. It seems that a numpy array of theano variables was not automatically converted into a theano tensor, which caused the error I was seeing. After the correction, it works fine, except that I have discovered that it is extremely slow, so I had to find a way to make my calculations outside pymc and theano anyway. They are much faster without theano differentiation.

To answer your other remarks.

Thanks for pointing me to the Cholesky distribution for the covariance matrix coefficient. I guess, however, it only makes sense if they are Cholesky distributed… In my case, I don’t think it is a valid assumption, but I have reasonable assumptions on each coefficient, which explains the expressions I used in the example.

As for the MAP, thank you, I’ve indeed seen the warning, but my plan is to find the marginal likelihood using the Laplace approximation, and I definitely need to know mode of the distribution of that.

Thanks again for your time!