Value error of pm.model

Hi,

I wanted to use the package to perform MCMC for a very simple neural network model, but there is some error which I am not able to figure out (I am not very familiar with the pymc3 and theano). The codes are the following. Appreciate if you can offer any idea!

import numpy as np
import scipy.stats as ss
import pymc3 as pm
import theano.tensor as tt

d_i = 2 #number of inputs
d_o = 2 #number of outputs
hidden_units = 2 # number of dimension 
sigma = 0.01 # sigma of prior distribution
sigma2_e = 0.000001

nnmodel = pm.Model()

data_x = np.loadtxt('input.txt',dtype='f', delimiter=' ')
data_y = np.loadtxt('output.txt',dtype='f', delimiter=' ')
ns = np.size(data_x,0) # the number of data samples

with nnmodel: 
    x1 = pm.Normal('x1', mu=0, tau=1, shape=(d_i+1)*hidden_units )
    x2 = pm.Normal('x2', mu=0, tau = 1, shape=(1+hidden_units)*d_o )
    mu0 = np.zeros((1,d_o*ns))
    cove = np.eye(d_o*ns)*sigma2_e
    b1 = x1[d_i*hidden_units : ]
    b1 = np.tile(b1, (ns,1))
    W1 = x1[:d_i*hidden_units] #assign weights associated to connection between input and hidden layer 
    W1 = W1.reshape((d_i, hidden_units))       
    W2 = x2[:d_o*hidden_units] #assign weights associated to connection between output and hidden layer 
    W2 = W2.reshape((hidden_units, d_o))
    b2 = x2[d_o*hidden_units : ]
    b2 = np.tile(b2, (ns,1))
    angle = tt._shared(data_x) # convert format of input np.array data    
    W1 = tt._shared(W1)
    b1 = tt._shared(b1)
    W2 = tt._shared(W2)
    b2 = tt._shared(b2)
    z1 = tt.dot(angle, W1) + b1
    a1 = np.tanh(z1)
    out = tt.dot(a1, W2) + b2
    dif = out - data_y
    dif = np.reshape(dif, (1, d_o*ns))
    mu0 = tt._shared(mu0)
    cove = tt._shared(cove)
    y = pm.MvNormal('y', mu = mu0, cov =cove, observed=dif )

    trace = pm.sample(5000,core=12)

My error is

W_1 = tt._shared(W1)
  File "/software/Anaconda/5.3.1/lib/python3.7/site-packages/theano/tensor/sharedvar.py", line 45, in tensor_constructor
    raise TypeError()

TypeError

You can just remove these lines as they are already a tensor. Also, within a pm.Model block all operations are tensors - you should replace all the np.{} function with theano functions.

Hi junpeng,

Thanks a lot. I corrected my code and it can run now. However, I got a new problem when trying to save the samples: I added a line “db = pm.backends.NDArray(‘nnbayes’)” beneath “with nnmodel:” and changed trace = pm.sample(5000, core=12, trace=db) as instructed by the online document. But the error message says: File “/utrc/home/longq/multimode/NN/bayes.py”, line 72, in
trace = pm.sample(50000,core=4, trace=db)

File “/utrc/software/Anaconda/5.3.1/lib/python3.7/site-packages/pymc3/sampling.py”, line 439, in sample
trace = _mp_sample(**sample_args)

File “/utrc/software/Anaconda/5.3.1/lib/python3.7/site-packages/pymc3/sampling.py”, line 994, in _mp_sample
trace.record(draw.point, draw.stats)

File “/utrc/software/Anaconda/5.3.1/lib/python3.7/site-packages/pymc3/backends/ndarray.py”, line 229, in record
for varname, value in zip(self.varnames, self.fn(point)):

File “/utrc/software/Anaconda/5.3.1/lib/python3.7/site-packages/pymc3/model.py”, line 1108, in call
return self.f(**state)

File “/utrc/software/Anaconda/5.3.1/lib/python3.7/site-packages/theano/compile/function_module.py”, line 797, in call
raise TypeError(“Too many parameter passed to theano function”)

TypeError: Too many parameter passed to theano function

Could you please provide some advise ?

Thanks,
Quan

The text backend is quite buggy and we will likely deprecate it soon. I suggest you to have a look at https://github.com/arviz-devs/arviz. It transformed the sample to xarray that is much easier and clearer to serialized/saved.

Can you give me a line or two about how to save a trace and load a trace correctly ? Thanks.

@canyon289, @colcarroll any suggestion of best practice here?

@longq
These three methods from ArviZ could come in useful

data = az.from_pymc3(trace=trace)
az.to_netcdf(data, "myfile.nc")
az.to_from("myfile.nc")

In words the first one converts the PyMC3 trace to az.InferenceData which is a standard container for Inference results. And the other two save or load the data from disk.

More information is available on the ArviZ docs
https://arviz-devs.github.io/arviz/api.html

Thanks for the message @canyon289
The first line works. But it seems az is not able to handle the data dtype in data.observe…I will directly save the arrays in data.posterior.

@longq

Glad to hear the first line works but unhappy to hear that ArviZ isn’t working as expected for the rest.

If you’d (optionally) like would you mind filing an issue on the ArviZ github? It’ll help us fix the issue so you won’t have problems in the future