I wanted to use the package to perform MCMC for a very simple neural network model, but there is some error which I am not able to figure out (I am not very familiar with the pymc3 and theano). The codes are the following. Appreciate if you can offer any idea!
import numpy as np
import scipy.stats as ss
import pymc3 as pm
import theano.tensor as tt
d_i = 2 #number of inputs
d_o = 2 #number of outputs
hidden_units = 2 # number of dimension
sigma = 0.01 # sigma of prior distribution
sigma2_e = 0.000001
nnmodel = pm.Model()
data_x = np.loadtxt('input.txt',dtype='f', delimiter=' ')
data_y = np.loadtxt('output.txt',dtype='f', delimiter=' ')
ns = np.size(data_x,0) # the number of data samples
with nnmodel:
x1 = pm.Normal('x1', mu=0, tau=1, shape=(d_i+1)*hidden_units )
x2 = pm.Normal('x2', mu=0, tau = 1, shape=(1+hidden_units)*d_o )
mu0 = np.zeros((1,d_o*ns))
cove = np.eye(d_o*ns)*sigma2_e
b1 = x1[d_i*hidden_units : ]
b1 = np.tile(b1, (ns,1))
W1 = x1[:d_i*hidden_units] #assign weights associated to connection between input and hidden layer
W1 = W1.reshape((d_i, hidden_units))
W2 = x2[:d_o*hidden_units] #assign weights associated to connection between output and hidden layer
W2 = W2.reshape((hidden_units, d_o))
b2 = x2[d_o*hidden_units : ]
b2 = np.tile(b2, (ns,1))
angle = tt._shared(data_x) # convert format of input np.array data
W1 = tt._shared(W1)
b1 = tt._shared(b1)
W2 = tt._shared(W2)
b2 = tt._shared(b2)
z1 = tt.dot(angle, W1) + b1
a1 = np.tanh(z1)
out = tt.dot(a1, W2) + b2
dif = out - data_y
dif = np.reshape(dif, (1, d_o*ns))
mu0 = tt._shared(mu0)
cove = tt._shared(cove)
y = pm.MvNormal('y', mu = mu0, cov =cove, observed=dif )
trace = pm.sample(5000,core=12)
My error is
W_1 = tt._shared(W1)
File "/software/Anaconda/5.3.1/lib/python3.7/site-packages/theano/tensor/sharedvar.py", line 45, in tensor_constructor
raise TypeError()
You can just remove these lines as they are already a tensor. Also, within a pm.Model block all operations are tensors - you should replace all the np.{} function with theano functions.
Thanks a lot. I corrected my code and it can run now. However, I got a new problem when trying to save the samples: I added a line “db = pm.backends.NDArray(‘nnbayes’)” beneath “with nnmodel:” and changed trace = pm.sample(5000, core=12, trace=db) as instructed by the online document. But the error message says: File “/utrc/home/longq/multimode/NN/bayes.py”, line 72, in
trace = pm.sample(50000,core=4, trace=db)
File “/utrc/software/Anaconda/5.3.1/lib/python3.7/site-packages/pymc3/sampling.py”, line 439, in sample
trace = _mp_sample(**sample_args)
File “/utrc/software/Anaconda/5.3.1/lib/python3.7/site-packages/pymc3/sampling.py”, line 994, in _mp_sample
trace.record(draw.point, draw.stats)
File “/utrc/software/Anaconda/5.3.1/lib/python3.7/site-packages/pymc3/backends/ndarray.py”, line 229, in record
for varname, value in zip(self.varnames, self.fn(point)):
File “/utrc/software/Anaconda/5.3.1/lib/python3.7/site-packages/pymc3/model.py”, line 1108, in call
return self.f(**state)
File “/utrc/software/Anaconda/5.3.1/lib/python3.7/site-packages/theano/compile/function_module.py”, line 797, in call
raise TypeError(“Too many parameter passed to theano function”)
TypeError: Too many parameter passed to theano function
The text backend is quite buggy and we will likely deprecate it soon. I suggest you to have a look at https://github.com/arviz-devs/arviz. It transformed the sample to xarray that is much easier and clearer to serialized/saved.
@longq
These three methods from ArviZ could come in useful
data = az.from_pymc3(trace=trace) az.to_netcdf(data, "myfile.nc") az.to_from("myfile.nc")
In words the first one converts the PyMC3 trace to az.InferenceData which is a standard container for Inference results. And the other two save or load the data from disk.
Thanks for the message @RavinKumar
The first line works. But it seems az is not able to handle the data dtype in data.observe…I will directly save the arrays in data.posterior.