Hello everyone! I am a beginneer to PyMC3, and I got a problem when I try to implement Bayesian Neural Networks (BNN) with PyMC3.
The problem is that I want to combine GMM with BNN during the training process of BNN with NUTS. When I run the following simple code, the problem raise.
####### The problrm code #######
import numpy
from sklearn import mixture
import theano
import theano.tensor as T
import pymc3 as pm
input_nodes = 50 #Dimension of the objective problem
hidden_nodes1 = 10
hidden_nodes2 = 10
hidden_nodes3 = 50
loc1=1
scale1=0.2
#Normal Sampling
x_train = scale1*numpy.random.randn(1000,50)+loc1
y_train = 2*x_train+1
def construct_NN(x_train, y_train):
# set number of nodes in each input,hidden,output layer
# set number of nodes in each input,hidden,output layer
inodes = input_nodes #input layer
hnodes1 = hidden_nodes1
hnodes2 = hidden_nodes2
hnodes3 = hidden_nodes3
#Initial parameter for NN
wh1 = numpy.random.normal(0.0, pow(hnodes1,-0.5), (inodes, hnodes1))
b1 = numpy.random.normal(0.0, pow(hnodes1,-0.5), (hnodes1,))
wh2 = numpy.random.normal(0.0, pow(hnodes2,-0.5), (hnodes1, hnodes2))
b2 = numpy.random.normal(0.0, pow(hnodes2,-0.5), (hnodes2,))
wh3 = numpy.random.normal(0.0, pow(hnodes2,-0.5), (hnodes2, hnodes3))
b3 = numpy.random.normal(0.0, pow(hnodes2,-0.5), (hnodes3,))
with pm.Model() as neural_network:
ann_input = x_train #pm.Data('ann_input1', x_train)
ann_output = y_train #pm.Data('ann_output1', y_train)
# Weights from input to hidden layer
weights_in_1 = pm.Normal('w_in_1', 0, sigma=1,
shape=(input_nodes, hidden_nodes1),
testval=wh1)
# Bias from input to hidden layer
weights_in_b = pm.Normal('w_in_b1', 0, sigma=1,
shape=(hidden_nodes1,),
testval=b1)
# Weights from 1st to 2nd layer
weights_in_2 = pm.Normal('w_in_2', 0, sigma=1,
shape=(hidden_nodes1, hidden_nodes2),
testval=wh2)
# Bias from input to hidden layer
weights_in_b2 = pm.Normal('w_in_b2', 0, sigma=1,
shape=(hidden_nodes2,),
testval=b2)
# Weights from 2nd to 3rd layer
weights_in_3 = pm.Normal('w_in_3', 0, sigma=1,
shape=(hidden_nodes2, hidden_nodes3),
testval=wh3)
# Bias from 2nd to 3rd layer
weights_in_b3 = pm.Normal('w_in_b3', 0, sigma=1,
shape=(hidden_nodes3,),
testval=b3)
ϵ = pm.HalfCauchy('ϵ', 5)
######################################################
#Fit the Multivariate distribution
######################################################
hidden_outputs1 = pm.math.tanh(pm.math.dot(ann_input,weights_in_1)+weights_in_b)
hidden_outputs2 = pm.math.tanh(pm.math.dot(hidden_outputs1,weights_in_2)+weights_in_b2)
hidden_outputs3 = pm.math.tanh(pm.math.dot(hidden_outputs2,weights_in_3)+weights_in_b3)
#Bayesian Gaussian Mixture Model
PDF = mixture.BayesianGaussianMixture(n_components=20,n_init=5,
covariance_type='full').fit(hidden_outputs3)
μ = sum(PDF.means_)
y_pred = pm.Normal('y_pred', mu=μ, sd=ϵ, observed=ann_output)
return neural_network
neural_network = construct_NN(x_train, y_train)
with neural_network:
trace_pce = pm.sample(5000,chains=4)
####### The error output here #######
When I run this code I got the following error information :
“ValueError: setting an array element with a sequence.”
I guss this error comes from :
“PDF = mixture.BayesianGaussianMixture(n_components=20,n_init=5,
covariance_type=‘full’).fit(hidden_outputs3)”
Because the at this stage the term “hidden_outputs3” is not an array. And I don’t known how to deal with this problem.
####### Versions and main components #######
- PyMC3 Version: 3.8
- Theano Version: 1.0.4
- Python Version: 3
- Operating system: Windows 10
- How did you install PyMC3: (conda/pip): pip install
Could you please give me some advices? Thanks for your help.