Bayesian Regressor: Sampling Error: "Bad Initial Energy"

Questions
1

Regards,

I am trying to create a Bayesian Linear Regression model with one independent variable. However, when I try to sample my model using the NUTS sampler, I get the following error: “Sampling Error: Bad Initial Energy”.

The code below shows the approach I followed to build my model:

basic_model = pm.Model()

with basic_model:

   alpha = pm.Gamma('alpha', mu=alpha_mean, sigma=alpha_std, shape=(1))
   beta = pm.Beta('beta', mu=beta_mean, sigma=beta_std, shape=(1))
   sigma = pm.HalfNormal('sigma', tau=1)

   mu = alpha + beta*model_input

   Y_obs = pm.Beta('Y_obs', mu=mu, sigma=sigma, observed=model_output)

with basic_model:

   trace = pm.sample(500, cores=1, init='adapt_diag')

I had pre-calculated the inputs to the model as follows:

# Slice out Variables
X = bayesian_regressor_frame['X']
Y = bayesian_regressor_frame['Y']
X = X.values
Y = Y.values
model_input = theano.shared(X)
model_output = theano.shared(Y)

# Calculate Parameters
alpha_mean = Rolling_OLS_params['alpha'].mean()
alpha_std = Rolling_OLS_params['alpha'].std()
beta_mean = Rolling_OLS_params['beta'].mean()
beta_std = Rolling_OLS_params['beta'].std()

I have taken the liberty to attach the input files that were used to build the model. Where am I going wrong?

bayesian_regressor_frame.csv (113.2 KB) Rolling_OLS_params.csv (102.8 KB)

2

Hi,
There are various reasons why Bad initial energy can happen. Have you tried the solutions in our FAQ?
A few quick suggestions related to this:

  • Make sure your data are standardized somehow if they are on a wide scale.
  • Make sure your priors are not too wide, i.e informative enough – here are good practices on choosing priors. In general, I wouldn’t advice setting your priors with the data’s mean and std. First, this can force the sampler to start in a region of low curvature. Second, you’re using the data twice: once to set the priors, and another time to update them. However, the priors encode the scientific information you have before seeing any data.
  • Make sure there are no inappropriate NaNs somewhere – you’ll find more details in the linked FAQ.
  • Finally, you don’t have to specify shape when the random variable is one-dimensional.

Hope this helps :vulcan_salute:

3

Thanks for the pointers…

I believe that the problem has to do with my priors. I tried the datasets using canned models from the high-level “pycm3_models” module and the sampling proceeded smoothly. So, the problem must be my priors.

Will go through all the links you sent.

1 Like