Initial evaluation of model at starting point failed when running a mixture model

Questions
1

Hi, when I was running a code as below, pymc3 shows an error message and I’m not sure what to do. Any help would be appreciated!
with pm.Model() as mnl_model:

hyper_mean = pm.Normal("hyper_mean", mu = 0, sigma = 100)
sigma_2 = pm.InverseGamma("sigma_2", alpha = 0.01, beta = 0.01)
sigma = pm.Deterministic("sigma", sigma_2**0.5)

component = pm.Uniform.dist(lower=0, upper=0)
component1 = pm.Normal.dist(mu=hyper_mean, sd=sigma)
component2 = pm.Laplace.dist(mu=hyper_mean, b=(sigma_2/2)**0.5)

w = pm.Dirichlet('w', a=np.array([1, 1, 1]))
like = pm.Mixture('like', w=w, comp_dists=[component, component1,component2], observed=effect)

step = pm.Metropolis()
trace_mnl =pm.sample(20000, step=step)
#trace_mnl = pm.sample(20000, step = step, cores=1, start={'hyper_mean': np.array(0.), 'sigma_2_log__': np.array(0.), 'w_stickbreaking__': np.array([0., 0.])})
burned_trace_mnl=trace_mnl[1000:]

The error message is as follows:

SamplingError Traceback (most recent call last)
in
13
14 step = pm.Metropolis()
—> 15 trace_mnl =pm.sample(20000, step=step)
16 #trace_mnl = pm.sample(20000, step = step, cores=1, start={‘hyper_mean’: np.array(0.), ‘sigma_2_log__’: np.array(0.), ‘w_stickbreaking__’: np.array([0., 0.])})
17 burned_trace_mnl=trace_mnl[1000:]

~/opt/anaconda3/lib/python3.8/site-packages/pymc3/sampling.py in sample(draws, step, init, n_init, start, trace, chain_idx, chains, cores, tune, progressbar, model, random_seed, discard_tuned_samples, compute_convergence_checks, callback, jitter_max_retries, return_inferencedata, idata_kwargs, mp_ctx, pickle_backend, **kwargs)
426 start = deepcopy(start)
427 if start is None:
→ 428 check_start_vals(model.test_point, model)
429 else:
430 if isinstance(start, dict):

~/opt/anaconda3/lib/python3.8/site-packages/pymc3/util.py in check_start_vals(start, model)
235
236 if not np.all(np.isfinite(initial_eval)):
→ 237 raise SamplingError(
238 “Initial evaluation of model at starting point failed!\n”
239 “Starting values:\n{}\n\n”

SamplingError: Initial evaluation of model at starting point failed!
Starting values:
{‘hyper_mean’: array(0.), ‘sigma_2_log__’: array(-4.61512052), ‘w_stickbreaking__’: array([0., 0.])}

Initial evaluation results:
hyper_mean -5.52
sigma_2_log__ -5.61
w_stickbreaking__ -1.50
like NaN
Name: Log-probability of test_point, dtype: float64

2

The distribution component = pm.Uniform.dist(lower=0, upper=0) seems odd. The code stands for a uniform distribution over the interval [0,0]. I am not sure how PyMC3 interprets this and this probably causes trouble. Mixing unbound distributions with bound distributions may also not be a good idea.

The following function might help you debug: mnl_model.check_test_point(). If the output shows some -np.inf your start point is too far from the support of the posterior.

3

I appreciate your help! I use pm.Uniform.dist(lower=0, upper=0) to specify a mass point in my distribution. I applied the suggested function and found out this error message is because there are too many 0 in my observation. I solved this problem by adding 0.0000001 to all the observations with a value of 0. Thank you so much for your help again, and the suggested function is very useful!