The number of effective samples is smaller than 25% for some parameters

#1

I’m trying to run the code in chapter 1 of the book that Thomas Wiecki recommended: Probabilistic Programming and Bayesian Methods for Hackers.
However, I get the error: “The number of effective samples is smaller than 25% for some parameters.”

The data being analyzed is the number of text messages sent per day for a period of 70 days.

``````with pm.Model() as model:
alpha = 1.0/count_data.mean()
lambda_1 = pm.Exponential("lambda_1", alpha)
lambda_2 = pm.Exponential("lambda_2", alpha)

tau = pm.DiscreteUniform("tau", lower=0, upper=n_count_data - 1)

idx = np.arange(n_count_data)
lambda_ = pm.math.switch(tau > idx, lambda_1, lambda_2)

observation = pm.Poisson("obs", lambda_, observed=count_data)

step = pm.Metropolis()
trace = pm.sample(10000, tune=5000,step=step)
``````

What does this error mean exactly, and why am I getting it? I tried changing sample numbers but that didn’t help.

#2

The low number of effective samples is usually an indication of strong autocorrelation in the chain. You can plot the trace autocorrelation or look at the trace plot directly.

Usually, this could be improved by using more efficient sampler like NUTS. However, in this case since you are mixing discrete random variable with continuous variables, sampling is likely going to be a bit difficult.

You can find more information about effective sample size from Gelman et al’s book Bayesian Data Analysis. The stan language manual has some detail information on how they are computed.

I think the warning threshold in PyMC3 is currently a bit too sensitive (for example, in Stan they only output warning when < .01%). But big thumbs up to you for actually read the warning and try to improve your model!!

#3

I modeled it again without assigning ‘step’, so PyMC3 can automatically assign an appropriate sampler:

``````with model:
trace = pm.sample(10000, tune=5000)
``````

As a result, pymc3 used two samplers:

NUTS: [lambda_2_log__, lambda_1_log__]
Metropolis: [tau]

I still got the same warning of effective sample < 25%. I’m assuming this is due to the Metropolis sampler as opposed to the NUTS. So i continued reading…
This post (https://stats.stackexchange.com/questions/212442/what-is-causing-autocorrelation-in-mcmc-sampler) mentions: “when using MH algorithms, to some extent you can reduce or increase your autocorrelations by adjusting the step size of proposal distribution”.

I looked at the pymc3 documentation but couldn’t find ways to reduce step size for Metropolis (https://docs.pymc.io/api/inference.html#module-pymc3.step_methods.metropolis). How can I do that? Do you think that would help?