Sampling speeding up

Hey, I was interested in this same thing a couple of months ago. During warmup, it is adjusting the number of steps and the step size. Taking a lot of short steps will cause the progress bar to move slowly. So the tuning process is trying to find a way to take a small number of very long steps. At some point, it uncovers a good balance and there is a massive jump in efficiency. You can explore the tuning process in detail by setting discard_tuned_samples = False.

x = stats.norm(0,1).rvs(100)

with pm.Model():
    mu = pm.Normal('mu',0,1)
    y = pm.Normal('y',mu,1,observed=x)
    trace = pm.sample(chains=1,discard_tuned_samples = False)

Then if look at the step size with plt.plot(trace.warmup_sample_stats.step_size[0])

image

And the number of steps with plt.plot(trace.warmup_sample_stats.n_steps[0]) you can see the jump.

image

3 Likes