PyMC3 NUTS Sampler Progress Bar Stops Before 100%

Questions
1

I am running the following code:

import numpy as np
import matplotlib.pyplot as plt
import pymc3 as pm
import seaborn as sns
#print('Running on PyMC3 v{}'.format(pm.__version__))
plt.style.use('seaborn-darkgrid')

# Initialize random number generator
np.random.seed(123)

# True parameter values
alpha, sigma = 1, 1
beta = [1, 2.5]

# Size of dataset
size = 100

# Predictor variable
X1 = np.random.randn(size)
X2 = np.random.randn(size) * 0.2

# Simulate outcome variable
Y = alpha + beta[0]*X1 + beta[1]*X2 + np.random.randn(size)*sigma

fig, axes = plt.subplots(1, 2, sharex=True, figsize=(10,4))
axes[0].scatter(X1, Y)
axes[1].scatter(X2, Y)
axes[0].set_ylabel('Y'); axes[0].set_xlabel('X1'); axes[1].set_xlabel('X2');
plt.show()

basic_model = pm.Model()
with basic_model:

    # Priors for unknown model parameters
    alpha = pm.Normal('alpha', mu=0, sd=10)
    beta = pm.Normal('beta', mu=0, sd=10, shape=2)
    sigma = pm.HalfNormal('sigma', sd=1)

    # Expected value of outcome
    mu = alpha + beta[0]*X1 + beta[1]*X2

    # Likelihood (sampling distribution) of observations
    Y_obs = pm.Normal('Y_obs', mu=mu, sd=sigma, observed=Y)

map_estimate = pm.find_MAP(model=basic_model, method='powell')

print("Map estimate = ", map_estimate)

with basic_model:
    # draw 500 posterior samples
    trace = pm.sample(1000)
    #mean_field = pm.fit(method='advi')

#pm.plot_posterior(mean_field.sample(1000), color='LightSeaGreen');
#plt.show()

Because of the number of cores on my laptop, the program appears to run with njobs = 2. However, the progress bar for one of them appears to stop short of 100%:

Auto-assigning NUTS sampler...
Initializing NUTS using jitter+adapt_diag...
Multiprocess sampling (2 chains in 2 jobs)
NUTS: [sigma, beta, alpha]
C:\Anaconda2\lib\site-packages\h5py\__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
 from ._conv import register_converters as _register_converters
62%|################################################3                             | 929/1500 [00:01<00:00, 677.11it/s]C:\Anaconda2\lib\site-packages\h5py\__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
 from ._conv import register_converters as _register_converters
100%|#############################################################################| 1500/1500 [00:01<00:00, 777.20it/s]

This seems odd to me. Why did the first job stop at 62%?

2

It is not stopping at 62%, we used to use joblib for parallel sampling, which give us quite some problem and one of them is we can only print progress bar for the first chain. And some times, the progress bar was interrupted by the system output (FutureWarning in this case), so it prints the output, and continue showing the progress in a new threat. If you watch closely, you can see the second progress bar starts at 62% and continue running.

This issue should be now fixed, as we are no longer using joblib for parallel sampling. You can try upgrade to master, or wait a few days as we will release v3.5 soon.

3

Thanks very much for the clarification.