Frequently Asked Questions

[ricardoV94]

• Multi-core sampling is very slow / using more cores than specified.

PyMC graphs can invoke computations from other libraries (such as scipy, numpy, PyTensor) that are natively multi-threaded by default. This can cause the number of cores used to be larger than the number of cores specified in sample and may slowdown sampling altogether. Calculating Deterministics on the main thread can also use cores beyond those needed by the individual chains.

See more details and tips about how to control this behavior in the following threads:

Regarding the use of multiple cores

Maybe it helps to explain a bit how the parallelization works: When you specify cores>1 in pm.sample, pymc will start one new process for each chain. The main process then tells cores of those processes to start sampling, the others will just wait and do nothing. When one of the processes is finished, one of the waiting processes is told to start sampling. There will never be more than cores processes working at the same time. If you have n cores in your computer, it makes sense for most models…

Parallelizing chains with custom likelihood on multiple cores

I think the original model is so small, that the parallelization doesn’t do anything useful. If we try something a bit more costly it does: with pm.Model() as model: x = pm.Normal("x", shape=(50, 50)) y = pt.linalg.solve(x + x.T, np.ones(50)).sum() pm.Normal("y", mu=y, observed=100.) times = [] for cores in [1, 4, 8]: start = time.time() with model: pm.sample(cores=cores, chains=cores, tune=100, draws=100, compute_convergence_checks=False) end = time.time() …