Chains parameter and Multiprocess sampling

I have a model that I believe has a big surface to explore, so I have passed to the sample function a high draws parameter, and I am also passing 4 as the number of cores:

model.trace = pm.sampling.sample(num_draws=10000, num_chains=20, num_cores=4, trace=db)

But when I run this, I see the following:

Multiprocess sampling (4 chains in 4 jobs)

and indeed when I look at the CSV files, there are only four of them, not 20. I have tried to track this down in the source, but I’m getting lost. Can someone explain what’s going on here, and how I can get a number of chains that is not equal to the number of cores?


You are not inputting the right kwargs, example see below:

In [2]: with pm.Model():
   ...:     mu = pm.Normal('mu')
   ...:     x = pm.Normal('x', mu, observed=1.)
   ...:     trace = pm.sample(1000, chains=20, cores=4)
Auto-assigning NUTS sampler...
Initializing NUTS using jitter+adapt_diag...
Multiprocess sampling (20 chains in 4 jobs)
NUTS: [mu]
Sampling 20 chains: 100%|████████████| 30000/30000 [00:08<00:00, 3594.84draws/s]
The acceptance probability does not match the target. It is 0.8881342132914561, but should be close to 0.8. Try to increase the number of tuning steps.

Thank you!!! It would have taken me forever to figure this out. Not sure why those keywords got stuck in my head. I know why you need them, but this is one of the hazards of **kwargs.

1 Like

One more thing: I revised the call to sample per your instructions, and got 20 chains, but then PyMC3 errored out in _run_convergence_checks, because it found a trace whose shape was (20,) and it expected a 2D array.

I will investigate this further, and make a bugreport, but I thought I would ask you if I was doing something obviously wrong. My first guess was that maybe the set of variables was empty, but that is not the problem: I checked the varnames in _run_convergence_checks() and it’s what I expected. I will check to see if it could be related to my use of a pm.backends.Text. But any suggestions would be welcome.

Thanks as always for your thoughtful answers.

OK, I think I see what’s wrong. The MultiTrace I have has, for one variable, a python array of length 20 (one per chain), each of whose entries is an ndarray of (2500,).
I believe that the code here expects either a single python array that would be (20, 2500) in shape, or a single ndarray likewise, but instead of a 2D array it gets a one-dimensional array, each of whose entries is a 1D array.

Yes, it looks like it’s the use of the text backend that causes this to break. When I drop the trace keyword argument, the call to sample terminates successfully.


1 Like