Chains parameter and Multiprocess sampling

Questions
1

I have a model that I believe has a big surface to explore, so I have passed to the sample function a high draws parameter, and I am also passing 4 as the number of cores:

model.trace = pm.sampling.sample(num_draws=10000, num_chains=20, num_cores=4, trace=db)

But when I run this, I see the following:

Multiprocess sampling (4 chains in 4 jobs)

and indeed when I look at the CSV files, there are only four of them, not 20. I have tried to track this down in the source, but I’m getting lost. Can someone explain what’s going on here, and how I can get a number of chains that is not equal to the number of cores?

Thanks!

2

You are not inputting the right kwargs, example see below:

In [2]: with pm.Model():
   ...:     mu = pm.Normal('mu')
   ...:     x = pm.Normal('x', mu, observed=1.)
   ...:     trace = pm.sample(1000, chains=20, cores=4)
   ...:     
Auto-assigning NUTS sampler...
Initializing NUTS using jitter+adapt_diag...
Multiprocess sampling (20 chains in 4 jobs)
NUTS: [mu]
Sampling 20 chains: 100%|████████████| 30000/30000 [00:08<00:00, 3594.84draws/s]
The acceptance probability does not match the target. It is 0.8881342132914561, but should be close to 0.8. Try to increase the number of tuning steps.
3

Thank you!!! It would have taken me forever to figure this out. Not sure why those keywords got stuck in my head. I know why you need them, but this is one of the hazards of **kwargs.

1 Like
4

One more thing: I revised the call to sample per your instructions, and got 20 chains, but then PyMC3 errored out in _run_convergence_checks, because it found a trace whose shape was (20,) and it expected a 2D array.

I will investigate this further, and make a bugreport, but I thought I would ask you if I was doing something obviously wrong. My first guess was that maybe the set of variables was empty, but that is not the problem: I checked the varnames in _run_convergence_checks() and it’s what I expected. I will check to see if it could be related to my use of a pm.backends.Text. But any suggestions would be welcome.

Thanks as always for your thoughtful answers.

5

OK, I think I see what’s wrong. The MultiTrace I have has, for one variable, a python array of length 20 (one per chain), each of whose entries is an ndarray of (2500,).
I believe that the code here expects either a single python array that would be (20, 2500) in shape, or a single ndarray likewise, but instead of a 2D array it gets a one-dimensional array, each of whose entries is a 1D array.

6

Yes, it looks like it’s the use of the text backend that causes this to break. When I drop the trace keyword argument, the call to sample terminates successfully.

7

Bugreport https://github.com/pymc-devs/pymc3/issues/3179

1 Like