I was hoping someone could clarify a few things for me when it comes to the use of multiple CPU cores in models. reading the docs it states chainsint The number of chains to sample. Running independent chains is important for some convergence statistics and can also reveal multiple modes in the p…

Maybe it helps to explain a bit how the parallelization works: When you specify cores>1 in pm.sample, pymc will start one new process for each chain. The main process then tells cores of those processes to start sampling, the others will just wait and do nothing. When one of the processes is finish…

Thanks @aseyboldt for the informative answer. I now have a clear grasp of how the work is distributed for the sampler. I will definitely try to change the number of threads and test if it helps performance. Will the number of cores specify the number of processes for each core by default? Like in yo…

Is there documentation / examples of how to set those variables for BLAS or etc? This is a simple four-chain estimate. It uses 128 threads!! : [pymc-sampling-htop] Crazy! And I’m not sure how efficient this is? Aren’t all those kernel processes (ie the red color in the load bars) bad news?

Regarding the use of multiple cores

Questions

junpenglao December 23, 2019, 10:36am 2

maybe @aseyboldt knows a bit more on this.

Cores=x doesn't impact the number of cores actually used

Topic		Replies	Views
Cores not optimally used version agnostic bug	16	279	November 26, 2024
What is the effect of running sample() with 4 chains on 16 cores? Questions	2	543	May 7, 2019
Number of cores settings in sampling Questions	3	736	October 30, 2021
Does anyone know of any guides or docs for parallel sampling in current version of PYMC (mp_ctx) v5	3	1307	July 24, 2023
Sample with multiple cores Questions	3	1547	September 10, 2020

Regarding the use of multiple cores

Related topics