Hi,
I have a large model that implements several latent GPs in big datasets and uses ADVI for fitting. Right now, the model only runs on a single CPU and I was wondering if it is possible to parallelize the computation using a higher number of resources. How can one control multiprocessing with PyMC3?
Thanks,
Luis
Hi @luisroque, to my knowledge the parallelization of operation used in ADVI depends on the used backend. ADVI cannot be parallelized trivially but some operations it uses may use multiple cores, if possible. So, how often multiple cores are used can depend on the model. You can control the maximum number of used CPU, just like you would control it for numpy, e.g., with threadpoolctl. Further, you may like the upcoming update - PyMC v4 - where backends like JAX and numba become available, that likely support even more parallelization.