I just tested some sample code from http://twiecki.github.io/blog/2017/02/08/bayesian-hierchical-non-centered/ I tried both using gpu and cpu with identical code. This is the first time I observed cpu running much faster than gpu, by something that seems like 10 times. Is there some obvious reason…

Running on GPU has a large overhead because the sampler are in Python (CPU) but the gradient and the logp function is under GPU (via theano). So unless you have very large matrix operations with expensive gradient computation that could leverage the GPU performance, using GPU is usually not worth.

It will slow down the whole thing, not just the compiling. Because at each step, the data is copied to GPU for gradient evaluation, and copied back to CPU for leapfrog.

Gpu much slower than cpu

Questions

bhomass May 14, 2018, 5:18am 3

That’s plausible. is this overhead consistent with the slowing down after the NUTS % counting started? I mean its not just in the beginning of the program execution that suffers from the cpu sampler overhead.

Topic		Replies	Views
Sampling time GPU vs CPU v5 modeling	1	3073	February 7, 2023
GPU utilization is high but memory usage is very low leading to subpar sampling performance Questions gpu , theano	2	6286	August 26, 2017
GPU is running the model much slower than the CPU v5	7	1599	August 16, 2023
Slow sampling with pm.sampling_jax.sample_numpyro_nuts() on Vertex AI v5 modeling	1	438	September 6, 2024
Sampling became much slower after upgrading pymc3 Questions gpu	3	2163	April 12, 2020

Gpu much slower than cpu

Related topics