I realize this is an old topic, but I figure it may still be of interest for many. Ray has already been mentioned, and it has developed incredibly fast since the question was posted. I haven’t tried it myself, but it should be possible to use Ray at least via JAX: PyMC3 can use JAX as a backend (see here: Using JAX for faster sampling — PyMC3 3.10.0 documentation), and JAX can utilize Ray. There has just been a keynote on this combination by JAX developer Matt Johnson on the Ray Summit that is currently taking place.
3 Likes