Increasingly I’m seeing Pyro, Uber’s probabilistic programming extension to PyTorch, pop up in my news feed, job postings, etc. Personally, I find PyTorch syntax to be overly verbose and idiomatic, so I’m hesitant to jump into Pyro. Nonetheless, data science, machine learning, and artificial intelligence circles seem to be flirting with Pyro. So, I’m curious, what are the community’s thoughts on it:
Do we see pyro as an existential threat to PyMC3?
Is Pyro gaining popularity simply since it’s built on a flashy, corporate-funded backend, or does it truly do something unique?
It’s my opinion that probabilistic programming is starting to get the deep learning hype treatment. Cynically, I think that to justify the sudden surge in interest, thought leaders are simply tying the hype to new libraries while marginalizing tools like PyMC3 and/or Stan. Thoughts?
I’ve just looked at a couple of the notebooks, and I think that pyro looks very close neural-network style code; in that it helps to build up a loss function, and then you define your optimizer and go through the training loop. This will be familiar anyone who has taken a course in neural networks, and suddenly has to do inference. It is also probably faster to fit VI, since you have more direct control over the optimizer.
On my side, I haven’t seen any hype or surge in probabilistic programming.
I think there will always be room for a python framework that stylistically emphasizes the model first, and abstracts away the sampling and optimizer as much as possible.
Different PPLs lie on a spectrum of flexibility vs usability. Pyro allows you to do certain things that PyMC3 does not, like stochastic variational inference where you can set which distribution you want to use to fit for a parameter. This comes at the cost of a simpler API. PyMC3 focuses mostly on usability and the Inference Button ™ and simple turn-key inference methods.
The other downside I see is that PyTorch is kinda slow, so running NUTS on a Pyro model should take longer than the same model in PyMC3.
Ah got it, thanks! Since posting this, I’ve come across some commentary that Pyro is really big into variational inference and its various forms, such as SVI, as you noted.
Sounds like Pyro would be a decent choice if you want to explore the middle ground between Bayesian ML and deep learning. I know PyMC3 supports BNNs, but perhaps there’s more flexibility to experiment, provided you know what you’re doing, with Pyro.