TensorFlow backend for PyMC4


#1

Should we use TensorFlow for PyMC4? What would be the issues of doing so?

Pros are that some successful approaches already exist in the forms of (1) Edward (http://edwardlib.org/) which will be moved into TensorFlow.contrib which might ensure that TensorFlow will have more capabilities to support Probabilistic Programming, and (2) ZhuSuan https://github.com/thu-ml/zhusuan.


#2

Discussion elsewhere, input welcomes
Tensorflow(Backed by Google)

  • Pros:
    • Strong support, large community
    • Tensorboard for visualization
    • parallel within graph execution that makes things fast
    • Convenient deployment
  • Cons:
    • Slow?
    • Too many similar package already (e.g., Edward, ZhuSuan, Greta in R)

Other remarks:
Compiler for graphs in development: XLA. It is supposed to support memory optimisations and Op fusion when it’s done.
Custom Ops can be written in python, but C++ ops require us to play games with that strange build system (written in java, and probably not going to end up in any linux packages)


#3

To my knowledge of the PyMC3 code, an important issue is that TF doesn’t support a theano.clone() equivalent officially. This will make it hard to reuse model graphs. May need to come up with an inference scheme that doesn’t rely on graph copying.


#4

I agree with the Pros of TensorFlow and I see the concern of similar packages. Just curious, has there been any conversation about a potential merge with Edward or ZhuSuan? I believe NumPy started out in a similar way. . . I prefer PyMC over the others by far. I also think there are more resources and PyMC than Edward or ZhuSuan; that I have seen at least. I think if TensorFlow was in the backend it would open up a lot of the community to probabilistic programming which would mean more resources and knowledge flowing around.


#5

Does this answer the question?