Theano deprecation at 1.0

Development
1

https://groups.google.com/forum/#!topic/theano-users/7Poq8BZutbY

FYI time to plan for the future.

2

Hi all - Iā€™ll be missing the meeting tomorrow. Iā€™ve a work meeting at the same time :frowning: My thoughts are aligned with TF - due to developer mindshare. Iā€™m also ok with the notion of using another backend or multiple backends - to allow us to handle Open Source deprecation risk. cc @ferrine @colcarroll @twiecki @fonnesbeck

1 Like
3

I am not really familiar with TensorFlow or PyTorch, hence I canā€™t say anything about which one should be a good replacement for Theano. But I have a question. Could autograd be and alternative to Theano? One of the pain-points for users of PyMC3 is having to deal with Theano (erros, scan, integrating external functions, etc), using autograd should reduce these problems because everything could be written using just NumPy. Numba could also help to accelerate things if needed.

4

Hi,

I just wanted to come and say that this is interesting option, considering that numba has GPU support. Could this then also use dask/xarray to optimize calculations?

5

I think that TensorFlow is probably the right long-term answer, although we may need to work with them to improve it in some ways first.

6

There is mxnet option too. We can think about. It is supported by Amazon

7

Except for Numba, autograd+GPU is what pytorch offers, all similar to working with Numpy. I have limited experience with deep learning in pytorch but itā€™s definitely very close to the Numpy experience. And it also has the possibility to add external functions. I donā€™t know how much the pymc3 guts would have to be changed in this case.

2 Likes
8

It looks like numba and autograd donā€™t play well together: https://github.com/HIPS/autograd/issues/47

I would put in a vote for PyTorch because:

  1. Itā€™s easy to use and debug because itā€™s tightly integrated with Python.
  2. It doesnā€™t need to be compiled, making iterative development faster.
  3. (the big one) A dynamic graph allows for arbitrarily complex logic, as in a ā€œuniversalā€ probabilistic programming language.
1 Like
9

Iā€™ve not touched the Theano guts of PyMC3 before, so my opinion should be taken with a grain of salt. That said, I have implemented simple DL algorithms with autograd, and I think that the ability to do flow control (conditionals + loops) in autograd-like frameworks (PyTorch, Chainer) make them very attractive for implementation of complicated algorithms.

10

Hi all - first time posting here but Iā€™ve been using PyMC3 for a lot of my thesis work and would really like to help out.

This could be a blessing in disguise - I advocate quite a lot for PyMC3 in my research group but the need to write ops to integrate code already written in numpy/scipy is a big hurdle.

Is there a list of potential replacement options and a ā€˜spec sheetā€™ that they need to fulfil anywhere? As I see it we currently have:

Happy to do some work evaluating these possibilities, but not sure what a ā€˜minimum viable samplerā€™ would be.

11

Dear all,
We recently opened a new Github repository for the experiment of new backends.

If you have idea or code snippet, please send a PR!

12

Theano is open source. I hope (actually, Iā€™m very confident) that someone else will countinue to develop and maintain it.

Btw, in the choice of a new backend for pymc3, I wonder if speed issues are considered.

13

@mcavallaro Donā€™t worry, we will consider speed :slight_smile:
For now we are mostly trying to properly understand the different frameworks, and build some prototypes, so that we can make an informed decision.

1 Like
14

Certainly speed is a big factor. Although I personally wouldnā€™t rank it at the top. Mainly because it can change once we decide on something. TensorFlow for example is pretty slow, but when XLA is merged that might change.

15

autograd is not sufficiently performant to use as the computational backend for PyMC. Itā€™s mostly useful for prototyping. From what I hear, PyTorch was influenced by autograd, however.

1 Like
16

I wrote a question with some issues I encountered in MXNet on their forum: https://discuss.mxnet.io/t/moving-pymc3-from-theano-to-mxnet/86/1

1 Like
17

I should also add this point:

1 Like