User experience: Python vs R, PyMC vs Stan, PyTensor vs JAX

[aseyboldt]

@bob-carpenter

This is the common refrain I’ve heard from PyMC users—they really really don’t want to learn another language on top of Python. For reasons I don’t understand, the R users are less resistant. Maybe because the syntax is more similar? I would have expected more resistance because R is also implicitly typed. If I had it to do over again, I would have followed NumPy’s structure and not tried to differentiate arrays and linear algebra types the way that Boost (C++ matrix library) does. The advantage of doing it the way Boost does is that you can infer the types of the result. A vector times a row vector is a matrix and a row vector times a vector is a scalar. Back when I was a professor I worked on a mix of PL (programming languages) and NLP (natural language processing), largely around typing.

The numpy broadcasting rules are actually a bit of a pain to work with in pytensor too. If you want to follow it strictly, you can’t tell until runtime what for instance a multiplication of two arrays will actually do. Depending on if a shape is one, it might broadcast, error out, or do elementwise multiplication. But which one happens has implications for the derivatives, because those won’t be the same in those cases.
If I could go back in time, I’d love to tell the numpy devs to not use a dimension length of 1 to indicate that something can be broadcast, but a dimension length None instead. I think that would have prevented a lot of strange bugs over the years…