Elemwise{abs,no_inplace}.grad error for Multinomial likelihood

Questions v5
1

Hi all,

I have some code that has continuous priors and a multinomial likelihood which is throwing the error:

TypeError: Elemwise{abs,no_inplace}.grad illegally returned an integer-valued variable. (Input index 0, dtype complex128)

The only thing that i think could be causing this error is the multinomial function but both n,p and ‘data’ which I use for the observed argument are all float64. I came across past posts that indicate this error pops up because theano (so now pytensor?) can’t handle complex gradients and in part of the calculation to determine ‘p’, complex values do come up although the calculation is all in a single deterministic node where the absolute is then taken and thus I would think shouldn’t be an issue!

The relevant (tidied up) code block is:

with pm.Model() as model_multinomial:
    #frac = pm.Dirichlet("frac", a=np.ones(M))
    # Define priors
    eta = pm.Normal("eta", mu=0.5, sigma=0.05,initval=0.5)
    #a= pm.Uniform("a", lower=-np.pi, upper=np.pi,initval=0)
    a= pm.Normal("a", mu=0, sigma=np.pi/200,initval=0)
    b= pm.Normal("b", mu=0.7, sigma=0.07,initval=0.5)
    
    Volt=pm.Deterministic("Volt",pt.as_tensor(V_dist))
    phi=pm.Deterministic("phi",(a+b*pm.math.sqr(Volt)))
    p=pm.Deterministic("p", pm.math.sqr(pm.math.abs(pm.math.sqrt(eta)*pm.math.exp(1j*phi/2))))
    pinv=pm.Deterministic("pinv",1-p)
    P=pm.Deterministic("P",pm.math.stack([p,pinv],axis=-1))
    #C=pm.Deterministic("C",1000*pm.math.ones_like(N,dtype=np.float64))
    #need to make joint multinomial (adding log probs across 'experiments')
    likelihood=pm.Multinomial("likelihood",n=C,p=P,shape=(N,M),observed=data)

The full code is:
Neville_MZI.py (5.3 KB)

Any help would be immensely appreciated!

2

Actually PyTensor/Theano tests explicitly know complex grads are not supported: https://github.com/ricardoV94/pytensor/blob/1da2891c7bf8e4cb60d29430e18fa96111f0f401/tests/tensor/test_complex.py#L55-L129

That #178 does not belong to Github tracker, so I don’t know the context of the change.

The message could be better, since it isn’t because the grad is integer, but because it’s complex that the error is being raised.

In your graph you still would get a complex number in the grad at some point because you have a multiplication by a complex constant.

3

I think you can implement this model, but you will have to be a bit clever.

Is eta strictly positive? It’s got a Normal prior, but it’s also set so that “all” of the mass is positive. I ask because if it is, and the only source of complex numbers is the \frac{\phi i}{2}, you could re-write the function with some linear algebra operations, something like:

p = \left \lVert \sqrt{\eta} \begin{bmatrix} \cos{\frac{\phi}{2}} \\ \sin{\frac{\phi}{2}} \end{bmatrix} \right \rVert_2^2

If \eta can be negative (so that it’s potentially a source of a complex component), your job gets a bit harder, but the ideas will stay the same. Just use a switch to handle the positive and negative branches of the square-root function.

4

Yes eta is strictly positive (and bounded between 0 and 1) so that suggestion should in all likelihood work. And thanks to Ricardo’s input I now know why the error popped up.

Many thanks all!