Gradient of a Theano custom operator returning a matrix

Questions
1

I have to write a Theano custom operator which return a matrix

class MyOp(tt.Op):
    itypes = [tt.dscalar, tt.dscalar, tt.dscalar]
    otypes = [tt.dmatrix]
    
    def perform(self, node, inputs, outputs):
        c11, c12, c22 = inputs
        outputs[0][0] = np.array([[c11,c12],[c12,c22]])

Said this, how to implement the grad() method for this operator?
Since the variables are c11, c12 and c22, this method should return the array

[g1,g2,g3]

where

g1=[[1,0],[0,0]]
g2=[[0,1],[1,0]]
g3=[[0,0],[0,1]]

Thanks,
F

2

Is your grad operation performed on matrix? I am a bit confuse - why not take gradient of each input directly?

3

well… yes, but how to do this? In general I have problems in defining custom operators for multivariate (gaussians). I think I missed something in the learning process about Theano. For instance see another question Multivariate Gaussian with Custom Operator Mean. I know how to implement custom operator for univariates and they works, but when I come to multivariates (such as the present example) I do not how to do.

4

In your code above, you dont need to wrap it in a custom operator, as your operation is reshaping the 3 scaler into a matrix, the easiest way to do is something like:

with pm.Model() as m:
    c11 = ...
    c12 = ...
    c22 = ...
    x = tt.zeros((2, 2))
    x = tt.inc_subtensor(x[0, 0], c11)
    x = tt.inc_subtensor(x[0, 1], c12)
    x = tt.inc_subtensor(x[1, 0], c12)
    x = tt.inc_subtensor(x[1, 1], c22)

And theano will take care of the gradient for each scaler.

I will reply to you in Multivariate Gaussian with Custom Operator Mean shortly.