# Defining custom multivariate Distribution with gradients

**URL:** <https://discourse.pymc.io/t/defining-custom-multivariate-distribution-with-gradients/1770>\
**Category:** Questions\
**Tags:** theano\
**Created:** [August 21, 2018, 3:50pm UTC](https://discourse.pymc.io/t/defining-custom-multivariate-distribution-with-gradients/1770 "2018-08-21T15:50:44Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![axsk](https://avatars.discourse-cdn.com/v4/letter/a/3d9bf3/32.png) [@axsk](https://discourse.pymc.io/u/axsk)\
**Post date:** [August 21, 2018, 3:50pm UTC](https://discourse.pymc.io/t/defining-custom-multivariate-distribution-with-gradients/1770/1 "2018-08-21T15:50:44Z")

</div>

Hello,

I am trying to sample from a custom distribution whose derivatives I also compute myself.  
So far I learned that I have to define my own Theano Op, but I have problems matching the types.

```auto
    import pymc3 as pm
    import theano
    import numpy as np

    class myOp(theano.gof.Op):
        itypes=[theano.tensor.dvector]
        otypes=[theano.tensor.dscalar]

        #def make_node(self, v):
            #v = theano.tensor.as_tensor_variable(v)
            #return theano.Apply(self, [v], [v.type()])

        def __init__ (self):
            self.logp = lambda x: x
            self.dlogp = lambda x: x

        def perform(self, node, inputs, outputs):
            outputs[0] = self.logp(inputs[0])

        def grad(self, inputs, g):
            return [g[0] * self.dlogp(inputs[0])]

    class myDist(pm.distributions.Continuous):
        def __init__ (self, *args, **kwargs):
            self.op = myOp()
            super(myDist, self). __init__ (*args, **kwargs)
        def logp(self, x):
            return self.op(x)

    with pm.Model() as model:
        mydist = myDist('m', testval=np.random.rand(3))

```

This returns

```auto
    TypeError: We expected inputs of types '[TensorType(float64, vector)]' but got types '[TensorType(float64, scalar)]'

```

Right now `logl and dlogl` are just the identity, but I want to replace them by some custom code. Do I really need to define a second Op for the derivative, as suggested in [this post](https://discourse.pymc.io/t/connecting-pymc3-to-external-code-help-with-understanding-theano-custom-ops/670)?

---

<div class="post-metadata">

**Author:** ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)\
**Post date:** [August 21, 2018, 3:58pm UTC](https://discourse.pymc.io/t/defining-custom-multivariate-distribution-with-gradients/1770/2 "2018-08-21T15:58:09Z")

</div>

The easiest is if you can rewrite the likelihood as a theano function, then the gradient is defined automatically. Otherwise, yes you need to define a Op for the derivative, for more see: [http://docs.pymc.io/advanced\_theano.html](http://docs.pymc.io/advanced_theano.html).

---

<div class="post-metadata">

**Author:** ![axsk](https://avatars.discourse-cdn.com/v4/letter/a/3d9bf3/32.png) [@axsk](https://discourse.pymc.io/u/axsk)\
**Post date:** [August 21, 2018, 5:09pm UTC](https://discourse.pymc.io/t/defining-custom-multivariate-distribution-with-gradients/1770/3 "2018-08-21T17:09:21Z")

</div>

The likelihood is based on an external solver, so the pure theano way is not possible. I will go for the second Op then.

But do you have any advice on how to fix the type-issues (accepting a vector)?

---

<div class="post-metadata">

**Author:** ![axsk](https://avatars.discourse-cdn.com/v4/letter/a/3d9bf3/32.png) [@axsk](https://discourse.pymc.io/u/axsk)\
**Post date:** [August 21, 2018, 6:14pm UTC](https://discourse.pymc.io/t/defining-custom-multivariate-distribution-with-gradients/1770/4 "2018-08-21T18:14:22Z")

</div>

I now realized that `logp(self, x)` is being passed the nodes name as value `x`:

```python
import pymc3 as pm

class myDist(pm.Continuous):
    def __init__ (self, **kwargs):
        super(myDist, self). __init__ (**kwargs)
    def logp(self, x):
        print('value:', x)
with pm.Model() as model:
    myDist(name='likelihood', testval=0)

```

returns

```auto
value likelihood
[...]
AsTensorError: ('Cannot convert None to TensorType', <class 'NoneType'>)

```

In the end `logp` represents a posterior (up to normalizing constant) that I want to sample from, but the data/priors will be hardcoded in the `myOp` definitions. So I just want to sample from the multinomial distribution `myDist` (I guess its an unobserved variable in this context).

Do I have to define/instantiate it differently for this to work?

---

<div class="post-metadata">

**Author:** ![axsk](https://avatars.discourse-cdn.com/v4/letter/a/3d9bf3/32.png) [@axsk](https://discourse.pymc.io/u/axsk)\
**Post date:** [August 22, 2018, 12:06am UTC](https://discourse.pymc.io/t/defining-custom-multivariate-distribution-with-gradients/1770/5 "2018-08-22T00:06:32Z")

</div>

So I had many problems in my old code, but I’m halfway done now.

If theres any interest I’ll keep you updated.

---

<div class="post-metadata">

**Author:** ![madarshahian](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/madarshahian/32/722_2.png) [@madarshahian](https://discourse.pymc.io/u/madarshahian)\
**Post date:** [August 24, 2018, 12:37am UTC](https://discourse.pymc.io/t/defining-custom-multivariate-distribution-with-gradients/1770/6 "2018-08-24T00:37:43Z")

</div>

That would be great if you can update us. Very interesting topic.