Creating a Dirichlet distribution from a random variable

Questions
1

Hi,

I’d like to compile a model that uses a Dirichlet distribution that takes a random variable as a parameter. I tried:

import pymc3 as pm
import numpy as np

def sigmoid(x):
    return 1 / (1 + np.exp(-x))

with pm.Model() as model:
    x = pm.Normal("x")
    s = sigmoid(x)
    w = pm.Dirichlet.dist(np.array([1 - s, s]))
    pm.Mixture("y", w, [pm.Normal(mu=0), pm.Normal(mu=3)], observed=[0.1, 2.9])

But it failed to compile with a long exception message (pasted at the end). How can I create a random simplex that can be passed to a Dirichlet distribution as a parameter?

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
~/miniconda3/lib/python3.6/site-packages/theano/tensor/type.py in dtype_specs(self)
    269                 'complex64': (complex, 'theano_complex64', 'NPY_COMPLEX64')
--> 270             }[self.dtype]
    271         except KeyError:

KeyError: 'object'

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
~/miniconda3/lib/python3.6/site-packages/theano/tensor/basic.py in constant_or_value(x, rtype, name, ndim, dtype)
    249             rval = rtype(
--> 250                 TensorType(dtype=x_.dtype, broadcastable=bcastable),
    251                 x_.copy(),

~/miniconda3/lib/python3.6/site-packages/theano/tensor/type.py in __init__(self, dtype, broadcastable, name, sparse_grad)
     50         self.broadcastable = tuple(bool(b) for b in broadcastable)
---> 51         self.dtype_specs()  # error checking is done there
     52         self.name = name

~/miniconda3/lib/python3.6/site-packages/theano/tensor/type.py in dtype_specs(self)
    272             raise TypeError("Unsupported dtype for %s: %s"
--> 273                             % (self.__class__.__name__, self.dtype))
    274 

TypeError: Unsupported dtype for TensorType: object

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
~/miniconda3/lib/python3.6/site-packages/theano/tensor/basic.py in as_tensor_variable(x, name, ndim)
    205     try:
--> 206         return constant(x, name=name, ndim=ndim)
    207     except TypeError:

~/miniconda3/lib/python3.6/site-packages/theano/tensor/basic.py in constant(x, name, ndim, dtype)
    263     ret = constant_or_value(x, rtype=TensorConstant, name=name, ndim=ndim,
--> 264                             dtype=dtype)
    265 

~/miniconda3/lib/python3.6/site-packages/theano/tensor/basic.py in constant_or_value(x, rtype, name, ndim, dtype)
    258     except Exception:
--> 259         raise TypeError("Could not convert %s to TensorType" % x, type(x))
    260 

TypeError: ('Could not convert [Elemwise{sub,no_inplace}.0 Elemwise{true_div,no_inplace}.0] to TensorType', <class 'numpy.ndarray'>)

During handling of the above exception, another exception occurred:

AsTensorError                             Traceback (most recent call last)
<ipython-input-172-a6b8b36b0427> in <module>()
      5     x = pm.Normal("x")
      6     s = sigmoid(x)
----> 7     w = pm.Dirichlet.dist(np.array([1 - s, s]))
      8     pm.Mixture("y", w, [pm.Normal(mu=0), pm.Normal(mu=3)], observed=[0.1, 2.9])

~/miniconda3/lib/python3.6/site-packages/pymc3/distributions/distribution.py in dist(cls, *args, **kwargs)
     45     def dist(cls, *args, **kwargs):
     46         dist = object.__new__(cls)
---> 47         dist.__init__(*args, **kwargs)
     48         return dist
     49 

~/miniconda3/lib/python3.6/site-packages/pymc3/distributions/multivariate.py in __init__(self, a, transform, *args, **kwargs)
    412 
    413         self.k = tt.as_tensor_variable(shape)
--> 414         self.a = a = tt.as_tensor_variable(a)
    415         self.mean = a / tt.sum(a)
    416 

~/miniconda3/lib/python3.6/site-packages/theano/tensor/basic.py in as_tensor_variable(x, name, ndim)
    210         except Exception:
    211             str_x = repr(x)
--> 212         raise AsTensorError("Cannot convert %s to TensorType" % str_x, type(x))
    213 
    214 # this has a different name, because _as_tensor_variable is the

AsTensorError: ('Cannot convert [Elemwise{sub,no_inplace}.0 Elemwise{true_div,no_inplace}.0] to TensorType', <class 'numpy.ndarray'>)
2

The correct syntax goes like this:

import pymc3 as pm
import theano.tensor as tt
def sigmoid(x):
    return 1 / (1 + tt.exp(-x))

with pm.Model() as model:
    x = pm.Normal("x")
    s = sigmoid(x)
    w = pm.Dirichlet('w', a=tt.stack([1. - s, s]), shape=2)
    pm.Mixture("y", w, [pm.Normal.dist(mu=0), pm.Normal.dist(mu=3)], observed=[0.1, 2.9])

If you are using Normal mixture there is also a pm.NormalMixture: pm.NormalMixture("y", w, mu=np.array([0.,1.]), sd=1., observed=[0.1, 2.9]).

More information could be found in http://docs.pymc.io/notebooks/dependent_density_regression.html

3

Thank you! It works perfectly.

I’m not actually using normal distributions but the link is helpful.

4

You can further condense the code through using pm.math.sigmoid rather than defining it manually