Using distribution.random(size=number) on NormalMixture RV

Questions
1

The following code compiles

import pymc3 as pm

import theano
theano.config.gcc.cxxflags = "-Wno-c++11-narrowing"

with pm.Model() as model_mixture3:
    z_mix = pm.Normal('z_mix', mu=[0, 6], shape=(2, ))  
    z = pm.NormalMixture('z', w=[0.5, 0.5], mu=z_mix, sd=1)

But when I try to draw a set of 10 random samples,

model_mixture3['z'].distribution.random(size=10)

I get following errror

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-113-fcc4579f6146> in <module>
----> 1 model_mixture3['z'].distribution.random(size=10)

~/.pyenv/versions/3.6.4/envs/test_app/lib/python3.6/site-packages/pymc3/distributions/mixture.py in random(self, point, size)
    167         if (size is None) or (distshape.size == 0):
    168             with draw_context:
--> 169                 comp_samples = self._comp_samples(point=point, size=size)
    170             if comp_samples.ndim > 1:
    171                 samples = np.squeeze(comp_samples[np.arange(w_samples.size), ..., w_samples])

~/.pyenv/versions/3.6.4/envs/test_app/lib/python3.6/site-packages/pymc3/distributions/mixture.py in _comp_samples(self, point, size)
    134     def _comp_samples(self, point=None, size=None):
    135         try:
--> 136             samples = self.comp_dists.random(point=point, size=size)
    137         except AttributeError:
    138             samples = np.column_stack([comp_dist.random(point=point, size=size)

~/.pyenv/versions/3.6.4/envs/test_app/lib/python3.6/site-packages/pymc3/distributions/continuous.py in random(self, point, size)
    458         return generate_samples(stats.norm.rvs, loc=mu, scale=tau**-0.5,
    459                                 dist_shape=self.shape,
--> 460                                 size=size)
    461 
    462     def logp(self, value):

~/.pyenv/versions/3.6.4/envs/test_app/lib/python3.6/site-packages/pymc3/distributions/distribution.py in generate_samples(generator, *args, **kwargs)
    631             dist_shape: {dist_shape}
    632             broadcast_shape: {broadcast_shape}
--> 633         '''.format(size=size, size_tup=size_tup, dist_shape=dist_shape, broadcast_shape=broadcast_shape, test=broadcast_shape[:len(size_tup)] == size_tup))
    634 
    635     # reshape samples here

TypeError: Attempted to generate values with incompatible shapes:
            size: 10
            size_tup: (10,)
            broadcast_shape[:len(size_tup)] == size_tup: False
            dist_shape: ()
            broadcast_shape: (2,)

The equivalent model without marginalization works fine:

with pm.Model() as model_mixture2:
    z_mix = pm.Normal('z_mix', mu=[0, 6], shape=(2, ))  
    dest = pm.Categorical('dest', p=[0.5, 0.5]) 
    z = pm.Normal('z', mu=z_mix[dest], sd=1)

model_mixture2['z'].distribution.random(size=10)
array([ 3.70438843, -1.77756621,  1.06143234,  0.46131986, -2.57417776,
        7.2821125 ,  0.9095331 ,  1.43246977,  4.83188849,  5.48264315]
2

What happens when you try setting the comp_shape like this?

import pymc3 as pm

import theano
theano.config.gcc.cxxflags = "-Wno-c++11-narrowing"

with pm.Model() as model_mixture3:
    z_mix = pm.Normal('z_mix', mu=[0, 6], shape=(2,))  
    z = pm.NormalMixture('z', w=[0.5, 0.5], mu=z_mix, sd=1,
                         comp_shape=(2,))
1 Like
3

@lucianopaz Thanks for your answer.
That works.

By the way, do you have any idea why one has to specify comp_shape in pm.NormalMixture? can it be figured out automatically? Also without comp_shape parameter, why does the code compile and not fail?

4

Yeah, the problem is that distributions cannot infer the resulting shape of a given sample from their tensor parameter values. That means that a normal distribution cannot infer the shape it should have from its mu and sigma tensor parameters. This is why, in order to help sampling, we must usually declare the distribution’s shape by hand.

The reason is that distribution instances are not tensors, nor Ops, so there is no inplace mechanism that allows them to do the symbolic math required to infer shapes.

5

@lucianopaz I see! That makes sense. Thanks for clarification.

1 Like