On mixtures of normals in the documentation

Questions
1

In the documentation of mixture model (https://docs.pymc.io/api/distributions/mixture.html)
I saw the following snippet:

What’s the difference between the sigma in pm.Normal and the sigma in pm.Normal.dist?

npop = 5
nd = (3, 4)
with pm.Model() as model:
    mu = pm.Normal('mu', mu=np.arange(npop), sigma=1, shape=npop) # Each component has an independent mean

    w = pm.Dirichlet('w', a=np.ones(npop))

    components = pm.Normal.dist(mu=mu, sigma=1, shape=nd + (npop,))  # nd + (npop,) shaped multinomial

    like = pm.Mixture('like', w=w, comp_dists = components, observed=data, shape=nd)  # The resulting mixture is nd-shaped
2

The pm.Normal defines the prior distribution of the component means. pm.Normal.dist is the component-specific distribution. So the exact mean of each component is unknown but has the prior distribution given by pm.Normal and the random variables of a given component i are known to be distributed as given by pm.Normal.dist(mu=mu[i], sigma=1).

Setting a small sigma in pm.Normal.dist and a large sigma in pm.Normal would result in a mixture distribution that consists of narrow peaks that are far apart. sigma of pm.Normal.dist corresponds to the width of the peaks and sigma of pm.Normal to how sparsely they are distributed.

3 Likes
3

Also, from a program execution point of view, pm.Normal.dist(...) returns an object with type <pymc3.distributions.continuous.Normal>, it is not a random variable and will not be added to your model directly (as one of the node in the computation). And since it is not a variable you cannot do operation on it (you cannot multiple, add, etc).

4

Thank you for the answers and insights. @Dominik @junpenglao