# About set of Multivariate Normal Distributions

**URL:** <https://discourse.pymc.io/t/about-set-of-multivariate-normal-distributions/1441>\
**Category:** Questions\
**Created:** [June 29, 2018, 2:28pm UTC](https://discourse.pymc.io/t/about-set-of-multivariate-normal-distributions/1441 "2018-06-29T14:28:13Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kyrish](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/kyrish/32/940_2.png) [@Kyrish](https://discourse.pymc.io/u/Kyrish)\
**Post date:** [June 29, 2018, 2:28pm UTC](https://discourse.pymc.io/t/about-set-of-multivariate-normal-distributions/1441/1 "2018-06-29T14:28:13Z")

</div>

Let’s say we have a matrix:

![M =
\begin{bmatrix}
    \mu_{11} & \mu_{12} & \dots & \mu_{1m} \\
    \mu_{21} & \mu_{22} & \dots & \mu_{2m} \\
    \hdotsfor{4} \\
    \mu_{n1} & \mu_{n2} & \dots & \mu_{nm}
\end{bmatrix}](https://tex.s2cms.ru/svg/%20M%20%3D%0A%5Cbegin%7Bbmatrix%7D%0A%20%20%20%20%5Cmu_%7B11%7D%20%20%20%20%20%20%20%26%20%5Cmu_%7B12%7D%20%20%26%20%5Cdots%20%26%20%5Cmu_%7B1m%7D%20%5C%5C%0A%20%20%20%20%5Cmu_%7B21%7D%20%20%20%20%20%20%20%26%20%5Cmu_%7B22%7D%20%20%26%20%5Cdots%20%26%20%5Cmu_%7B2m%7D%20%5C%5C%0A%20%20%20%20%5Chdotsfor%7B4%7D%20%5C%5C%0A%20%20%20%20%5Cmu_%7Bn1%7D%20%20%20%20%20%20%20%26%20%5Cmu_%7Bn2%7D%20%20%26%20%5Cdots%20%26%20%5Cmu_%7Bnm%7D%0A%5Cend%7Bbmatrix%7D%0A)

and a matrix:

![\Sigma =
\begin{bmatrix}
    \sigma_{11} & \sigma_{12} & \dots & \sigma_{1m} \\
    \sigma_{21} & \sigma_{22} & \dots & \sigma_{2m} \\
    \hdotsfor{4} \\
    \sigma_{m1} & \sigma_{m2} & \dots & \sigma_{mm}
\end{bmatrix}](https://tex.s2cms.ru/svg/%20%5CSigma%20%3D%0A%5Cbegin%7Bbmatrix%7D%0A%20%20%20%20%5Csigma_%7B11%7D%20%20%20%20%20%20%20%26%20%5Csigma_%7B12%7D%20%20%26%20%5Cdots%20%26%20%5Csigma_%7B1m%7D%20%5C%5C%0A%20%20%20%20%5Csigma_%7B21%7D%20%20%20%20%20%20%20%26%20%5Csigma_%7B22%7D%20%20%26%20%5Cdots%20%26%20%5Csigma_%7B2m%7D%20%5C%5C%0A%20%20%20%20%5Chdotsfor%7B4%7D%20%5C%5C%0A%20%20%20%20%5Csigma_%7Bm1%7D%20%20%20%20%20%20%20%26%20%5Csigma_%7Bm2%7D%20%20%26%20%5Cdots%20%26%20%5Csigma_%7Bmm%7D%0A%5Cend%7Bbmatrix%7D%0A)

where each row ![\inline (\mu_i_1, \mu_i_2, \dots, \mu_i_m)](https://tex.s2cms.ru/svg/%5Cinline%20(%5Cmu_i_1%2C%20%5Cmu_i_2%2C%20%5Cdots%2C%20%5Cmu_i_m)) of a matrix ![\inline M](https://tex.s2cms.ru/svg/%5Cinline%20M) is supposed to be a mean vector of some m-dimensional Multivariate Normal Distribution with Covariance matrix ![\inline \Sigma](https://tex.s2cms.ru/svg/%5Cinline%20%5CSigma).

And we need to define ![\inilin n](https://tex.s2cms.ru/svg/%5Cinilin%20n) Multivariate Normal Distributions with different mean vectors but with the same covariance matrix inside of a Model. How to achieve this without explicit "for loop" which is a quite performance overhead?

A rough code:

```auto
with pm.Model():
    Sigma = pm.Deterministic(...)
    M = pm.Deterministic(...)
    
    # unoptimal solution
    for i in xrange(n):
        _Y = pm.MvNormal('_Y_' + str(i), mu=mu_[:, i], cov=sd_, shape=(m, ))
    
    # Is it possible to vectorize last lines using pm.MvNormal or pm.MatrixNormal ??

```

Thanks,  
Best Regards  
Alex

---

<div class="post-metadata">

**Author:** ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)\
**Post date:** [June 30, 2018, 9:12am UTC](https://discourse.pymc.io/t/about-set-of-multivariate-normal-distributions/1441/2 "2018-06-30T09:12:18Z")

</div>

If the MvNormal is a latent variable, it is much more straightforward to reparameterized it as `Y = mu + L.dot(s)` with `s ~ Normal(0, 1)`

```python
n, m = 100, 5
Mean = np.random.randn(m, n)*5

with pm.Model() as model:
    sd_dist = pm.HalfNormal.dist(2.5)
    packed_chol = pm.LKJCholeskyCov('chol', n=m, eta=2, sd_dist=sd_dist)
    # compute the covariance matrix
    L = pm.expand_packed_triangular(m, packed_chol, lower=True)
    mu = pm.Normal('mu', 0., 1., shape=(m, n))
    Mv = pm.Deterministic('Mv', L.dot(mu) + Mean)

```

Using MvNormal directly is fine too:

```python
n, m = 100, 5
Mean = np.random.randn(m, n)*5
with pm.Model() as model:
    sd_dist = pm.HalfNormal.dist(2.5)
    packed_chol = pm.LKJCholeskyCov('chol', n=m, eta=2, sd_dist=sd_dist)
    # compute the covariance matrix
    L = pm.expand_packed_triangular(m, packed_chol, lower=True)
    Mv = pm.MvNormal('Mv', Mean.T, chol=L, shape=(n, m))

```

Note that the shape from MvNormal is different compare to above where you use dot product.

---

<div class="post-metadata">

**Author:** ![Kyrish](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/kyrish/32/940_2.png) [@Kyrish](https://discourse.pymc.io/u/Kyrish)\
**Post date:** [July 3, 2018, 9:58am UTC](https://discourse.pymc.io/t/about-set-of-multivariate-normal-distributions/1441/3 "2018-07-03T09:58:55Z")

</div>

Thanks! The second solution is exactly what I need.

Best Regards  
Alex