Mixture accepts arbitrary distributions as components, in your case you want to pass MvNormal dists as components. The last example in the docstrings has a mixture of Dirichlets which is also multivariate.
When you have multivariate components there is no mixing within the core dimensions.