How to discover deep mixture model parameters

Questions
2

I would expect it to work as well, maybe try setting stronger prior on the weight, something like Dirichlet(.5, .5) so that it push the latent label to either 0 or 1.