Hi there, I’m having an issue with a simple mixture model. The samples are coming out correctly but they’re a little annoying to use. I have a model for 3 factories that mint coins with different biases: with mc.Model(): rates = mc.Beta('rates', 2, 2, shape=3) factory_choice = mc.Dirichlet…

You can restrict the rates to be ordered: class Ordered(pymc3.distributions.transforms.ElemwiseTransform): name = "ordered" def forward(self, x): out = tt.zeros(x.shape) out = tt.inc_subtensor(out[0], x[0]) out = tt.inc_subtensor(out[1:], tt.log(x[1:] - x[:-1])) …

@aseyboldt wow this is a very nice trick! I was doing # use sort to break multimodality rate_sort = pm.Deterministic('mixing', tt.sort(rate)) But it breaks the geometry. [Edit] actually tt.sort also has the correct gradient. Could you please add this to pymc3? It is very useful for workin…

Thanks for your help! Unfortunately this immediately causes an error for me in the Beta definition: ValueError: The index list is longer (size 1) than the number of dimensions of the tensor(namely 0). You are asking for a dimension of the tensor that does not exist! You might need to use dimshuff…

Ah, I posted old code that used to work in slightly different circumstances. Lesson for next time: Always execute code before posting… This should work (tested on master): class Ordered(pm.distributions.transforms.ElemwiseTransform): name = "ordered" def forward(self, x): out = tt…

This is seriously amazing. Thank you so much! [image]

No problem, glad I could help. :slight_smile: If you’d like to improve convergence further, you could try to marginalize out the discrete parameters. Sadly, we don’t have a good guide about how to do that yet, but you could have a look at the stan documentation, they have a chapter about this (Chap…

Or a special Gibbs-MH sampler… which we do not have currently as well…

Or discrete hamiltonian, which we will hopefully get soon :wink:

@junpenglao , since tt.sort also has the correct gradient, is it recommend to it this way or @aseyboldt ’s way? tt.sort seems a lot cleaner.

tt.sort should give the correct answer. You should go with @aseyboldt ’s solution as the sorting does not produce the correct logp

Mixture models and breaking class symmetry

Questions

Topic		Replies	Views
Error using Ordered() in Mixture of Uniform Distributions Questions	14	1925	February 7, 2019
Constrain one variable to be greater than another Questions	14	3609	January 30, 2018
Mixture of bounded normals as a prior Questions	4	1384	April 27, 2018
Constraining order in a Mixture model Questions	6	1497	August 9, 2021
Complex ordering mixture models	7	292	April 25, 2024

Mixture models and breaking class symmetry

Related topics