I’m having an issue with a simple mixture model. The samples are coming out correctly but they’re a little annoying to use. I have a model for 3 factories that mint coins with different biases:

The true rates are 0.1, 0.3, 0.8 so the samples look about right when subject to the appropriate squint. What’s annoying is that they oscillate between which class is assigned to which rate, ie [0.1, 0.3, 0.8] is exactly as likely as [0.8, 0.1, 0.3] etc.

Here’s my question: is there a way to enforce a constraint so that rates[0] < rates[1] < rates[2] or something similar?

So far, I tried adding a call to theano.tensor.sort to the rates declaration and its call as the Binomial variable parameter. I also tried tweaking the parameters of the Dirichlet prior to break the symmetry but the of course there’s still local extrema at each permutation of the correct rates and class switching still happens (albeit less frequently). I’d like to avoid baking in too much external bias anyway. I can also sort each sample afterwards but I imagine there may still be some bad behavior that this constraint could limit.

Thanks for your help! Unfortunately this immediately causes an error for me in the Beta definition:

ValueError: The index list is longer (size 1) than the number of dimensions
of the tensor(namely 0). You are asking for a dimension of the tensor that
does not exist! You might need to use dimshuffle to add extra dimension
to your tensor.

The offending line is

out = tt.inc_subtensor(out[0], x[0])

where it seems x == 0.5, not sure where that comes from.

Anyway, I’m new to pymc and theano so it’s tough for me to figure this one out. I’ll start reading up on theano and see if anything sticks out.

No problem, glad I could help.
If you’d like to improve convergence further, you could try to marginalize out the discrete parameters. Sadly, we don’t have a good guide about how to do that yet, but you could have a look at the stan documentation, they have a chapter about this (Chapter 15.2 in https://github.com/stan-dev/stan/releases/download/v2.16.0/stan-reference-2.16.0.pdf).