I’m having an issue with a simple mixture model. The samples are coming out correctly but they’re a little annoying to use. I have a model for 3 factories that mint coins with different biases:
with mc.Model(): rates = mc.Beta('rates', 2, 2, shape=3) factory_choice = mc.Dirichlet('factory_choice', np.array([1, 1, 1])) factory = mc.Categorical('factory', p=factory_choice, shape=100) flips = mc.Binomial('flips', n=10, p=rates[factory], observed=heads)
Which produces this trace:
The true rates are 0.1, 0.3, 0.8 so the samples look about right when subject to the appropriate squint. What’s annoying is that they oscillate between which class is assigned to which rate, ie [0.1, 0.3, 0.8] is exactly as likely as [0.8, 0.1, 0.3] etc.
Here’s my question: is there a way to enforce a constraint so that rates < rates < rates or something similar?
So far, I tried adding a call to theano.tensor.sort to the rates declaration and its call as the Binomial variable parameter. I also tried tweaking the parameters of the Dirichlet prior to break the symmetry but the of course there’s still local extrema at each permutation of the correct rates and class switching still happens (albeit less frequently). I’d like to avoid baking in too much external bias anyway. I can also sort each sample afterwards but I imagine there may still be some bad behavior that this constraint could limit.