Constrain one variable to be greater than another

IMO the proper explanation is in Betancourt’s case study Identifying Bayesian Mixture Models
Especially these two paragraphs:

This identification also has a welcome geometric interpretation. The region of parameter space satisfying a given ordering constraint, such as α_1≤ … ≤ α_K, defines a square pyramid with the apex point at zero. The K-dimensional parameter space neatly decomposes into K! of these pyramids, each with a distinct ordering and hence association with a unique labeling.

When the priors are exchangeable the mixture posterior aliases across each of these pyramids in parameter space: if we were given the mixture posterior restricted to one of these pyramids then we could reconstruct the entire mixture distribution by simply rotating that restricted distribution into each of the other K!−1 pyramids. As we do this we also map the mode in the restricted distribution into each pyramid, creating exactly the expected K! multimodality. Moreover, those rotations are exactly given by permuting the parameter indices and reordering the corresponding parameter values.

You can loosely think of it as cutting out some of the regions in your model parameter space so the sampler dont wonders around and becomes more efficient.

1 Like