I’m trying to model a mixture of multi-variate Bernoullis, so that my dataset X is an NxP binary matrix and I’m saying that there are K latent groups in the data with different probabilities for each variable, i.e. each X_{i} is drawn from a mixture of K Bernoullis so that there are K x P Bernoulli …

[image] stulacy: I can’t quite get it to work however. I try to manually create a multi-variate Bernoulli for each component as below but I get the error theano.tensor.var.AsTensorError: ('Cannot convert <bound method FreeRV.mean of K_0> to TensorType', <type 'instancemethod'>) which is thrown o…

[image] lucianopaz: About the cluster membership, @junpenglao answered this elsewhere but I can’t seem to find the particular thread to link with. You can basically compute the obs.comp_dist.logp values of each row of observed , for each mixture component separately. This gives you an intuition …

Thanks for the link @junpenglao , that is exactly what I need. Also the pickling issue was occurring on a Linux VM and native Windows install - when I ran it on a native Linux install it was fine. In terms of the mixture modelling, @lucianopaz , running the mixture of the .distributions still seems …

I’m now quite keen to get this working on Windows (where I have a machine with more cores) so have returned back to the pickling issue (no such problem on Linux). I see I have 2 options: Get my current custom logp method picklable Get the Mixture method working (as I assume this won’t have any iss…

I can’t test this right now but I’ll try to give you some pointers to help implement the mixture. When you give Mixture a multivariate distribution as comp_dists, the last axis is assumed to be the mixture components. This means that you should make a (P, K) shaped binomial distribution and pass t…

Ok I tried that and it still complains about the shape of the inputs, it seems to be expecting a PxK matrix in obs. I.e. I’m using: obs = pm.Mixture('obs', w, pm.Bernoulli.dist(mu, shape=(P, K)), shape=(P,), observed=df) And get error messag…

Great that you managed to get the picking working! A shame that the mixture still complains about shape. I see now why, mixture’s logp delegates the logp computation to the distribution comp_dists. That is where the shape raises errors. One last hackish attempt you could try is to reshape your obse…

Adding an extra dimension has got it working! Thanks so much for your help!

Glad to hear it! Nevertheless, I think that adding this extra dimension is hackish and kind of a defect of the current implementation. I’ll see if something can be done about it.

@stulacy : I am trying out your mixture of multivariate Bernoullis code with Pymc version 4. I find that DensityDist has changed, and therefore, the following function non longer works: with pm.Model() as model: # The DP priors to obtain w, the cluster weights alpha = pm.Gamma('alpha', 1., …

Mixture of multivariate Bernoullis

Questions

junpenglao January 31, 2019, 5:48am 3

I believe this is the link you are referring to Get probability of parameter given new data - #4 by junpenglao

Topic		Replies	Views
How can we build a mixture of mixtures? Questions	28	4453	February 17, 2021
Is there an example on how to work with generalized mixture models? Questions	15	3645	March 16, 2019
Problem with fitting Multivariate Mixture of Gaussians Questions	6	3588	June 4, 2018
Sampling many dimensions, with different PDF in each dimension version agnostic modeling	13	784	March 30, 2023
KeyError Setting Mixture Proportions for Mixture Model Questions	26	1531	May 13, 2019

Mixture of multivariate Bernoullis

Related topics