The topic is very much related to my work too. My question may be a bit naive: Looking at code, it seems to me that value passed into logp_ of mix_mixlogp contains the whole dataset of both normal and lognormal. So, I am wondering how comp_dist.logp(value) in the list comprehension knows which portion of data is relevant to its distribution for log operation.
Inspired by this thread, I tried to create a mixture model classification case as the following piece of code of mix binomial and normal,
#imports
import sys
import numpy as np
import pymc3 as pm
import theano.tensor as tt
# Generate data
n, p = 1, .5
n_data = 500
data=np.dstack((
np.random.binomial(n, p, size=n_data)+1,
np.random.normal(loc=1,scale=1,size=n_data)
))
Y=np.random.randint(0, high=3, size=n_data, dtype='int')
# Define mixed logp
def mix_mixlogp(w, comp_dists):
def logp_(value):
print(value)
comp_logp = tt.squeeze(tt.stack([comp_dist.logp(value)
for comp_dist in comp_dists], axis=1))
return pm.math.logsumexp(tt.log(w) + comp_logp, axis=-1)
return logp_
# Define model
with pm.Model() as model:
nbr = 1
# mixture components of two features
# A feature of binomial dist data
p = pm.Beta('p', alpha=1, beta=1)
Xge = pm.Binomial('Xge', p=p, n=n, shape=nbr, observed=data[0,:,0])
# A feature of normal dist data
Xnos = pm.Normal("Xnos", mu=0, sd=1,shape=nbr, observed=data[0,:,1])
# weight vector for the mixtures
# Assume mixed features have three distribution components according to Y labels.
mix_w = pm.Dirichlet('mix_w',a=np.array([1]*3))
# mixtures
mixed = pm.DensityDist('mixed', mix_mixlogp(mix_w, [Xge, Xnos]), observed=Y)
I got an error:
TypeError: can't turn [TensorConstant{[0. 1. 0. .. 2. 2. 0.]}] and {} into a dict. cannot convert dictionary update sequence element #0 to a sequence
It points to the cause in def logp_(value)
4 comp_logp = tt.squeeze(tt.stack([comp_dist.logp(value)
----> 5 for comp_dist in comp_dists], axis=1))
It seems try to turn data sequence into a dictionary.
Does my approach make any sense?
Thanks in advance
Chris