Multilevel conditional multinomials in MCMC

Questions
1

Hi there,

In my case, I have a multinomial diagnosis variable taking values {“A”, “B”, “C”}. Each of them may have different scores represented by a multinomial score variable. For example, “A” has its score domain {1, 2} while “B” takes {1, 2, 3, 4}. Each record takes one diagnosis value and one corresponding score value (e.g. “A”, 2). They have unknown categorical probabilities (parameters) for these multinomials. All have to be learned from data.

In Bayesian network, the score variable (child) is conditioning on the diagnosis variable (parent). From textbook, these categorical probabilities can be counted and normalized through data with a Dirichlet prior at both parent and child levels. How this parent-child causal relationship can correctly be set up in PyMC3 (my second question indeed). In our case, the multinomial variables are inputs (X), along with the other continuous variables (conditionally independent from multinomials), used for classifying treatments and thereafter estimated treatment distributions.

Set up parent multinomial seems straightforward like
alphas_diagnosis = np.array([1., 1., 1.])
p_diagnosis = Dirichlet(“p_diagnosis”, a=alphas_diagnosis)
Xdiagnosis_ = Multinomial(n=data.shape[0], p=p_diagnosis, observed=data[:0])

Could anyone point to me the way set up the child multinomial conditioning on the parent?

A side note: the experiment also noticed that the ‘observed’ argument takes the array of the summarized counts of category occurrences instead of the array of each individual observed categorical value.

Best regards
Chris

2

Depending on whether the parent process is observed or not. If it is observed, then you can write down the child process nested under each parent process:

for parent in ['A', 'B', 'C']:
    pm.Multinomial('child', n, p=p[parent], observed=...)

If the parent is not observed, then some thing like a latent dirichlet allocation model should work. You can have a look at a recent discussion: Naive Bayes model with PyMC3

3

Thanks Junpenglao. The parents are observed indeed. Does it mean that the parents become the prior of the child with this code? I would think that the child has its own Dirichlet prior since it has different discrete value domain from the parents’. Could you please elaborate a bit further on this? The PyMC3 syntax becomes unclear once the model requires a bit complicated construction. Many thanks.

4

No - you still need prior for the child. Actually, since here you different child have different score domain, something like below probably more appropriate:

p_parent = pm.Dirichlet(...)
parent = pm.Multinomial(...)
if parent == 'A':
    p_a = pm.Dirichlet(...)
    child_a = pm.Multinomial(...)
elif ...
5

Sorry for my slow pickup. I don’t see that the child actually physically conditions on the parent. I was expecting that the child multinomial would take both its own prior and the conditioning parents via pm.Multinomial. But I don’t see how I can pass the conditioning parents into the API. Does the condition statement (if parent == ‘A’) actually create graph dependency inside PyMC3 engine? Thank you

6

Yes. Since the parent node are observed, you can do it much more explicit.

7

Many thank for your explanation. I wish there was/is a nitty-gritty book for a comprehensive coverage on PyMC3(:slight_smile: But then it is going to cease anyway.

1 Like