Condition Categorical Variable on Bernoulli Parents

#1

I recently asked a question about how to condition a normally distributed variable on bernoulli parents. After looking at the code, and after some discussion, I understood what was going on.

In my setting, the dependent variable either takes the form of a particular distribution given the parents, or it should take the NA value (or its equivalent). So, in this case, suppose that I have two Bernoulli variables, On and Triangle, and I have one Categorical variable, Name, that depends on both of these. If On is true, then Name follows a categorical distribution determined by On. If Triangle is true, Name takes another categorical distribution determined by Triangle. In the case where both On and Triangle are False, Name should essentially remain undefined.

The following is mock code:

tri_names_given_on = numpy.array([.1, .5, .2, .2])
tri_names_given_triangle = numpy.array([.1, .1, .1, .1, .1, .3, .3])

block_names_given_on = numpy.array([.3, .4, .3])
block_names_given_block = numpy.array([.1, .2, .3, .4])

NA_ENCODING = -10.

with pymc3.Model() as model:

   on = pymc3.Bernoulli('on', pOn)
   triangle = pymc3.Bernoulli('triangle', pTri_given_not_on + on * tri_delta_on)
   block = pymc3.Bernoulli('block', pBlock_given_not_on + on* block_delta_on)
   arg1 = NA_ENCODING + on * (-NA_ENCODING + tri_names_given_on) + (1 - on) * triangle * (-NA_ENCODING + tri_names_given_triangle)
   arg2 = NA_ENCODING + on * (-NA_ENCODING + block_names_given_on) + (1 - on) * block * (-NA_ENCODING + block_names_given_block)

   triangle_name = None
   block_name = None

   if arg1 != NA_ENCODING:
      triangle_name = pymc3.Categorical('triangle_name', arg1)
   else:
      triangle_name = pymc3.Deterministic('triangle_name', NA_ENCODING)

   if arg2 != NA_ENCODING:
      block_name = pymc3.Categorical('block_name', arg2)
   else:
      block_name = pymc3.Deterministic('block_name', NA_ENCODING)

I’ve been having difficulty specifying my model in PyMC, so I think I’m missing some fundamental knowledge. I’ve looked at some of the tutorials, but right now, they don’t seem comprehensive. I seem to have a lot of gaps in my ability. So, any help is greatly appreciated.

0 Likes

#2

You should have a look at marginalized mixture model - whenever you have discrete variables in your model, you should first try to find way to marginalized it :slight_smile:

Frequently Asked Questions is a good place to start.

0 Likes

#3

Ok, I will look into this! Thank you

1 Like

#4

@junpenglao, I came up with this. Please let me know what you think.

tri_name_giv_tri_on_dist = numpy.array([.4, .4, .2])
tri_name_giv_tri_not_on_dist = numpy.array([.2,.3, .5])

block_name_giv_block_on_dist = numpy.array([.3, .3, .4])
block_name_giv_block_not_on_dist = numpy.array([.1, .3, .6])

NA_ENCODING = -10.

with pymc3.Model() as model:

   # Internal NA random variable
   NA = pymc3.Deterministic('NA', NA_ENCODING)
   
   on = pymc3.Bernoulli('on', pOn)
   triangle = pymc3.Bernoulli('triangle', pTri_given_not_on + on * tri_delta_on)
   block = pymc3.Bernoulli('block', pBlock_given_not_on + on* block_delta_on)

   triangle_mixture_weights = numpy.array([on * triangle, (1 - on) * triangle, (1 - on) * (1 - triangle)])   
   tri_name_given_tri_and_on = pymc3.Categorical.dist('tri_name_given_tri_and_on', tri_name_giv_tri_on_dist)
   tri_name_given_tri_and_not_on = pymc3.Categorical.dist('tri_name_given_tri_and_not_on', tri_name_giv_tri_not_on_dist)
   triangle_name = pymc3.mixture('triangle_name' w=triangle_mixture_weights, comp_dists=[tri_name_given_tri_and_on, tri_name_given_tri_and_not_on, NA])
  
   block_mixture_weights = numpy.array([on * block, (1 - on) * block, (1 - on) * (1 - block)])   
   block_name_given_block_and_on = pymc3.Categorical.dist('block_name_given_block_and_on', block_name_giv_block_on_dist)
   block_name_given_block_and_not_on = pymc3.Categorical.dist('block_name_given_block_and_not_on', block_name_giv_block_not_on_dist)
   block_name = pymc3.mixture('block_name' w=block_mixture_weights, comp_dists=[block_name_given_block_and_on, block_name_given_block_and_not_on, NA])
0 Likes

#5

You are getting there! You should rewrite the following into a continuous variable, either use the parameters (i.e., pOn, pTri_given_not_on + on * tri_delta_on here) directly, or wrap it into a Beta distribution:

   on = pymc3.Bernoulli('on', pOn)
   triangle = pymc3.Bernoulli('triangle', pTri_given_not_on + on * tri_delta_on)
   block = pymc3.Bernoulli('block', pBlock_given_not_on + on* block_delta_on)

Into:

   on = pOn
   triangle = pTri_given_not_on + on * tri_delta_on
   block = pBlock_given_not_on + on* block_delta_on
0 Likes