Connecting two Dirichlet distributions through a probability (correlation?) matrix

Problem definition

Hello.
I am using Dirichlet distribution as a distribution over distributions in a Bayesian network. It models how material types (metal, plastic, etc…) connect with density, elasticity and so on.

Now I have another node, which I would also like to model with Dirichlet distribution, the Object category (e.g. bottle, wineglass, spoon, etc.). I have a correlation probability matrix that says how each material correlates with each category (i.e. how probable is it that some object is a bottle, if I know its from plastic and vice versa). I am not sure if I am using the word “correlation” correctly.

I would like to connect those two Dirichlet nodes together through this correlation probability matrix, but I did not find any easy solution for this within the PyMC3 API. I currently create a dummy Multinomial distribution which can accept an observation. The observation is sampled from a multinomial distribution that corresponds to the category @ correlation_matrix, which is basically the category translated to material. This material dummy Multinomial is connected to the material Dirichlet distribution which changes all connected nodes with new observations. This is depicted in the second picture.

Visual representation of the problem

1)

This is how I would like to connect the network inside the PyMC3 model. The problem I am describing is depicted in red color. I need both to be Dirichlet distribution in order to connect them with NormalMixtures “Density” and “Volume”.

2)

On the picture below is how I solve the problem now, but it puts the left part out from the model. I do not want to have two PyMC3 models, the system is a Bayesian network that should be wholly connected in one model.

Code representation of the problem

I am looking for a solution that would look something like this:

T1  # transition matrix material_given_category
T2  # transition matrix category_given_material

with pm.Model() as model:
    material = pm.Dirichlet('mat', w=alphas1)
    category = pm.Dirichlet('cat', w=alphas2)
    pm.ConnectDirichlet(material, category, [T1, T2])  # this line is totally made up
    
    density = pm.NormalMixture('density', w=material, mu=mus, sigma=sigmas, observed=observation_density)
    volume = pm.NormalMixture('volume', w=category, mu=mus_2, sigma=sigmas_2)

Could you please direct me in the direction of solving this problem in a PyMC3 manner? I would like to make my code as general as possible. Thank you.

1 Like

Have you thought about using latent multivariate normals to specify this correlation across categories/properties?

Its straightforward to convert their output to the simplex domain (see here), while at the same time using the tools that already exist for modelling covariance structures.

Otherwise it seems like you are looking for more generalized versions of the Dirichlet, which have also been described in the literature. Not sure if there’s something taylor made for your situation, as your explanation of the problem is a bit vague (no offense!).

1 Like

Hello Ricardo,

thank you for your response. I will check out the link you provided.
I tried to keep the problem definition as concise as possible, not to overwhelm the reader with unnecessary details.

The described system is a Bayesian network. So via a measurement at any node, I need to recompute the probabilities inside the network accordingly, using PyMC3. I need every node to change accordingly to the measurement. Doing the whole Bayesian network inference in cases with a non-trivial number of nodes manually is quickly taking off in complexity and computation time.

Best regards

1 Like

Hello Ricardo,

I checked out the Logistic Normal Distribution. I understand that there is some additive logistic transformation that could be done on my probability vector, which should be represented by a multivariate normal?

Citing from wikipedia.org:

This follows from applying the additive logistic transformation to map a multivariate normal random variable y ∼ N ( μ , Σ ) , y ∈ R D − 1 to the simplex…

Although the connection between the two nodes (category and material) is not in fact a random variable. That connection is known and static. I know how materials relate to categories (if it’s made of glass, it’s probably a wineglass or sunglasses) and how category relates to material (if its a wineglass its probably made of: glass - 80% or out of plastic 20% e.g.). I just need to connect two probability vectors, one for category and one for material (currently represented by the Dirichlet distributions) together in such a way, that if on any vector an observation is done, the new updated information would flow through the connections to other nodes. The values in the vectors of the nodes would change, although the values of the connections would not.

Question 1)

Is the probability matrix T1 (`T2’ respectively) just a representation of the Σ in the multivariate normal random variable in the wikipedia entry cited above?

Question 2)

I could not find any direct implementation of such logits in PyMC3 documentation. Would I need to create custom distributions for that? Regarding transformations there is this scarce entry in the docs. Could perhaps the pymc3.DirichletMultinomial be the distribution I am looking for? (link here).

Thank you.