Apply multivariate normal log-likelihood find the posterior distribution

Questions
1

Dear everyone,
I have a question about using multivariate log-likelihood find the posterior.
I have 3 input parameter: each input has 20 value.

nodes1 = np.array([0.05675, 0.05934, 0.05633, 0.0557 , 0.05702, 0.06401, 0.06322, 0.06571, 0.06099, 0.05832, 0.06196, 0.06463, 0.05507, 0.06351, 0.06287, 0.06122, 0.05407, 0.05985, 0.05774,0.06015])
nodes2 = np.array([0.9486, 0.9095, 0.9856, 0.9318, 1.0477, 1.0489,1.0663, 0.9184, 0.9646, 1.0345, 1.0168, 1.0565, 0.9727, 0.9907, 0.9277, 0.9548, 1.0933, 1.0751,1.0026, 1.0231])
nodes3 = np.array([51.813, 54.279, 52.659, 51.197 , 46.629, 49.791, 48.581, 54.799, 46.413, 47.078, 52.367, 48.204, 50.389, 45.402, 47.893, 50.796 , 49.332, 53.323, 53.713, 45.757])

I perform the experiment for each group of 3 corresponding input values : example (node1_1,node2_1,node3_1) have a output value (0.0204232).
I repeated the experiment with 20 times. I have the 20 corresponding output values.
evals = np.array([0.0204232,0.0205054,0.0204971,0.0204463,0.0206686,0.0206678,0.0206883,0.0204627,0.020426,0.0206532,0.0206322,0.020677,0.0204431,0.0205319,0.0204508,0.0204115,0.020721,0.0206988,0.0206179,0.0206418])

Now, I want to use the multivariate normal log-likelihood to consider posterior distribution P(node1, node2,node3| evals). How to find the posterior distribution? If it is not enough condition to find it what do things need to find the posterior distribution?
please get me your recommendation.
thank you.

2

What is the function example here? Is it a linear function or something?

3

It was just got the output from a experiment. Can not consider linear or not.

4

So in a way in an experiment you observed (node1_1,node2_1,node3_1) and output value right?

5

I have the values group of parameters (node1_i,node2_i,node3_i) i is from 1 to 20. I have the corresponding output values.

6

Right, so IIUC in a way you are interested in the function that maps the values group of parameters (node1_i,node2_i,node3_i) to the ouput value ouput_i right? I would start with a linear regression, something like: y = pm.Normal('observed', tt.dot(np.asarray([node1, node2, node3]), beta), sd=sigma, observed=output), with beta and sigma as the free parameters.

7

maybe don’t have the solution for my problem. So now I assume that the node 1, node 2, node 3 have uniform distributions. they are the linear regression with output value. How to apply multivariate normal log-likelihood to find the posterior distribution?
detail :
nodes1: uniform distribution (0.054,0.066),
nodes2: uniform distribution (0.9,1.1)
nodes3: unifrom distribution (45,55)
I use the chaospy which is package to frist generation to have 20 input values.

nodes1 = np.array([0.05675, 0.05934, 0.05633, 0.0557 , 0.05702, 0.06401, 0.06322, 0.06571, 0.06099, 0.05832, 0.06196, 0.06463, 0.05507, 0.06351, 0.06287, 0.06122, 0.05407, 0.05985, 0.05774,0.06015])
nodes2 = np.array([0.9486, 0.9095, 0.9856, 0.9318, 1.0477, 1.0489,1.0663, 0.9184, 0.9646, 1.0345, 1.0168, 1.0565, 0.9727, 0.9907, 0.9277, 0.9548, 1.0933, 1.0751,1.0026, 1.0231])
nodes3 = np.array([51.813, 54.279, 52.659, 51.197 , 46.629, 49.791, 48.581, 54.799, 46.413, 47.078, 52.367, 48.204, 50.389, 45.402, 47.893, 50.796 , 49.332, 53.323, 53.713, 45.757])

from the input I have 20 output values:
evals=np.array([0.0204232,0.0205054,0.0204971,0.0204463,0.0206686,0.0206678,0.0206883,0.0204627,0.020426,0.0206532,0.0206322,0.020677,0.0204431,0.0205319,0.0204508,0.0204115,0.020721,0.0206988,0.0206179,0.0206418])

The main problem want to use the multivariate normal log-likelihood find the parameter posterior distribution (node1,node2,node3). (now assume that it is the linear regression)
please help me. Thanks you

8

I think you really need to be more specific of what is the mapping between your input (in this case node1, node2, node3) and your output (in this case evals), otherwise we are just talking in circle. Do you use chaospy that takes the 20*3 input values and gives you 20 output values? And you want to infer the posterior distribution of the 3 input parameters now given the 20 observed (output values)?

1 Like
9

I can use the chaospy to samples 10000 output values. but can not use chaospy that take 20*3 input.

10

Hi you, I have just estimate the function between (node1,node2,node3) and output.
this is the function:
output = -1.2422069376608758q0^2+0.010979502286122458q0q1+0.005065108695132867q1^2-0.0003998832861218647q0q2+0.16015398278991752q0-0.00013362145005939259q1q2+2.0112165363058473e-06q2^2-0.0020280971401678734q1-4.0276945348026916e-05q2+0.01657469140501759

with (node1 is q0, node2 is q1, node3 is q2)
how to find the posterior distribution of the each parameter (node1, node2,node3)?
thanks you

11

Is this something like what you need?

a = -1.2422069376608758
b = 0.010979502286122458
c = 0.005065108695132867
d = 0.0003998832861218647
e = 0.16015398278991752
f = 0.00013362145005939259
g = 2.0112165363058473e-06
h = 0.0020280971401678734
i = 4.0276945348026916e-05
j = 0.01657469140501759

with pm.Model():
    q0 = pm.Uniform('nodes 1', 0.054, 0.066)
    q1 = pm.Uniform('nodes2', 0.9, 1.1)
    q2 = pm.Uniform('nodes3', 45, 55)
    mu = a * q0**2 + b * q0 * q1+ c * q1**2- d * q0 * q2+ e * q0 - f * q1 * q2+ g * q2**2 - h * q1 - i * q2 + j
    pm.Normal('observed', mu, 0.1, observed=observed)
    trace = pm.sample()
    pm.traceplot(trace)