Simple model question

Having looked at it a bit closer, I see now that your problem is under-identified. You are trying to estimate two parameters from only one data point, which means there is an infinite combination of solutions.

Since there are only two parameters, we can visualize the log likelihood function. I plotted it using a 100x100 evenly spaced grid for p and N, for 10, 100, 1000, and 10,000 observations. Note that the shape of the likelihood function is not a function of the number of observations, only the magnitude increases since it’s summing over more stuff. The shape is the same in all the pictures I just rotated it so we an get a better view. The true parameter values are at the red column. The little bump up when N is small is because I set all likelihoods where obs > N to 0; that doesn’t really exist.

You can see that the gradients of the function are essentially zero once you get up the hill that falls down towards N=50 and P=1. In particular, the true values are sitting squarely in the middle of the plateau. From the perspective of the sampler, all the points at the top of the plateau are going to be equally good.

The results @BjornHartmann got are due to how samplers explore a posterior space. Once a chain wandered somewhere onto the plateau, it will be very difficult to accept a move, so it just stays put (Metropolis specifically uses draws from a normal to randomly accept moves to parameters with equal or lower log probability, so curves you see in the trace are likely equal to this acceptance probability). If you actually evaluate the log probability of some data at p=0.24, n=19 and p=0.32 n=25, they are roughly equal, so the sampler is giving you two possible solutions (out of infinity).

I’m sure if you colored the chains in @maab 's trace plot (so we can tell which N goes to which p) and computed the log probs, you would find that all 10 are equivalent solutions.

Another way to see this degeneracy is to look at the pair plots from the trace:

image

As you can see, the parameters are just a deterministic function of each other.

1 Like