Choosing the right likelihood function

Questions
1

Hi,
I have a question regarding the choise of the likelihood distribution. Lets assume the simple case of a normal distributed prior used in inference with some dataset evidence with an uknown distribution. If we fabricate this data using a normal distribution, choosing the right likelihood distribution is trivial, but if the datapoints are to few to be certain. what does the choise of likelihood distribution matter in pymc3?
eksample:

 dataset = data of length x with unknown distribution
 prior = pm.Normal("prior", mu=5,sd=1)
 likelihood = pm.Normal("likelihood", mu= prior, observed = dataset)

Here the likelihood function has a normal distribution, but what if i were to choose a beta distribution here?

Thank you

2

You can start narrowing down the choice of reasonable likelihoods with any constraints. If you know the data is going to be purely positive, then you can rule out distributions that assign data to the entire real line. For discrete counts you can use the Poisson or negative binomial.

One line of reasoning for likelihood selection is that you should use the maximum entropy distribution that satisfies any constraints you have. For a random variable with finite mean and variance, the max entropy distribution is the normal distribution. Can you supply any constraints on the possible values of the observed data?

5 Likes
3

hi, thank you for getting back to me, I will try to revive this old post.

By maximizing my distribution entropy (in this case a normal distribution), the sampler will not be able to change the likelihood distribution?

Lets say i for example have a exponentially distributed variable, that I use a normal likelihood function for, my posterior predictive checks will be constrained to a normal distribution?

4

So I could be mistaken here, but it sounds like more understanding is needed on the differences between a proper and posterior and a likelihood.

Your posterior distribution can ‘change shape’ from your prior due to your data.

But there’s likelihood is something different. This is not something that changes, it is a statement about how your data are distributed. So choice of the likelihood is often very important. You need to consider whether your data are continuous or discrete and what the support is ie. Positive only, bounded etc

Let me know if I’ve misdiagnosed the issue here but hopefully that’s useful

1 Like