Perhaps more of a stats question, basically coming down to “how do you implement penalties on function curvature from a Bayesian perspective?”
I was reading about Generalized Additive Models. They are basically defined as
where s(x) is a smooth function, generated with a weighted sum of basis functions b(x)
You could get pretty close to a Bayesian GAM pretty simply by setting up some basis functions and priors over the \beta weightings.
But one of the interesting properties about GAMs is that they have a penalty function applied to the curvature of the function, e.g. the second derivative.
So my question is, how would you think about this curvature penalty from a Bayesian perspective? It is easy to think about applying constraints via priors, but I don’t know if that would achieve the same effect of penalising the wiggliness?
My only idea would be to calculate the wiggliness in a PyMC model and use
pm.Potential to manually add a constant to the model logo based on the calculated wiggliness. Is that sane? Any other ideas?
EDIT: After more research, looks like you can achieve this by priors on parameters, Generalized additive model - Wikipedia. Would be interested to see if anyone’s implemented this.