Why does this Bayesian regression analysis fail?

I think this is a scale problem. Your Date variable takes values that are quite large, in the thousands. So if you think about drawing a line back from your data points (around 30) to the x-intercept, it will take a while to get there. What I’m trying to say is that the intercept is likely to be a large negative number. Computing the OLS coefficients can be instructive:

X = np.c_[np.ones(7), df.Date.values]
np.linalg.solve(X.T @ X, X.T @ df.Height)

>>>Out: np.array([-302.9709337     0.28391778])

So the intercept is -300, which according to your prior has a probability very close to zero (0.0008 i believe). Since you’re ruling out intercepts that would allow slopes consistent with your data, the model compromises by abandoning the slope and increasing the uncertainty around a flat intercept.

You can solve this by one of two ways. First, you could crank up the sigma on the intercept prior so that it allows values like -300, or even switch it to something fat-tailed, like a Cauchy. This works fine, but a more general solution (and the recommended one by pretty much everyone) is to scale your inputs. Instead of fitting height ~ a + b * date, fit height ~ a + b * z_date, where z_date = (df.Date - df.Date.mean()) / df.Date.std() . By normalizing your inputs to have zero mean and unit variance, it makes it much easier to reason about priors. It also helps the sampler cruise along in many cases. See this discussion, for example.

1 Like