Is it a good idea to use the mean and standard deviation of coefficients from other models as my prior?

Questions
1

Hi, I have a dataset that I’ve been playing around with for school
I have gotten very good results with a bunch of methods (Ridge, Lasso, ElasticNet, SVM, Bagging, Stacking and NN even)

Now I’m having a range of different coefficients of my predictors, is it a good idea to use them as my priors (I did so, I think the result has been ok) or should I use noninformative priors instead.

Or if different use cases, when does it make sense to use one or the other?

Thanks in advance!

2

Most of the methods you describe won’t all have the same number or type of parameters. Are you perhaps asking about what happens if you have, say, coefficients from a ridge regression along with those from an ordinary least squares?

1 Like
3

hi ckrapu

No, actually all these methods can be used to give linear coefficients that can be used as a solution to ordinary least squares.

The coefficients computed by all of these, along with the closed form solution (normal equations) are what I used to compute mean and standard deviation for each parameter’s coefficients.

So in detail. I used the LinearRegression, Ridge, Lasso, ElasticNet, LinearSVR models and the closed form solution to compute coefficients for a solution to my regression problem.
I combined some of these with bagging and stacking.
For NN I used keras and a single layer, linear activation.

I verified each of the coefficients by multiplying X@w (w is the weights of the respective models) to compute errors, they are all within .02 of one another for a dataset of about 700 samples.
There is a lot of redundancy in the data and thus the solutions (weight vector varies) quite a bit. So standard Deviation is quite large.

Best

oli