L1 regularization with positivity constraint in hierarchical model

Hello everyone!

I’m currently working on a hierarchical implementation of a linear regression model to predict electricity consumption values for buildings.

As of now, I only tried implementing Gaussian priors, but I would be interested in testing how L1 regularisation performs. I read in different places that it’s possible to implement L1 regularization in pymc3 by using Laplace priors in place of the Gaussian.

From a building physics point of view, it would make sense that coefficients for some of the variables (e.g. outdoor temperature) would be constrained to be non-negative, so the question is: is there is a prior that can allow me to implement L1 regularization, and at the same time forcing the coefficients to be non-negative? Would it be some kind of special case of the Laplace prior?

I didn’t include a code snippet of my model here since I think it would not add much to the question, but if needed I can add it in a following edit.

Thanks in advance for your help!

Hi @bgrillone I’m pretty new to probabilistic programming and to PyMC3, but in reference to:

I’m not sure but I think you can at least try to use pm.Bound:

bounded_laplacian = pm.Bound(pm.Laplace, lower = 0 , upper = 10)
coef_1 = bounded_laplacian('coef_1', mu = 1, b = 4)

Hope it helps!

2 Likes

It seems to work, thanks!