Logistic Regression with a very sparse matrix

[junpenglao]

I would start with reducing the scale of the prior, sigma=10. or even sigma=1..
You should also try a matrix factorization like QR decomposition: https://betanalpha.github.io/assets/case_studies/qr_regression.html