Thanks to @junpenglao, I was now able to use pymc3’s glm module to fit a logistic regression to my data (see original post quoted below) and obtain a loo-Score.
Now I’m facing another problem (?). I want to compare the loo score with those obtained with two other models. These other models are some sigmoids (psychometric functions) that model the success probability for the Binomially distributed data at different levels.
These are plots of the data (points) and posterior predictive samples (line shows mean, shaded bands are 95 % HDIs):
(Sorry for the small font; perhaps you need zoom)
Now the loo-Score (see plots) for the logistic regression via the GLM module is unreasonably low compared to the other models. I’m not sure why this happens.
Could this happen due to the fact the other two models use Binomial likelihoods of the aggregated counts and the GLM module’s logistic reg. is based on the individual binary responses?
And if so, if there a way to correct for this to make the models comparable?
I thought because ultimately those models are fitted on the same data, their LOOs would be comparable… But perhaps the structure of the data (whether it enters Binomial nodes in aggregated form or as individual Bernoulli samples) plays a role?
Any help would be highly appreciated!
Jan
Original post (just in case someone came here to look for how to use the GLM module to fit multiple slopes/intercepts):