Interpreting Output of Binomial Model

[sealibra]

@cluhmann

I might suggest inspecting the actual distribution of predicted values rather than just plotting the means/HDI and making assumptions about what the underlying distribution actually looks like.

Good point. I was just show the forest plot as one example of how to work with the distributions in order to confirm I wasn’t misunderstanding something but I do intend to explore them further.

@cluhmann

This scenario looks a bit like a non-hierarchical version of multilevel regression with poststratification (MRP). You are interested in population-level treatment effects and believe sex to moderate those treatment effects? And you suspect that you will have non-representative samples wrt to sex? Is that all accurate? If so, I might suggest looking into MRP (e.g., check out Austin’s blog post ; it’s rather old and thus uses an older version of pymc, but should give you an idea about the approach). Your setup is much simpler and will thus required fewer levels in the hierarchy, but I would definitely recommend the hierarchical approach. Depending on your sample size and the sex imbalance you are expecting in your sample, a hierarchical model can extract more information (e.g., using data from males to help inform female parameters and vice versa) and may allow you to better generalize.

I should have mentioned this in the original post but this is exactly what I was going for. I actually used the post you mention to guide things a bit, however it was a bit difficult given the data no longer seems to be available and as you mentioned it uses an older version of pymc. I’d love to publish an updated version if things work out.

Eventually I intend to use a hierarchically model as well as introduce additional predictors but I wanted to make sure I was at least on the right track with a very simple version of the model. Thanks for the feedback!