Interpreting Output of Binomial Model

I think so. I might suggest inspecting the actual distribution of predicted values rather than just plotting the means/HDI and making assumptions about what the underlying distribution actually looks like. That would give you a better sense of whether you are recovering the parameters used to generated the simulated data.

This scenario looks a bit like a non-hierarchical version of multilevel regression with poststratification (MRP). You are interested in population-level treatment effects and believe sex to moderate those treatment effects? And you suspect that you will have non-representative samples wrt to sex? Is that all accurate? If so, I might suggest looking into MRP (e.g., check out Austin’s blog post; it’s rather old and thus uses an older version of pymc, but should give you an idea about the approach). Your setup is much simpler and will thus required fewer levels in the hierarchy, but I would definitely recommend the hierarchical approach. Depending on your sample size and the sex imbalance you are expecting in your sample, a hierarchical model can extract more information (e.g., using data from males to help inform female parameters and vice versa) and may allow you to better generalize.

4 Likes