Binomial AB Testing

Welcome!

There are several things here that seem less than ideal. First, you should probably stick to calling pm.sample(), passing no arguments so that it executes using all the default values for all arguments (this is particularly true for the step argument). Because observations is always 30, you can don’t have to pass n=np.array(observations) into the binomial prior, you can just set n=30.

I’m not sure what you mean by “flattening” here. When I run the following model:

n_flips = 30
observations = [n_flips] * 20
occurrences = [2] * 10 + [28] * 10

with pm.Model() as model:
    p_t = pm.Uniform("test_distn", 0, 1, shape=(len(observations),))
    obs_test = pm.Binomial("obs_test", n=n_flips, p=p_t, observed=occurrences)
    idata = pm.sample()

I inspect the parameter you asked about:

idata.posterior["test_distn"].mean(dim=["chain", "draw"])

I get this:

<xarray.DataArray 'test_distn' (test_distn_dim_0: 20)> Size: 160B
array([0.09349925, 0.09319441, 0.09354546, 0.09385241, 0.09396087,
       0.09368252, 0.09379583, 0.09246933, 0.09423234, 0.09350552,
       0.9054838 , 0.90666993, 0.90597412, 0.90627215, 0.90680455,
       0.9063937 , 0.90617682, 0.90684242, 0.90594251, 0.90559609])
Coordinates:
  * test_distn_dim_0  (test_distn_dim_0) int64 160B 0 1 2 3 4 ... 15 16 17 18 19

which seems about right to me.

Happy to answer questions. I might also suggest checking out this notebook for some ideas about how to attack these sorts of models in a slightly more general manner.