Hierarchical AB Testing Posterior Delta

Hi Zappageck, you should change the np.log and np.power to T.log and T.power respectively. This should speed up the sampling.

I think your delta definition is wrong. I presume you want the differences between the probabilities of each trial? In that case you want to take differences of the rates variable.

the above posts suggests using a different parameterisation for the ab prior. On my machine this example ran about 10x quicker than your one.

However your data each suggest a probability around 1% after more than 7000 trials. No matter what model you use the output rates can’t move too far from the frequentist approach.

2 Likes