I have recently been working on an A/B testing problem and have analyzed it the “Frequentist” way.
However, I would like to do it the Bayesian way now and compare. I am not sure if we can can do these kind of questions here so pardon my ignorance. Here’s what I have :
A total population size of 380000, with a treatment size = 350000, control size = 30000.
The prior conversion rates for both groups were almost similar before applying the treatment to the treatment group --> 0.85 for the control group, 0.86 for the treatment group.
After applying treatment to the treatment group, this is the situation :
Control : Converted = 291, Not-converted = 29709
Target : Converted = 2987, Not-converted = 347013
Can somebody help me with how to go about it and do a bayesian analysis of it to see if the treatment group turned out to be better than the control?