Noob Question: I have recently gotten into Bayesian AB testing, but I am lost on how to specify duration of test and number of observation we are looking to observe. Compared to frequentist approaches that have a clear formula for this, how do we a approach this as bayesians?

Maybe a precision-based power analysis could help? You can check Kruschke’s Doing Bayesian Data Analysis chapter 13. The idea is to iteratively run the model with simulated data until reaching a desired precision (pre-specified HDI width) for one or more parameters in the model. This could provide …

Bayesian AB Test Duration and Sample Size

Questions version agnostic

DangoNoChain January 18, 2024, 11:52pm 4

This was perfect. I really like the concept of specifying a desired HDI width and then simulating how many samples you would need to achieve it.

A very practical approach. Thank you again!

1 Like

Topic		Replies	Views
Bayesian sample size estimation for given HPD Questions	10	4821	August 5, 2021
Noob question: Bayesian inference in AB test vs frequentist approach (N size) version agnostic doc , modeling	6	547	June 21, 2022
Strategies for speeding up power analyses Questions	8	953	April 26, 2021
A/B testing with a Bayesian approach Questions	1	495	March 18, 2020
How can I apply bayesian statistics and pymc3 on the following problem Questions	2	590	August 15, 2018

Bayesian AB Test Duration and Sample Size

Related topics