Noob Question: I have recently gotten into Bayesian AB testing, but I am lost on how to specify duration of test and number of observation we are looking to observe. Compared to frequentist approaches that have a clear formula for this, how do we a approach this as bayesians?
Maybe a precision-based power analysis could help? You can check Kruschke’s Doing Bayesian Data Analysis chapter 13.
The idea is to iteratively run the model with simulated data until reaching a desired precision (pre-specified HDI width) for one or more parameters in the model. This could provide an estimate of an ideal sample size.
Ill check this out! Thank you!
This was perfect. I really like the concept of specifying a desired HDI width and then simulating how many samples you would need to achieve it.
A very practical approach. Thank you again!