Strategies for speeding up power analyses

[KyleJCaron]

@sammosummo

The simplest way to perform multiple jobs in parallel in Python is via the threading built-in module, but that won’t help much for anything computational because all jobs run on the same core. The multiprocessing module works similarly to the threading module but uses more than one core. I assume that it is not recommended to use either of these approaches in conjunction with PyMC3 since parallelization is integrated already when sampling, but I’ve never actually tried it.

The problem is I have nsim=1000 different datasets and from my understanding threading implies fitting a model to the same dataset… although maybe I could add all 1000 datasets into 1 dataframe, index them, and make variable shapes the same length as nsim… I might try that and see if things speed up

In terms of Power Analyses for Bayesian approaches, from my very limited experience NHST is quick operationally and pretty clear cut for decision making and figuring out timelines in industry - I have yet to have someone show me a better approach apart from bandit algorithms (though I’d love to see one).

I’ve also found its difficult to find the “right” priors due to seasonality and co-interventions so using an NHST approach with a bayesian model feels like a slightly more conservative approach that matches my experience level.