To clarify my position, I wasn’t saying it doesn’t makes sense to do power analysis in a Bayesian context, just that I had never heard of it called that. I’ve only ever seen the term used in grant applications, etc. were the goal is to figure out how much data you need to achieve sufficient power or how sensitive your study will be given the budget … in my experience this has always been strictly frequentist. I would call what your describing a simulation study or a parameter-recovery experiment … which I guess is just splitting hairs.
I think @chartl is correct about figuring out the posterior value of p in closed form, then sampling from this distribution directly. I don’t think you need PyMC3 at all for this. For example, if you know p you can generate posterior samples via scipy.stats.binomial.rvs.