In working with reliability, there is a method of assessing your probability of success using the beta distribution (with parameters based on the passes and the fails of your testing). This will be familiar to many of you, because it serves as a simple example of the Jeffreys or Uniform priors in Bayesian analysis.

There is a theory that the effectiveness of fixes introduced could be used to project the reliability of your updated designs (with parameters based on the passes and the fails-fixes of your testing/redesign). This leads to non-integer values in the observed data.

I need to present a model to my boss soon and would appreciate help sanity checking my approach.

I have built my model as I would for a binomial analysis. But I do not use pm.Binomial for my observed line; rather I use:

pm.Potential(‘logp_fun’, pm.logp(pm.Beta.dist(1+ pass,1+unfixed), theta))

Where,

pass == float: quantity of test successes

unfixed == float: quantity of test failures minus fixes

theta == RV: my probability of success math model with all of its parameters and hyper-parameters

This seems to sample properly and I do not know of a better method.

P.S.-

I believe pm.Binomial would either require me to input integers or convert my observations to integers.

I believe pm.Beta(….,observed=pass/(pass+unfixed)) would lose the information about the highly discrepant quantity of tests per items.