I’m working on a partially pooled AB testing model in the same way that was described in this forum.
Albeit not understanding it fully the problem with the hyperpriors from the post is solved using this source:
import theano.tensor as T delta= with pm.Model() as bb_model: def ab_likelihood(value): a = value b = value return T.switch(T.or_(T.le(a, 0), T.le(b, 0)), -np.inf, np.log(np.power((a + b), -2.5))) ab = pm.DensityDist('ab', ab_likelihood, shape=2, testval=[1.0, 1.0]) a = ab b = ab rates = pm.Beta('rates', a, b, shape=7) trials = np.array([7543,7214,7375,7539,7356,7426,7336]) successes = np.array([67,83,70,81,58,75,77]) obs = pm.Binomial('observed_values', n=trials, p=rates, observed=successes) delta = pm.Deterministic('deltaAB',obs - obs) with bb_model: bb_trace = pm.sample(10000, tune=5000,random_seed=777)
However the diagnostic for my model do not look as expected and it also takes around 40 mins. There seems to be a problem with the calculation of the “delta” between the distributions.
The r_hats of the deltas are all NAN but also the means of the original posteriors (“rates”) have all become almost similar.
I’m not sure how to interpret this. But I must have made an obvious mistake in the setup as the data should lead to some meaningful results unlike this.
The same happens when I do this
deltaAB = pm.Deterministic(‘deltaAB’,obs - obs)
deltaAC = pm.Deterministic(‘deltaAC’,obs - obs)
deltaAD = pm.Deterministic(‘deltaAD’,obs - obs)
delta = pm.Deterministic(‘deltaAB’,obs - obs)
It is important to me to be able making calculations like a delta (B-A) or B/A-1. Could you please point me to where the error originates?