Hello, everyone,
I am pretty new to data analysis in python and the code is still “work in progress” which is why I apologize for the mess in advance.
With the following code I try to estimate the partsworth of the participants in a conjoint task.
Most of the code is from Cam Davidson-Pilon (which is free to use). The idea is to implement new things after it runs through smoothly the first time.
This is the model:
with pm.Model() as HB_v3:
alpha = pm.Normal('alpha', 0,
sd=alpha_prior_std,
shape=n_attributes_in_model,
testval=np.random.randn(n_attributes_in_model))
partsworth = pm.MvNormal("partsworth", alpha,
tau=np.eye(n_attributes_in_model), shape=(n_participants, n_attributes_in_model))
observations = []
for i, (_, selection) in enumerate(selections.iteritems()):
to_be_stacked = []
tempPartworth = partsworth[i, :]
nan_mask = pd.notnull(selection)
for _, choice in choices.get_group(CJ_Version[i]).iloc[:,:-1].groupby(axis=1, level=0):
dotProduct = tt.dot(choice[nan_mask.values].values, tempPartworth)
to_be_stacked.append(dotProduct)
stack = tt.stack(to_be_stacked, axis = 0).T
softmax = tt.nnet.softmax(stack)
temp = pm.Categorical("Obs_%s" % selection.name,
softmax,
observed=selection[nan_mask.values].values)
observations.append(temp)
The problem is that the code doesn’t run for all participants. NUTS gives me the error message “bad initialize energy -inf” on startup.
I already tried the tips from this post:
https://discourse.pymc.io/t/get-nan-or-inf-from-model-logp-model-test-point-is-an-attestation-of-incorrectly-configured-model/111/11?u=junpenglao
and also the notes from the “diagnosis page”:
https://docs.pymc.io/api/diagnostics.html
and found with the following code out that some of the observed variables (but not all) get a logp of “-inf”:
for RV in HB_v3.basic_RVs:
tp = HB_v3.test_point
print("----------------------------------")
print("----------------------------------")
print("** ",RV.name )
print(RV.logp(tp))
print()
print(RV.tag.test_value)
print()
print("lenght: ", len(RV.tag.test_value))
(I am going to trim the output to the relevant observed variables)
Partsworth (marked in blue):
I then played with the “testval” parameter, which only led to all participants becoming “-inf” at some point.
Next I tried to switch off the “jitter” or sample with Metropolis. Both without success (Metropolis runs through for all participants, but afterwards all partsworth get the same value for all participants over all tasks).
I do not have a clue, why NUTS can sample some participants and others not.
Does anybody have an idea?
Thanks in advance!
Daniel
)
