Using Pandas within Pymc3 context?

Questions
26

Hi Chris,

Thanks! I got your idea and tried two ways:

  1. First used the method you suggested to create a array:

ls_ad_gamma = np.zeros((len(GroupId1), 1), dtype=object)
for index, group in enumerate(GroupId1):
    sub_group = xxdat_matrix[np.all(xxdat_matrix[:,[0,1,2,3]] == group, axis = 1)]
    sum_ad = tt.sum(sub_group[:,-1].tolist())
    ls_ad_gamma[index] = sum_ad**gamma
yydat_matrix = tt.set_subtensor(yydat_matrix[:,4], ls_ad_gamma)    
yydat_matrix = tt.set_subtensor(yydat_matrix[:,[0,1,2,3]], GroupId1)

Problem of this is that, the numpy array ls_ad_gamma cannot be added to the yydat_matrix tensor. It looks like this:


array([[Elemwise{pow,no_inplace}.0],
       [Elemwise{pow,no_inplace}.0],
       [Elemwise{pow,no_inplace}.0],
       ...,
       [Elemwise{pow,no_inplace}.0]], dtype=object)

and got this error:


('Cannot convert [[Elemwise{pow,no_inplace}.0]\n [Elemwise{pow,no_inplace}.0]\n [Elemwise{pow,no_inplace}.0]\n ...\n [Elemwise{pow,no_inplace}.0]\n [Elemwise{pow,no_inplace}.0]\n [Elemwise{pow,no_inplace}.0]] to TensorType', )

change ls_ad_gamma to ls_ad_gamma.tolist() still cannot resolve it. And also if we create a tensor type ls_ad_gamma, then we will do this recursion item assignment again.

  1. Then I tried to simplify my looping:

    yydat_matrix = tt.zeros((len(GroupId1), 5), dtype=T.config.floatX)
    
    for index, group in enumerate(GroupId1):
        sub_group = xxdat_matrix[np.all(xxdat_matrix[:,[0,1,2,3]] == group, axis = 1)]
        sum_ad = tt.sum(sub_group[:,-1].tolist())
        sum_ad_gamma = sum_ad**gamma
        yydat_matrix = tt.set_subtensor(yydat_matrix[index,4], sum_ad_gamma)

    yydat_matrix = tt.set_subtensor(yydat_matrix[:,[0,1,2,3]], GroupId1)

but it still got the recursion limit error. Any idea about this?

Thanks!
Fan