ValueError: length not known: Elemwise

Questions
1

I have a model with a couple of predictor variables, stored in a dataframe. I’m doing some deterministic transformations of those variables, and for some reason, some transforms give me ValueError: length not known: Elemwise{<operator>,no_inplace} and some don’t.

Here’s a sample, where t0 and theta are RVs in the model and t is a predictor variable coming from the data:

with pm.Model() as mod:
    theta = pm.Lognormal("theta", 0.0, 3.0)
    t0 = pm.Lognormal("t0", 0.0, 3.0)
    
    timeexp = -(t - t0)/theta

This yields (after some long Traceback, in full below):

ValueError: length not known: Elemwise{sub,no_inplace} [id A] ''   
 |TensorConstant{[  -1.   -..75. -176.]} [id B]
 |InplaceDimShuffle{x} [id C] ''   
   |Elemwise{true_div,no_inplace} [id D] ''   
     |ViewOp [id E] 't0'   
     | |Elemwise{exp,no_inplace} [id F] ''   
     |   |t0_log__ [id G]
     |ViewOp [id H] 'theta'   
       |Elemwise{exp,no_inplace} [id I] ''   
         |theta_log__ [id J]

Weirdly, if in the model I specify (just as an experiment) timeexp = t * t0/theta, I get a similar error, but if I use timeexp = t0/theta * t, it initializes fine. I’m not sure what’s going on :confused:

I also tried defining it as timeexp = pm.Deterministic('timeexp', -(t.to_list()-t0)/theta) - this avoids the error but then my output contains a whole list of timeexp variables, timeexp 0, timeexp 1... timeexp <len(t)>, which is not at all what I expect or intend - timeexp is supposed to be a simple transformation of the predictor t.

What’s going on / how can I get it to read the predictor in correctly?

Full traceback:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-65-25746c5d3158> in <module>
      3     t0 = pm.Lognormal("t0", 0.0, 3.0)
      4 
----> 5     timeexp = -(t-t0)/theta
      6 

~\AppData\Roaming\Python\Python38\site-packages\pandas\core\ops\common.py in new_method(self, other)
     63         other = item_from_zerodim(other)
     64 
---> 65         return method(self, other)
     66 
     67     return new_method

~\AppData\Roaming\Python\Python38\site-packages\pandas\core\ops\__init__.py in wrapper(left, right)
    343         result = arithmetic_op(lvalues, rvalues, op)
    344 
--> 345         return left._construct_result(result, name=res_name)
    346 
    347     wrapper.__name__ = op_name

~\AppData\Roaming\Python\Python38\site-packages\pandas\core\series.py in _construct_result(self, result, name)
   2755         # We do not pass dtype to ensure that the Series constructor
   2756         #  does inference in the case where `result` has object-dtype.
-> 2757         out = self._constructor(result, index=self.index)
   2758         out = out.__finalize__(self)
   2759 

~\AppData\Roaming\Python\Python38\site-packages\pandas\core\series.py in __init__(self, data, index, dtype, name, copy, fastpath)
    299                 raise TypeError(f"'{type(data).__name__}' type is unordered")
    300             else:
--> 301                 data = com.maybe_iterable_to_list(data)
    302 
    303             if index is None:

~\AppData\Roaming\Python\Python38\site-packages\pandas\core\common.py in maybe_iterable_to_list(obj)
    277     """
    278     if isinstance(obj, abc.Iterable) and not isinstance(obj, abc.Sized):
--> 279         return list(obj)
    280     return obj
    281 

~\AppData\Roaming\Python\Python38\site-packages\theano\tensor\var.py in __iter__(self)
    638     def __iter__(self):
    639         try:
--> 640             for i in xrange(theano.tensor.basic.get_vector_length(self)):
    641                 yield self[i]
    642         except TypeError:

~\AppData\Roaming\Python\Python38\site-packages\theano\tensor\basic.py in get_vector_length(v)
   4826     else:
   4827         msg = str(v)
-> 4828     raise ValueError("length not known: %s" % msg)
   4829 
   4830 

ValueError: length not known: Elemwise{sub,no_inplace} [id A] ''   
 |TensorConstant{[  1.   2...175. 176.]} [id B]
 |InplaceDimShuffle{x} [id C] ''   
   |ViewOp [id D] 't0'   
     |Elemwise{exp,no_inplace} [id E] ''   
       |t0_log__ [id F]
2

What is the data type of t? It might fix your issue if you do t = np.asarray(t), as theano interactive nicely with numpy array (or any python object with __array__ property, really). Which is also explain why doing something like timeexp = t0/theta * t works, as python convert t in this case to similar type as the result of t0/theta.

1 Like
3

t is a pd.Series (single row of dataframe) containing float64 data. I didn’t realise it might evaluate differently based on the order; will try casting as array and see if that works!

1 Like
4

Thank you so much. I got the same error and worked for me. Is it advisable to use np.arrays, instead of pd.dataframe when we input x’s, y’s in the model? Thanks.