Theano error debugging model

Questions
1

Here’s a degenerate model that I plucked out of a model I’m trying to debug:

with pm.Model() as model:
    beta_soc_dem_edu_spend = pm.Normal('β(soc_dem, edu spend)', shape=1)
    print('β(soc_dem, edu spend) shape is: %s'%beta_soc_dem_edu_spend.shape)
    print('β(soc_dem, edu spend) shape is: %s'%beta_soc_dem_edu_spend.shape.eval())

When I run this, the first print prints out the theano variable:

  • β(soc_dem, edu spend) shape is: Shape.0
    But then the second line raises a MissingInputError:
MissingInputError: Input 0 of the graph (indices start from 0), used to compute Shape(β(soc_dem, edu spend)), was not provided and not given a value. Use the Theano flag exception_verbosity='high', for more information on this error.

Why does this happen?
By the way, setting that Theano flag seems to do nothing at all.

1 Like
2

In the second print fails because beta_soc_dem_edu_spend has no value and therefore no shape yet. It only obtains a value once you start sampling from the distribution.

In the first print you refer to beta_soc_dem_edu_spend.shape which is a symbolic theano expression that can be used, e.g., to define a computation that only produces a value once its .eval() is called. Here is an example:

with pm.Model() as model:
    beta = pm.Normal('β', shape=1)
    a = pm.Normal('a', mu=beta.shape, testval=0)
    trace = pm.sample()

Looking at np.mean(trace['a']) shows 0.9695488955681962 so the mean of pm.Normal('a', ... is indeed the shape of beta.

If you want to use the shape of beta during the model definition you could store it in a separate variable, e.g.:

beta_shape = 1
with pm.Model() as model:
    beta_soc_dem_edu_spend = pm.Normal('β(soc_dem, edu spend)', shape=beta_shape)
    print('β(soc_dem, edu spend) shape is: %s'%beta_shape)
3

In addition, you can make it work by supplying a value to the graph:

In [13]: with pm.Model() as model: 
    ...:     beta_soc_dem_edu_spend = pm.Normal('β(soc_dem, edu spend)', shape=1
    ...: ) 
    ...:     print('β(soc_dem, edu spend) shape is: %s'%beta_soc_dem_edu_spend.s
    ...: hape) 
    ...:     print('β(soc_dem, edu spend) shape is: %s'%beta_soc_dem_edu_spend.s
    ...: hape.eval({beta_soc_dem_edu_spend: np.zeros(1)}))
1 Like
4

@Dominik
Thanks. I’m still a little confused because I set the shape of the RV to a constant, so I don’t understand why it is that evaluation is needed: why isn’t this a theano constant?

Similarly, I didn’t get what input was missing to the Theano shape variable, so that I get the MissingInputError (and it seems hard to winkle that information out of Theano).

If I understand what you’re saying, the shape depends on the actual value of the variable, and that’s why there’s a missing input.

As you might have guessed, this arises when I’m trying to debug a shape error in sampling (even the prior predictive fails), and I’m looking for a general way to diagnose such errors. There is discussion in the Theano docs about how one debugs theano graphs, but it’s not obvious (at least not to me!) how one gets hold of this graph when one is interacting with Theano through PyMC3.

5

Thanks. passing the dictionary is an important clue. As I said above, I stumbled on this trying to debug a shape mismatch – specifically in a model where there is missing data.

I still don’t quite get how to go from PyMC3 to debugging the underlying computation graph. I’ll see what I can figure out.

6

Since all RVs of model have a test value, you could use that to get its shape:

model.test_point['β(soc_dem, edu spend)'].shape

Not sure if that is what you are looking for but I also do not know how to debug the theano compute graph efficiently.

Edit: As @junpenglao points out, you can get the shape even more conveniently with the python variable through beta.tag.test_value.shape.

7

Doing rv.tag.test_value is also my go-to for debugging shape issue.

1 Like