Trace plots Hierarchical Models

Questions
1

Hi All,
I am trying to explore a hierarchical model with 3 levels. I am able to create the model and generate samples from it. However, when I try to use pm.traceplot() or pm.summary() I get the following error

ValueError: Input dimension mis-match. (input[3].shape[0] = 1860, input[4].shape[0] = 900)

Any idea why this is happening and how to resolve this??

2

You may have inconsistent versions of PyMC3 and ArviZ. How did you install them?

Also can you try the functions native in ArviZ

import arviz as az
az.summary()
az.plot_trace()
3

I tried using these commands and it is still giving me the same error.
I am using PyMC3 3.8 and ArviZ 0.9.

4

Can you paste an example that shows the error?

Also you can try updating both ArviZ and PyMC from the github master branch

1 Like
5

You can also just update PyMC to 3.9.3 from pip or conda-forge

6

I tried updating the both ArviZ and PyMC but I am still getting the following error

ValueError: Input dimension mis-match. (input[3].shape[0] = 1860, input[4].shape[0] = 900)
Apply node that caused the error: Elemwise{Composite{Switch(i0, (i1 * ((-(i2 * sqr((i3 - ((i4 * i5) + i6 + (i7 * i8) + (i9 * i8) + i10 + (i11 * i12) + i13 + i14 + (i15 * i16) + (i17 * i18 * i8)))))) + i19)), i20)}}[(0, 4)](Elemwise{Composite{Cast{int8}(GT(i0, i1))}}.0, TensorConstant{(1,) of 0.5}, Elemwise{Composite{inv(sqr(i0))}}[(0, 0)].0, TensorConstant{[3.3082953…7540281 ]}, AdvancedSubtensor1.0, <TensorType(float64, vector)>, AdvancedSubtensor1.0, AdvancedSubtensor1.0, <TensorType(float64, vector)>, AdvancedSubtensor1.0, AdvancedSubtensor1.0, InplaceDimShuffle{x}.0, <TensorType(float64, vector)>, AdvancedSubtensor1.0, AdvancedSubtensor1.0, InplaceDimShuffle{x}.0, <TensorType(float64, vector)>, AdvancedSubtensor1.0, <TensorType(float64, vector)>, Elemwise{Composite{log((i0 * i1))}}.0, TensorConstant{(1,) of -inf})
Toposort index: 15
Inputs types: [TensorType(int8, (True,)), TensorType(float64, (True,)), TensorType(float64, (True,)), TensorType(float64, vector), TensorType(float64, vector), TensorType(float64, vector), TensorType(float64, vector), TensorType(float64, vector), TensorType(float64, vector), TensorType(float64, vector), TensorType(float64, vector), TensorType(float64, (True,)), TensorType(float64, vector), TensorType(float64, vector), TensorType(float64, vector), TensorType(float64, (True,)), TensorType(float64, vector), TensorType(float64, vector), TensorType(float64, vector), TensorType(float64, (True,)), TensorType(float32, (True,))]
Inputs shapes: [(1,), (1,), (1,), (1860,), (900,), (900,), (900,), (900,), (900,), (900,), (900,), (1,), (900,), (900,), (900,), (1,), (900,), (900,), (900,), (1,), (1,)]
Inputs strides: [(1,), (8,), (8,), (8,), (8,), (8,), (8,), (8,), (8,), (8,), (8,), (8,), (8,), (8,), (8,), (8,), (8,), (8,), (8,), (8,), (4,)]
Inputs values: [array([1], dtype=int8), array([0.5]), array([2.19361905]), ‘not shown’, ‘not shown’, ‘not shown’, ‘not shown’, ‘not shown’, ‘not shown’, ‘not shown’, ‘not shown’, array([0.10929279]), ‘not shown’, ‘not shown’, ‘not shown’, array([0.2955149]), ‘not shown’, ‘not shown’, ‘not shown’, array([-1.05232435]), array([-inf], dtype=float32)]
Outputs clients: [[‘output’]]

HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag ‘optimizer=fast_compile’. If that does not work, Theano optimizations can be disabled with ‘optimizer=None’.
HINT: Use the Theano flag ‘exception_verbosity=high’ for a debugprint and storage map footprint of this apply node.

7

We really need to see some code to be able to assist more. It looks to be a problem with the model and sampling rather than plotting and summary statistics.

1 Like
8

It is difficult to guess without seeing any model code not the complete error message, but here are some ideas:

  • issues retrieving the log likelihood. see DE-Metropolis with blackbox likelihood does not terminate
  • issues with densitydist and multiple observations: see Blackbox likelihood example doesn't work
  • version issues: if mixing pip and conda, it could be that even when a newer version is installed, python is actually importing the old version anyways. check pm.__version__ and az.__version__ within the same code the model is executed
  • issues with coordinates: it could be if using dims and coords that a the length of the array of coordinate values for a given dim does not match the size of the dimension. The error should not mention theano in this case but who knows :man_shrugging:
  • something else entirely: I don’t think we will be able to help any further than this without the code or a reduced version of it reproducing the same issue
1 Like