OverflowError: Results too large

Questions v5
1

Hi

I’m pretty new to Bayesian modelling. After successfully fitting a simple linear regression model to real world data, I wanted to fit a model which should represent the data better in the form of Y=a*X**b, based on domain knowledge. Unfortunately sampling does not work with this model. After searching the forum, i did not find a question on this. If I missed it, I’m sorry. A hint on where to look (forum, textbook, articles) woul be greatly appreciated.

I’m using PyMC version 5.16.2

You can find a minimal example below which recreates the error. If further information would be required, I will gladly provide what I can.

x = [3, 9, 25]
Obs = [0.11, 0.26, 0.56]

with pm.Model() as model_1:
    # Prior
    a_mu = -2.94
    a_sigma = 0.246
    a = pm.Lognormal("a", mu=a_mu, sigma=a_sigma)

    b_sigma = 0.03
    b_mu = -0.29
    b = pm.Lognormal("b", mu=b_mu, sigma=b_sigma)
    
    epsilon = pm.HalfNormal("epsilon", 0.03)
    
    # Expected value
    mean = a * x**b
    
    # Likelyhood
    Y = pm.Normal("Y", mu=mean, sigma=epsilon, observed=Obs)


with model_1:
    trace = pm.sample(draws=2000, tune=1000, return_inferencedata=True)

I get the error following error message:

---------------------------------------------------------------------------
RemoteTraceback                           Traceback (most recent call last)
RemoteTraceback: 
"""
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pytensor\link\vm.py", line 405, in __call__
    thunk()
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pytensor\graph\op.py", line 524, in rval
    r = p(n, [x[0] for x in i], o)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pytensor\tensor\elemwise.py", line 754, in perform
    variables = ufunc(*ufunc_args, **ufunc_kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pytensor\scalar\basic.py", line 4076, in impl
    self.perform(None, inputs, output_storage)
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pytensor\scalar\basic.py", line 4314, in perform
    outputs = self.py_perform_fn(*inputs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\kml2\AppData\Local\Temp\tmpbtudztbn", line 3, in fgraph_to_python
    scalar_variable_2 = _perform(scalar_variable, scalar_variable_1)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pytensor\scalar\basic.py", line 4066, in _perform
    op.perform(node, inputs, outputs)
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pytensor\scalar\basic.py", line 1145, in perform
    output_storage[0][0] = self.impl(*inputs)
                           ^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pytensor\scalar\basic.py", line 2268, in impl
    return x**y
           ~^^~
OverflowError: (34, 'Result too large')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pymc\sampling\parallel.py", line 128, in run
    self._start_loop()
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pymc\sampling\parallel.py", line 180, in _start_loop
    point, stats = self._step_method.step(self._point)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pymc\step_methods\arraystep.py", line 173, in step
    return super().step(point)
           ^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pymc\step_methods\arraystep.py", line 101, in step
    apoint, stats = self.astep(q)
                    ^^^^^^^^^^^^^
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pymc\step_methods\hmc\base_hmc.py", line 198, in astep
    hmc_step = self._hamiltonian_step(start, p0.data, step_size)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pymc\step_methods\hmc\nuts.py", line 197, in _hamiltonian_step
    divergence_info, turning = tree.extend(direction)
                               ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pymc\step_methods\hmc\nuts.py", line 281, in extend
    tree, diverging, turning = self._build_subtree(
                               ^^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pymc\step_methods\hmc\nuts.py", line 371, in _build_subtree
    return self._single_step(left, epsilon)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pymc\step_methods\hmc\nuts.py", line 330, in _single_step
    right = self.integrator.step(epsilon, left)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pymc\step_methods\hmc\integration.py", line 82, in step
    return self._step(epsilon, state)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pymc\step_methods\hmc\integration.py", line 118, in _step
    logp = self._logp_dlogp_func(q_new, grad_out=q_new_grad)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pymc\model\core.py", line 361, in __call__
    cost, *grads = self._pytensor_function(*grad_vars)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pytensor\compile\function\types.py", line 959, in __call__
    self.vm()
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pytensor\link\vm.py", line 409, in __call__
    raise_with_op(self.fgraph, node, thunk)
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pytensor\link\utils.py", line 524, in raise_with_op
    raise exc_value.with_traceback(exc_trace)
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pytensor\link\vm.py", line 405, in __call__
    thunk()
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pytensor\graph\op.py", line 524, in rval
    r = p(n, [x[0] for x in i], o)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pytensor\tensor\elemwise.py", line 754, in perform
    variables = ufunc(*ufunc_args, **ufunc_kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pytensor\scalar\basic.py", line 4076, in impl
    self.perform(None, inputs, output_storage)
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pytensor\scalar\basic.py", line 4314, in perform
    outputs = self.py_perform_fn(*inputs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\kml2\AppData\Local\Temp\tmpbtudztbn", line 3, in fgraph_to_python
    scalar_variable_2 = _perform(scalar_variable, scalar_variable_1)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pytensor\scalar\basic.py", line 4066, in _perform
    op.perform(node, inputs, outputs)
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pytensor\scalar\basic.py", line 1145, in perform
    output_storage[0][0] = self.impl(*inputs)
                           ^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pytensor\scalar\basic.py", line 2268, in impl
    return x**y
           ~^^~
OverflowError: (34, 'Result too large')
Apply node that caused the error: Composite{...}([ 3  9 25], ExpandDims{axis=0}.0, ExpandDims{axis=0}.0, ExpandDims{axis=0}.0, T1{[0.11 0.26 0.56]}, [1.0986122 ... .21887582], Log.0, Gt.0)
Toposort index: 7
Inputs types: [TensorType(int32, shape=(3,)), TensorType(float64, shape=(1,)), TensorType(float64, shape=(1,)), TensorType(float64, shape=(1,)), TensorType(float64, shape=(3,)), TensorType(float64, shape=(3,)), TensorType(float64, shape=(1,)), TensorType(bool, shape=(1,))]
Inputs shapes: [(3,), (1,), (1,), (1,), (3,), (3,), (1,), (1,)]
Inputs strides: [(4,), (8,), (8,), (8,), (8,), (8,), (8,), (1,)]
Inputs values: [array([ 3,  9, 25]), array([6393289.44907887]), array([0.31178641]), array([0.08468763]), array([0.11, 0.26, 0.56]), array([1.09861229, 2.19722458, 3.21887582]), array([-2.46878572]), array([ True])]
Outputs clients: [[Sum{axes=None}(Composite{...}.0)], [Sum{axes=None}(Composite{...}.1)], [Sum{axes=None}(Composite{...}.2)], [Sum{axes=None}(sigma > 0)]]

HINT: Re-running with most PyTensor optimizations disabled could provide a back-trace showing when this node was created. This can be done by setting the PyTensor flag 'optimizer=fast_compile'. If that does not work, PyTensor optimizations can be disabled with 'optimizer=None'.
HINT: Use the PyTensor flag `exception_verbosity=high` for a debug print-out and storage map footprint of this Apply node.
"""

The above exception was the direct cause of the following exception:

OverflowError                             Traceback (most recent call last)
OverflowError: (34, 'Result too large')
Apply node that caused the error: Composite{...}([ 3  9 25], ExpandDims{axis=0}.0, ExpandDims{axis=0}.0, ExpandDims{axis=0}.0, T1{[0.11 0.26 0.56]}, [1.0986122 ... .21887582], Log.0, Gt.0)
Toposort index: 7
Inputs types: [TensorType(int32, shape=(3,)), TensorType(float64, shape=(1,)), TensorType(float64, shape=(1,)), TensorType(float64, shape=(1,)), TensorType(float64, shape=(3,)), TensorType(float64, shape=(3,)), TensorType(float64, shape=(1,)), TensorType(bool, shape=(1,))]
Inputs shapes: [(3,), (1,), (1,), (1,), (3,), (3,), (1,), (1,)]
Inputs strides: [(4,), (8,), (8,), (8,), (8,), (8,), (8,), (1,)]
Inputs values: [array([ 3,  9, 25]), array([6393289.44907887]), array([0.31178641]), array([0.08468763]), array([0.11, 0.26, 0.56]), array([1.09861229, 2.19722458, 3.21887582]), array([-2.46878572]), array([ True])]
Outputs clients: [[Sum{axes=None}(Composite{...}.0)], [Sum{axes=None}(Composite{...}.1)], [Sum{axes=None}(Composite{...}.2)], [Sum{axes=None}(sigma > 0)]]

HINT: Re-running with most PyTensor optimizations disabled could provide a back-trace showing when this node was created. This can be done by setting the PyTensor flag 'optimizer=fast_compile'. If that does not work, PyTensor optimizations can be disabled with 'optimizer=None'.
HINT: Use the PyTensor flag `exception_verbosity=high` for a debug print-out and storage map footprint of this Apply node.

The above exception was the direct cause of the following exception:

ParallelSamplingError                     Traceback (most recent call last)
Cell In[8], line 2
      1 with model_1:
----> 2     trace = pm.sample(draws=2000, tune=1000, random_seed=rng, return_inferencedata=True)

File C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pymc\sampling\mcmc.py:846, in sample(draws, tune, chains, cores, random_seed, progressbar, progressbar_theme, step, var_names, nuts_sampler, initvals, init, jitter_max_retries, n_init, trace, discard_tuned_samples, compute_convergence_checks, keep_warning_stat, return_inferencedata, idata_kwargs, nuts_sampler_kwargs, callback, mp_ctx, blas_cores, model, **kwargs)
    844 _print_step_hierarchy(step)
    845 try:
--> 846     _mp_sample(**sample_args, **parallel_args)
    847 except pickle.PickleError:
    848     _log.warning("Could not pickle model, sampling singlethreaded.")

File C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pymc\sampling\mcmc.py:1259, in _mp_sample(draws, tune, step, chains, cores, random_seed, start, progressbar, progressbar_theme, traces, model, callback, blas_cores, mp_ctx, **kwargs)
   1257 try:
   1258     with sampler:
-> 1259         for draw in sampler:
   1260             strace = traces[draw.chain]
   1261             strace.record(draw.point, draw.stats)

File C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pymc\sampling\parallel.py:471, in ParallelSampler.__iter__(self)
    464 task = progress.add_task(
    465     self._desc.format(self),
    466     completed=self._completed_draws,
    467     total=self._total_draws,
    468 )
    470 while self._active:
--> 471     draw = ProcessAdapter.recv_draw(self._active)
    472     proc, is_last, draw, tuning, stats = draw
    473     self._completed_draws += 1

File C:\ProgramData\Anaconda3\envs\pymc_env\Lib\site-packages\pymc\sampling\parallel.py:338, in ProcessAdapter.recv_draw(processes, timeout)
    336     else:
    337         error = RuntimeError(f"Chain {proc.chain} failed.")
--> 338     raise error from old_error
    339 elif msg[0] == "writing_done":
    340     proc._readable = True

ParallelSamplingError: Chain 1 failed with: (34, 'Result too large')
Apply node that caused the error: Composite{...}([ 3  9 25], ExpandDims{axis=0}.0, ExpandDims{axis=0}.0, ExpandDims{axis=0}.0, T1{[0.11 0.26 0.56]}, [1.0986122 ... .21887582], Log.0, Gt.0)
Toposort index: 7
Inputs types: [TensorType(int32, shape=(3,)), TensorType(float64, shape=(1,)), TensorType(float64, shape=(1,)), TensorType(float64, shape=(1,)), TensorType(float64, shape=(3,)), TensorType(float64, shape=(3,)), TensorType(float64, shape=(1,)), TensorType(bool, shape=(1,))]
Inputs shapes: [(3,), (1,), (1,), (1,), (3,), (3,), (1,), (1,)]
Inputs strides: [(4,), (8,), (8,), (8,), (8,), (8,), (8,), (1,)]
Inputs values: [array([ 3,  9, 25]), array([6393289.44907887]), array([0.31178641]), array([0.08468763]), array([0.11, 0.26, 0.56]), array([1.09861229, 2.19722458, 3.21887582]), array([-2.46878572]), array([ True])]
Outputs clients: [[Sum{axes=None}(Composite{...}.0)], [Sum{axes=None}(Composite{...}.1)], [Sum{axes=None}(Composite{...}.2)], [Sum{axes=None}(sigma > 0)]]

HINT: Re-running with most PyTensor optimizations disabled could provide a back-trace showing when this node was created. This can be done by setting the PyTensor flag 'optimizer=fast_compile'. If that does not work, PyTensor optimizations can be disabled with 'optimizer=None'.
HINT: Use the PyTensor flag `exception_verbosity=high` for a debug print-out and storage map footprint of this Apply node.
2

Welcome!

Your model works just fine for me:

In [2]: import arviz as az
   ...: az.summary(idata)
Out[2]: 
          mean     sd  hdi_3%  hdi_97%  mcse_mean  mcse_sd  ess_bulk  ess_tail  r_hat
a        0.050  0.003   0.045    0.055      0.000    0.000     446.0     804.0   1.01
b        0.752  0.016   0.720    0.780      0.001    0.001     403.0     851.0   1.01
epsilon  0.010  0.009   0.001    0.026      0.000    0.000      85.0      57.0   1.04

Does your model work if you request a single MCMC chain (i.e., idata=pm.sample(chains=1))? If so, you will need to test for if __name__ == "__main__" conditional to avoid multiprocessing issues on Windows.

3

“Result too large”, now there’s an error I never saw before.

Anyway it suggests you may need more constrained priors for it to work reliably

4

Many thanks for both replies. As the model worked for @cluhmann, I remembered that I have a missing package. After some diging, it turned out to be m2w64-toolchain. This solved the issue for me.

@ricardoV94 Right, I will have to further restrain my priors. But unfotunately I currently do not have further domain knowledge to do so. Maybe this will be resolve itselve if I have the actual data with more dispersion?

1 Like
5

You must have prior knowledge that forbids crazy large values that overflow in float64. It may, however, be hard to write down those prior constraints

6

Thank you for your answer.

This would mean to change the distribution type then I guess. Because with the given priors, crazy large values are very (very, very) unlikey but not impossible. I could use a distribution for a with support between 0 and 1 instead of lognormal. I’m still not sure what to do with b as such a constraint seems only reasonable but I can not completely eliminate values above 1 entirely.

When I check the prior predictive, they are what I can resonably constrain without knowing the data. I intentionally did these with 5000 draws and the highest values are no larger than 1.5

I will have to study sampling (and probably what tuning means) better, as I would never have expected values large enough to exceed a float64 with the given priors.

7

Ah okay if prior predictive behaves nicely maybe it was just NUTS that triggered it during tuning and it isn’t necessarily a problem with the model.

Without the C backend it raised an error when it should probably just have overflowed to +inf