ValueError: Bad initial energy: inf. The model might be misspecified error replicating Multilevel Regression and Poststratification with PyMC3 notebook

I´m having a very uncommon and large error trying to run a notebook I found:
Multilevel Regression and Poststratification with PyMC3

The error raises on cell 34, during model fitting. It takes a huge amount of time to finish and when it does gives me de following error:

---------------------------------------------------------------------------
RemoteTraceback                           Traceback (most recent call last)
RemoteTraceback: 
"""
Traceback (most recent call last):
  File "C:\Users\El_Pentagono\Anaconda3\envs\py36\lib\site-packages\joblib\_parallel_backends.py", line 350, in __call__
    return self.func(*args, **kwargs)
  File "C:\Users\El_Pentagono\Anaconda3\envs\py36\lib\site-packages\joblib\parallel.py", line 131, in __call__
    return [func(*args, **kwargs) for func, args, kwargs in self.items]
  File "C:\Users\El_Pentagono\Anaconda3\envs\py36\lib\site-packages\joblib\parallel.py", line 131, in <listcomp>
    return [func(*args, **kwargs) for func, args, kwargs in self.items]
  File "C:\Users\El_Pentagono\Anaconda3\envs\py36\lib\site-packages\pymc3\sampling.py", line 332, in _sample
    for it, strace in enumerate(sampling):
  File "C:\Users\El_Pentagono\Anaconda3\envs\py36\lib\site-packages\pymc3\sampling.py", line 430, in _iter_sample
    point, states = step.step(point)
  File "C:\Users\El_Pentagono\Anaconda3\envs\py36\lib\site-packages\pymc3\step_methods\arraystep.py", line 175, in step
    apoint, stats = self.astep(array)
  File "C:\Users\El_Pentagono\Anaconda3\envs\py36\lib\site-packages\pymc3\step_methods\hmc\nuts.py", line 182, in astep
    'might be misspecified.' % start.energy)
ValueError: Bad initial energy: inf. The model might be misspecified.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\El_Pentagono\Anaconda3\envs\py36\lib\multiprocessing\pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "C:\Users\El_Pentagono\Anaconda3\envs\py36\lib\site-packages\joblib\_parallel_backends.py", line 359, in __call__
    raise TransportableException(text, e_type)
joblib.my_exceptions.TransportableException: TransportableException
___________________________________________________________________________
ValueError                                         Wed May  9 18:50:49 2018
PID: 25680Python 3.6.3: C:\Users\El_Pentagono\Anaconda3\envs\py36\python.exe
...........................................................................
C:\Users\El_Pentagono\Anaconda3\envs\py36\lib\site-packages\joblib\parallel.py in __call__(self=<joblib.parallel.BatchedCalls object>)
    126     def __init__(self, iterator_slice):
    127         self.items = list(iterator_slice)
    128         self._size = len(self.items)
    129 
    130     def __call__(self):
--> 131         return [func(*args, **kwargs) for func, args, kwargs in self.items]
        self.items = [(<function _sample>, (), {'chain': 1, 'draws': 1500, 'live_plot': False, 'live_plot_kwargs': None, 'model': <pymc3.model.Model object>, 'progressbar': False, 'random_seed': 4260026, 'start': {'kerry': array(0.032483), 'relig': array(-0.44432057), 'Δ_age': array([0.93412839, 0.37075697, 0.01745296, 0.78666723]), 'Δ_age_edu': array([ 0.59174212, -0.71198657,  0.62474506, -0...8, -0.82434136,  0.04314941,
       -0.61229957]), 'Δ_edu': array([ 0.30165586, -0.30506267, -0.76876361,  0.43540856]), 'Δ_gender_race': array([-0.97320217, -0.19921256,  0.52635928, -0.3207633 ,  0.84945493,
        0.85494854]), 'Δ_poll': array([ 0.98573105,  0.48519804, -0.29354434,  0.19148254, -0.67718653]), 'Δ_region': array([-0.1989569 , -0.67445018, -0.54294033,  0.56675434,  0.39864459]), 'Δ_state': array([ 0.96631886, -0.71342499, -0.64917904,  0...1, -0.66963936, -0.99304416,
        0.20238956]), 'β0': array(-0.42800109), ...}, 'step': <pymc3.step_methods.hmc.nuts.NUTS object>, 'trace': None, ...})]
    132 
    133     def __len__(self):
    134         return self._size
    135 

...........................................................................
C:\Users\El_Pentagono\Anaconda3\envs\py36\lib\site-packages\joblib\parallel.py in <listcomp>(.0=<list_iterator object>)
    126     def __init__(self, iterator_slice):
    127         self.items = list(iterator_slice)
    128         self._size = len(self.items)
    129 
    130     def __call__(self):
--> 131         return [func(*args, **kwargs) for func, args, kwargs in self.items]
        func = <function _sample>
        args = ()
        kwargs = {'chain': 1, 'draws': 1500, 'live_plot': False, 'live_plot_kwargs': None, 'model': <pymc3.model.Model object>, 'progressbar': False, 'random_seed': 4260026, 'start': {'kerry': array(0.032483), 'relig': array(-0.44432057), 'Δ_age': array([0.93412839, 0.37075697, 0.01745296, 0.78666723]), 'Δ_age_edu': array([ 0.59174212, -0.71198657,  0.62474506, -0...8, -0.82434136,  0.04314941,
       -0.61229957]), 'Δ_edu': array([ 0.30165586, -0.30506267, -0.76876361,  0.43540856]), 'Δ_gender_race': array([-0.97320217, -0.19921256,  0.52635928, -0.3207633 ,  0.84945493,
        0.85494854]), 'Δ_poll': array([ 0.98573105,  0.48519804, -0.29354434,  0.19148254, -0.67718653]), 'Δ_region': array([-0.1989569 , -0.67445018, -0.54294033,  0.56675434,  0.39864459]), 'Δ_state': array([ 0.96631886, -0.71342499, -0.64917904,  0...1, -0.66963936, -0.99304416,
        0.20238956]), 'β0': array(-0.42800109), ...}, 'step': <pymc3.step_methods.hmc.nuts.NUTS object>, 'trace': None, ...}
    132 
    133     def __len__(self):
    134         return self._size
    135 

...........................................................................
C:\Users\El_Pentagono\Anaconda3\envs\py36\lib\site-packages\pymc3\sampling.py in _sample(draws=1500, step=<pymc3.step_methods.hmc.nuts.NUTS object>, start={'kerry': array(0.032483), 'relig': array(-0.44432057), 'Δ_age': array([0.93412839, 0.37075697, 0.01745296, 0.78666723]), 'Δ_age_edu': array([ 0.59174212, -0.71198657,  0.62474506, -0...8, -0.82434136,  0.04314941,
       -0.61229957]), 'Δ_edu': array([ 0.30165586, -0.30506267, -0.76876361,  0.43540856]), 'Δ_gender_race': array([-0.97320217, -0.19921256,  0.52635928, -0.3207633 ,  0.84945493,
        0.85494854]), 'Δ_poll': array([ 0.98573105,  0.48519804, -0.29354434,  0.19148254, -0.67718653]), 'Δ_region': array([-0.1989569 , -0.67445018, -0.54294033,  0.56675434,  0.39864459]), 'Δ_state': array([ 0.96631886, -0.71342499, -0.64917904,  0...1, -0.66963936, -0.99304416,
        0.20238956]), 'β0': array(-0.42800109), ...}, trace=None, chain=1, tune=500, progressbar=False, model=<pymc3.model.Model object>, random_seed=4260026, live_plot=False, live_plot_kwargs=None, **kwargs={})
    327                             tune, model, random_seed)
    328     if progressbar:
    329         sampling = tqdm(sampling, total=draws)
    330     try:
    331         strace = None
--> 332         for it, strace in enumerate(sampling):
        it = undefined
        strace = None
        sampling = <generator object _iter_sample>
    333             if live_plot:
    334                 if live_plot_kwargs is None:
    335                     live_plot_kwargs = {}
    336                 if it >= skip_first:

...........................................................................
C:\Users\El_Pentagono\Anaconda3\envs\py36\lib\site-packages\pymc3\sampling.py in _iter_sample(draws=1500, step=<pymc3.step_methods.hmc.nuts.NUTS object>, start={'kerry': array(0.032483), 'relig': array(-0.44432057), 'Δ_age': array([0.93412839, 0.37075697, 0.01745296, 0.78666723]), 'Δ_age_edu': array([ 0.59174212, -0.71198657,  0.62474506, -0...8, -0.82434136,  0.04314941,
       -0.61229957]), 'Δ_edu': array([ 0.30165586, -0.30506267, -0.76876361,  0.43540856]), 'Δ_gender_race': array([-0.97320217, -0.19921256,  0.52635928, -0.3207633 ,  0.84945493,
        0.85494854]), 'Δ_poll': array([ 0.98573105,  0.48519804, -0.29354434,  0.19148254, -0.67718653]), 'Δ_region': array([-0.1989569 , -0.67445018, -0.54294033,  0.56675434,  0.39864459]), 'Δ_state': array([ 0.96631886, -0.71342499, -0.64917904,  0...1, -0.66963936, -0.99304416,
        0.20238956]), 'β0': array(-0.42800109), ...}, trace=None, chain=1, tune=500, model=<pymc3.model.Model object>, random_seed=4260026)
    425     try:
    426         for i in range(draws):
    427             if i == tune:
    428                 step = stop_tuning(step)
    429             if step.generates_stats:
--> 430                 point, states = step.step(point)
        point = {'kerry': array(0.032483), 'relig': array(-0.44432057), 'Δ_age': array([0.93412839, 0.37075697, 0.01745296, 0.78666723]), 'Δ_age_edu': array([ 0.59174212, -0.71198657,  0.62474506, -0...8, -0.82434136,  0.04314941,
       -0.61229957]), 'Δ_edu': array([ 0.30165586, -0.30506267, -0.76876361,  0.43540856]), 'Δ_gender_race': array([-0.97320217, -0.19921256,  0.52635928, -0.3207633 ,  0.84945493,
        0.85494854]), 'Δ_poll': array([ 0.98573105,  0.48519804, -0.29354434,  0.19148254, -0.67718653]), 'Δ_region': array([-0.1989569 , -0.67445018, -0.54294033,  0.56675434,  0.39864459]), 'Δ_state': array([ 0.96631886, -0.71342499, -0.64917904,  0...1, -0.66963936, -0.99304416,
        0.20238956]), 'β0': array(-0.42800109), ...}
        states = undefined
        step.step = <bound method GradientSharedStep.step of <pymc3.step_methods.hmc.nuts.NUTS object>>
    431                 if strace.supports_sampler_stats:
    432                     strace.record(point, states)
    433                 else:
    434                     strace.record(point)

...........................................................................
C:\Users\El_Pentagono\Anaconda3\envs\py36\lib\site-packages\pymc3\step_methods\arraystep.py in step(self=<pymc3.step_methods.hmc.nuts.NUTS object>, point={'kerry': array(0.032483), 'relig': array(-0.44432057), 'Δ_age': array([0.93412839, 0.37075697, 0.01745296, 0.78666723]), 'Δ_age_edu': array([ 0.59174212, -0.71198657,  0.62474506, -0...8, -0.82434136,  0.04314941,
       -0.61229957]), 'Δ_edu': array([ 0.30165586, -0.30506267, -0.76876361,  0.43540856]), 'Δ_gender_race': array([-0.97320217, -0.19921256,  0.52635928, -0.3207633 ,  0.84945493,
        0.85494854]), 'Δ_poll': array([ 0.98573105,  0.48519804, -0.29354434,  0.19148254, -0.67718653]), 'Δ_region': array([-0.1989569 , -0.67445018, -0.54294033,  0.56675434,  0.39864459]), 'Δ_state': array([ 0.96631886, -0.71342499, -0.64917904,  0...1, -0.66963936, -0.99304416,
        0.20238956]), 'β0': array(-0.42800109), ...})
    170     def step(self, point):
    171         self._logp_dlogp_func.set_extra_values(point)
    172         array = self._logp_dlogp_func.dict_to_array(point)
    173 
    174         if self.generates_stats:
--> 175             apoint, stats = self.astep(array)
        apoint = undefined
        stats = undefined
        self.astep = <bound method NUTS.astep of <pymc3.step_methods.hmc.nuts.NUTS object>>
        array = array([ 2.27235415e+00,  9.85731051e-01,  4.8519...33e-01,  5.66754344e-01,
        3.98644594e-01])
    176             point = self._logp_dlogp_func.array_to_full_dict(apoint)
    177             return point, stats
    178         else:
    179             apoint = self.astep(array)

...........................................................................
C:\Users\El_Pentagono\Anaconda3\envs\py36\lib\site-packages\pymc3\step_methods\hmc\nuts.py in astep(self=<pymc3.step_methods.hmc.nuts.NUTS object>, q0=array([ 2.27235415e+00,  9.85731051e-01,  4.8519...33e-01,  5.66754344e-01,
        3.98644594e-01]))
    177         p0 = self.potential.random()
    178         start = self.integrator.compute_state(q0, p0)
    179 
    180         if not np.isfinite(start.energy):
    181             raise ValueError('Bad initial energy: %s. The model '
--> 182                              'might be misspecified.' % start.energy)
        start.energy = inf
    183 
    184         if not self.adapt_step_size:
    185             step_size = self.step_size
    186         elif self.tune:

ValueError: Bad initial energy: inf. The model might be misspecified.
___________________________________________________________________________
"""

My model.test_point gives me the following:

‘kerry’: array(0.),
‘relig’: array(0.),
‘Δ_age’: array([0., 0., 0., 0.]),
‘Δ_age_edu’: array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]),
‘Δ_edu’: array([0., 0., 0., 0.]),
‘Δ_gender_race’: array([0., 0., 0., 0., 0., 0.]),
‘Δ_poll’: array([0., 0., 0., 0., 0.]),
‘Δ_region’: array([0., 0., 0., 0., 0.]),
‘Δ_state’: array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]),
‘β0’: array(-0.69362033),
‘σ_age_edu_log__’: array(1.60943791),
‘σ_age_log__’: array(1.60943791),
‘σ_edu_log__’: array(1.60943791),
‘σ_gender_race_log__’: array(1.60943791),
‘σ_poll_log__’: array(1.60943791),
‘σ_region_log__’: array(1.60943791),
‘σ_state_log__’: array(1.60943791)

I looked for simmilar errors but in every of them an inf or -inf appeared in some of the nodes. Here is not the case. I´m just executing each cell, not doing any changes to them. Am I the only one who have this error?

Any help is appreciated.

The jittering in the current default initialisation jitter+adapt_diag sometimes makes the start value invalid. Try setting the init='adapt_diag' in pm.sample(...). You can also set init='advi' to completely replicate the notebook.

2 Likes

I didn´t notice until you pointed that out!. It completely solved my problem. I guess I´ll read more about these different initialization techniques to know their differences

Thanks for the tip!

1 Like