Sample Binomial module initial error

Questions
1

When I sample a Binomial module which is:

n=127
y=14
with pm.Model() as m_valid:
    # a priori
    theta = pm.Beta('theta', alpha=1, beta=1)
    # likelihood
    obs = pm.Binomial('observed', n=n, p=theta, observed=y)
    #sampling
    trace_m_valid = pm.sample(1000)

It will meet ‘Bad initial energy’
The strange thing is, only n=127 the error will occur, other number say n=126, n=129 all OK,
Why n=127 is so strange?

the error detail is:

Auto-assigning NUTS sampler…
Initializing NUTS using jitter+adapt_diag…
Multiprocess sampling (4 chains in 4 jobs)
NUTS: [theta]
Sampling 4 chains: 0%| | 0/6000 [00:00<?, ?draws/s]/home/huwendi/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:3118: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/home/huwendi/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:3118: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/home/huwendi/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:3118: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/home/huwendi/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:3118: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)

Bad initial energy, check any log probabilities that are inf or -inf, nan or very small:
observed inf

RemoteTraceback Traceback (most recent call last)
RemoteTraceback:
“”"
Traceback (most recent call last):
File “/home/huwendi/anaconda3/lib/python3.7/site-packages/pymc3/parallel_sampling.py”, line 123, in _start_loop
point, stats = self._compute_point()
File “/home/huwendi/anaconda3/lib/python3.7/site-packages/pymc3/parallel_sampling.py”, line 154, in _compute_point
point, stats = self._step_method.step(self._point)
File “/home/huwendi/anaconda3/lib/python3.7/site-packages/pymc3/step_methods/arraystep.py”, line 247, in step
apoint, stats = self.astep(array)
File “/home/huwendi/anaconda3/lib/python3.7/site-packages/pymc3/step_methods/hmc/base_hmc.py”, line 149, in astep
raise SamplingError(“Bad initial energy”)
pymc3.exceptions.SamplingError: Bad initial energy
“”"

The above exception was the direct cause of the following exception:

SamplingError Traceback (most recent call last)
SamplingError: Bad initial energy

The above exception was the direct cause of the following exception:

ParallelSamplingError Traceback (most recent call last)
in ()
10 obs = pm.Binomial(‘observed’, n=n, p=theta, observed=y)
11 #sampling
—> 12 trace_m_valid = pm.sample(1000)

~/anaconda3/lib/python3.7/site-packages/pymc3/sampling.py in sample(draws, step, init, n_init, start, trace, chain_idx, chains, cores, tune, nuts_kwargs, step_kwargs, progressbar, model, random_seed, live_plot, discard_tuned_samples, live_plot_kwargs, compute_convergence_checks, use_mmap, **kwargs)
437 _print_step_hierarchy(step)
438 try:
–> 439 trace = _mp_sample(**sample_args)
440 except pickle.PickleError:
441 _log.warning(“Could not pickle model, sampling singlethreaded.”)

~/anaconda3/lib/python3.7/site-packages/pymc3/sampling.py in _mp_sample(draws, tune, step, chains, cores, chain, random_seed, start, progressbar, trace, model, use_mmap, **kwargs)
988 try:
989 with sampler:
–> 990 for draw in sampler:
991 trace = traces[draw.chain - chain]
992 if (trace.supports_sampler_stats

~/anaconda3/lib/python3.7/site-packages/pymc3/parallel_sampling.py in iter(self)
343
344 while self._active:
–> 345 draw = ProcessAdapter.recv_draw(self._active)
346 proc, is_last, draw, tuning, stats, warns = draw
347 if self._progress is not None:

~/anaconda3/lib/python3.7/site-packages/pymc3/parallel_sampling.py in recv_draw(processes, timeout)
247 else:
248 error = RuntimeError(“Chain %s failed.” % proc.chain)
–> 249 six.raise_from(error, old_error)
250 elif msg[0] == “writing_done”:
251 proc._readable = True

~/anaconda3/lib/python3.7/site-packages/six.py in raise_from(value, from_value)

ParallelSamplingError: Bad initial energy

Thanks!

2

This samples fine on my machine (macOS and Python 3.6 running master). What platform and version of PyMC are you using?

3

I have the same error

Auto-assigning NUTS sampler...
Initializing NUTS using jitter+adapt_diag...
Multiprocess sampling (4 chains in 4 jobs)
NUTS: [theta]
Sampling 4 chains:   0%|                                                | 0/6000 [00:00<?, ?draws/s]/home/rosgori/anaconda3/lib/python3.6/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
  out=out, **kwargs)
/home/rosgori/anaconda3/lib/python3.6/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
  out=out, **kwargs)
/home/rosgori/anaconda3/lib/python3.6/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
  out=out, **kwargs)
/home/rosgori/anaconda3/lib/python3.6/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
  out=out, **kwargs)

Bad initial energy, check any log probabilities that are inf or -inf, nan or very small:
observed    inf
---------------------------------------------------------------------------
RemoteTraceback                           Traceback (most recent call last)
RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/rosgori/anaconda3/lib/python3.6/site-packages/pymc3/parallel_sampling.py", line 123, in _start_loop
    point, stats = self._compute_point()
  File "/home/rosgori/anaconda3/lib/python3.6/site-packages/pymc3/parallel_sampling.py", line 154, in _compute_point
    point, stats = self._step_method.step(self._point)
  File "/home/rosgori/anaconda3/lib/python3.6/site-packages/pymc3/step_methods/arraystep.py", line 247, in step
    apoint, stats = self.astep(array)
  File "/home/rosgori/anaconda3/lib/python3.6/site-packages/pymc3/step_methods/hmc/base_hmc.py", line 149, in astep
    raise SamplingError("Bad initial energy")
pymc3.exceptions.SamplingError: Bad initial energy
"""

The above exception was the direct cause of the following exception:

SamplingError                             Traceback (most recent call last)
SamplingError: Bad initial energy

The above exception was the direct cause of the following exception:

ParallelSamplingError                     Traceback (most recent call last)
<ipython-input-7-a88abb28b7f6> in <module>
      8     obs = pm.Binomial('observed', n=n, p=theta, observed=y)
      9     #sampling
---> 10     trace_m_valid = pm.sample(1000)
     11 

~/anaconda3/lib/python3.6/site-packages/pymc3/sampling.py in sample(draws, step, init, n_init, start, trace, chain_idx, chains, cores, tune, nuts_kwargs, step_kwargs, progressbar, model, random_seed, live_plot, discard_tuned_samples, live_plot_kwargs, compute_convergence_checks, use_mmap, **kwargs)
    437             _print_step_hierarchy(step)
    438             try:
--> 439                 trace = _mp_sample(**sample_args)
    440             except pickle.PickleError:
    441                 _log.warning("Could not pickle model, sampling singlethreaded.")

~/anaconda3/lib/python3.6/site-packages/pymc3/sampling.py in _mp_sample(draws, tune, step, chains, cores, chain, random_seed, start, progressbar, trace, model, use_mmap, **kwargs)
    988             try:
    989                 with sampler:
--> 990                     for draw in sampler:
    991                         trace = traces[draw.chain - chain]
    992                         if (trace.supports_sampler_stats

~/anaconda3/lib/python3.6/site-packages/pymc3/parallel_sampling.py in __iter__(self)
    343 
    344         while self._active:
--> 345             draw = ProcessAdapter.recv_draw(self._active)
    346             proc, is_last, draw, tuning, stats, warns = draw
    347             if self._progress is not None:

~/anaconda3/lib/python3.6/site-packages/pymc3/parallel_sampling.py in recv_draw(processes, timeout)
    247             else:
    248                 error = RuntimeError("Chain %s failed." % proc.chain)
--> 249             six.raise_from(error, old_error)
    250         elif msg[0] == "writing_done":
    251             proc._readable = True

~/anaconda3/lib/python3.6/site-packages/six.py in raise_from(value, from_value)

ParallelSamplingError: Bad initial energy

Python 3.6
PyMC3 3.6
Ubuntu 16.04.5 LTS

4

#49~14.04.1-Ubuntu

numpy 1.16.0
scipy 1.1.0
pymc3 3.6

compiler : GCC 7.2.0
system : Linux
release : 4.2.0-42-generic
machine : x86_64
processor : x86_64
CPU cores : 24
interpreter: 64bit

5

Python 3.7.0

6

I ran into this unusual bug as well. Here’s a simple reproduction:

def make_beta_binomial_model(n):
    k = n//2
    with pm.Model() as model:
        p = pm.Beta('p', 1, 1)
        obs = pm.Binomial('obs', n=n, p=p, observed=k)
    return model

# Good.
with make_beta_binomial_model(126):
    pm.sample()

# SamplingError!
with make_beta_binomial_model(127):
    pm.sample()

# Good
with make_beta_binomial_model(128):
    pm.sample()

The full traceback of the error is:

---------------------------------------------------------------------------
SamplingError                             Traceback (most recent call last)
Cell In [169], line 2
      1 with make_model(127):
----> 2     pm.sample()

File ~/.pyenv/versions/3.10.2/envs/bayes-experiment-job/lib/python3.10/site-packages/pymc3/sampling.py:428, in sample(draws, step, init, n_init, start, trace, chain_idx, chains, cores, tune, progressbar, model, random_seed, discard_tuned_samples, compute_convergence_checks, callback, jitter_max_retries, return_inferencedata, idata_kwargs, mp_ctx, pickle_backend, **kwargs)
    426 start = deepcopy(start)
    427 if start is None:
--> 428     check_start_vals(model.test_point, model)
    429 else:
    430     if isinstance(start, dict):

File ~/.pyenv/versions/3.10.2/envs/bayes-experiment-job/lib/python3.10/site-packages/pymc3/util.py:237, in check_start_vals(start, model)
    234 initial_eval = model.check_test_point(test_point=elem)
    236 if not np.all(np.isfinite(initial_eval)):
--> 237     raise SamplingError(
    238         "Initial evaluation of model at starting point failed!\n"
    239         "Starting values:\n{}\n\n"
    240         "Initial evaluation results:\n{}".format(elem, str(initial_eval))
    241     )

SamplingError: Initial evaluation of model at starting point failed!
Starting values:
{'p_logodds__': array(0.)}

Initial evaluation results:
p_logodds__   -1.39
obs             inf
Name: Log-probability of test_point, dtype: float64

Environment information:

PyMC version:  3.11.4
Python Version:  3.10.2 (main, Feb 15 2022, 15:26:26) [Clang 13.0.0 (clang-1300.0.29.3)]
Platform:  macOS-12.5.1-x86_64-i386-64bit

Some good news, this seems fixed in version 4:

In [7]: with make_beta_binomial_model(127):
   ...:     pm.sample()
   ...:
Auto-assigning NUTS sampler...
Initializing NUTS using jitter+adapt_diag...
Multiprocess sampling (4 chains in 4 jobs)
NUTS: [p]
Sampling 4 chains for 1_000 tune and 1_000 draw iterations (4_000 + 4_000 draws total) took 17 seconds. [8000/8000 00:01<00:00 Sampling 4 chains, 0 divergences]

In [8]: pm.__version__
Out[8]: '4.3.0'
7

I couldn’t replicate the problem locally nor on Colab: Google Colab