How to disable convergence checks?

Questions
1

My model, with less than 100 parameters and with ~5000 observable samples nicely…
But crashes just after samping ends…

image
image1312×542 88.6 KB

I guess that this occurs when computing end of sampling tests (to detect diverging samples for instance) where pymc3 uses a lot of memory…

Can we disable these tests so I can avoid crash, and get the trace data…?

2

I believe the answers to this question will solve the issue. To solve the memory error, disabling the convergence checks should not be necessary, only avoiding the storage of pointwise log likelihood data.

3

ok thanks.
But how do I avoid the storage of pointwise likelihood data ?
(I guess that is a pm.sample option but I don’t find it.)

4

You have to use idata_kwargs=dict(log_likelihood=False) to avoid pointwise log likelihood data storage or compute_convergence_checks=False to disable convergence checks completely as explained in the topic linked above.

2 Likes
5

Even if the “pm.sample” method has a idata_kwargs parameter (I’ve checked the documentation).
It’s not accepted !

image
image1137×320 45.7 KB

yours second suggestion ( compute_convergence_checks=False) kinda works, and is accepted by the pm.sample method, but only avoid direct crash after the sampling… But the memory is still full and the problem arise later (outside pymc). So it doesn’t really solve my problem.

So I guess that not storing the likelyhood is the solution for me, but I didn’t succeed to use that setting. ;(

This is the full error message about the " idata_kwargs=dict(log_likelihood=False)" not accepted :

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-7-8c0064668e24> in <module>()
     12     obs = pm.Binomial('obs',n=nbr_try,p=(ctr[range(n_items),order]),observed=observed) # re_ordonnancer pour coller aux observables
     13 
---> 14     trace = pm.sample(draws=1000, idata_kwargs=dict(log_likelihood=False))

~/anaconda3/lib/python3.6/site-packages/pymc3/sampling.py in sample(draws, step, init, n_init, start, trace, chain_idx, chains, cores, tune, progressbar, model, random_seed, discard_tuned_samples, compute_convergence_checks, **kwargs)
    420                 random_seed=random_seed,
    421                 progressbar=progressbar,
--> 422                 **kwargs
    423             )
    424             if start is None:

~/anaconda3/lib/python3.6/site-packages/pymc3/sampling.py in init_nuts(init, chains, n_init, model, random_seed, progressbar, **kwargs)
   1687         raise ValueError("Unknown initializer: {}.".format(init))
   1688 
-> 1689     step = pm.NUTS(potential=potential, model=model, **kwargs)
   1690 
   1691     return start, step

~/anaconda3/lib/python3.6/site-packages/pymc3/step_methods/hmc/nuts.py in __init__(self, vars, max_treedepth, early_max_treedepth, **kwargs)
    146         `pm.sample` to the desired number of tuning steps.
    147         """
--> 148         super().__init__(vars, **kwargs)
    149 
    150         self.max_treedepth = max_treedepth

~/anaconda3/lib/python3.6/site-packages/pymc3/step_methods/hmc/base_hmc.py in __init__(self, vars, scaling, step_scale, is_cov, model, blocked, potential, dtype, Emax, target_accept, gamma, k, t0, adapt_step_size, step_rand, **theano_kwargs)
     70         vars = inputvars(vars)
     71 
---> 72         super().__init__(vars, blocked=blocked, model=model, dtype=dtype, **theano_kwargs)
     73 
     74         self.adapt_step_size = adapt_step_size

~/anaconda3/lib/python3.6/site-packages/pymc3/step_methods/arraystep.py in __init__(self, vars, model, blocked, dtype, **theano_kwargs)
    226 
    227         func = model.logp_dlogp_function(
--> 228             vars, dtype=dtype, **theano_kwargs)
    229 
    230         # handle edge case discovered in #2948

~/anaconda3/lib/python3.6/site-packages/pymc3/model.py in logp_dlogp_function(self, grad_vars, **kwargs)
    817         varnames = [var.name for var in grad_vars]
    818         extra_vars = [var for var in self.free_RVs if var.name not in varnames]
--> 819         return ValueGradFunction(self.logpt, grad_vars, extra_vars, **kwargs)
    820 
    821     @property

~/anaconda3/lib/python3.6/site-packages/pymc3/model.py in __init__(self, cost, grad_vars, extra_vars, dtype, casting, **kwargs)
    553 
    554         self._theano_function = theano.function(
--> 555             inputs, [self._cost_joined, grad], givens=givens, **kwargs)
    556 
    557     def set_extra_values(self, extra_vars):

TypeError: function() got an unexpected keyword argument 'idata_kwargs'
6

Which PyMC3 and ArviZ version are you using? idata_kwargs was introduced on version 3.9, so I’d strongly recommend updating to the latest release.

Yes, the conversion to ArviZ InferenceData will compute and store pointwise log likelihood data unless you manually modify the argument. Even if you are not calling az.from_pymc3 explicitly, any stats/plotting function either from ArviZ or PyMC3 (PyMC ones are actually aliases to ArviZ ones) will first convert the input data to InferenceData, raising the memory error again.

To avoid this second issue (in case you could not update to the latest PyMC3) you should make sure to have ArviZ>=0.7, and explicitly call az.from_pymc3(trace, log_likelihood=False) so that you can use InferenceData as the input to ArviZ functions and avoid the internal conversion which relies purely on the defaults and is not suitable for your current situation.

On a more personal note, I’d recommend to switch to InferenceData instead of MultiTrace wherever possible. We recently updated the A Primer on Bayesian Methods for Multilevel Modeling example to show some of the advantages of InferenceData, it could be useful. It also shows how named coords and dims can now be used directly within the PyMC3 model.

2 Likes