Limit range of parameters in hamiltonian monte carlo

[Hamster_on_wheels]

Undefined values in the potential energy function can cause problems:

NaN occurred in optimization with NUTS

Is it difficult from an algorithmic point of view or because of the implementation? I think the “NaN occurred in optimization” errors are a major pain point for using NUTS right now. I’ve seen a couple of models that sampled well when using previous chains to initialize the scaling, but don’t work with the advi initialization because of some stray nan somewhere in the beginning. As a workaround I’ve used something like this: with model: stds = np.ones(model.ndim) for _ in range(5): …

Just to be safe, I am thinking about patching the singularities in the potential energy function for x = 0 and x = 1. The gradient of the potential energy function has many (2000) terms like 1 / x and 1 / (x - 1).

Before calculating the potential energy and the gradient, I would do the following within the custom theano operation that defines potential energy. If |x| is near 0, set x to 0.01. If |x - 1| is near 0, set x to 0.99.