If you are using a model with custom operations, it would be helpful if you could share it. It sounds to me like you get nans somewhere (not the initial point), so maybe add a res = switch(isnan(x), -np.inf, res) if you have a custom logp and see how it fares.
You should however, figure out the source of your nan, usually that highlights a bug or a bad parametrization.