NaN occurred in optimization (error in DL with variational minibatches)

[junpenglao]

Have you tried the methods to debug in this post?

Frequently Asked Questions

NaN occurred in optimization at the first iteration in VI (i.e., seeing something like 0%| | 0/5000 [00:00<?, ?it/s]) Check the initial approximation parameters: approx.approx.params[0].eval(). If it returns nan, that is what causing the error. Setting the value manually to zero could solve this. Below is an example if you are doing ADVI: mu_approx = approx.approx.params[0] rho_approx = approx.approx.params[1] mu_approx.set_value(np.zeros(mu_approx.eval().shape)) rho_approx.set_value(np.ze…