I tried lower initial learning rates from 0.1 to 0.0001 or 2e-4. But same error. From your previous posts I tried:
mu = inference.approx.params[0]
rho = inference.approx.params[1]
mu.eval() Resulted in: array([nan, nan, nan, …, nan, nan, nan])
Setting mu and rho to zeros and again running samples results in same NaN errors.
mu.set_value(np.zeros(mu.eval().shape))
rho.set_value(np.zeros(rho.eval().shape))