Hi all. A quick question. I’ trying to sample a fairly complex model with SVGD, but I’ve only found two posts regarding SVGD convergence, and I’m still quite puzzled. My current, silly, approach is to do this:
svgd = pm.SVGD(model=mod, random_seed=33)
tracker = pm.callbacks.Tracker(mean=svgd.approx.mean.eval, std=svgd.approx.std.eval, hist=svgd.approx.histogram.eval)
approx = svgd.fit(callbacks=[tracker])
fig = plt.figure(figsize=(16, 9))
mu_ax = fig.add_subplot(221)
std_ax = fig.add_subplot(222)
hist_ax = fig.add_subplot(212)
mu_ax.plot(np.array(tracker["mean"]).mean(axis=1))
mu_ax.set_title("Mean track")
std_ax.plot(np.array(tracker["std"]).mean(axis=1))
std_ax.set_title("Std track")
hist_ax.plot(np.array(tracker["hist"]).mean(axis=1).T)
hist_ax.set_title("Negative ELBO track");
plt.savefig("svgd_convergence.png", dpi=300)
I just followed the approach in here: Introduction to Variational Inference with PyMC — PyMC example gallery, which produces the plot below but for ADVI convergence. I guess this may not be the correct approach for SVGD (I only used 30 sample as a example). What would be a good way to check SVGD convergence?
Many thanks in advance.