I would like to better understand how to manually initialise a pymc.fit when using method='advi'.
Specifically I am using
advi = pymc.ADVI()
tracker = pymc.variational.callbacks.Tracker(
mean = advi.approx.mean.eval,
std = advi.approx.std.eval,
)
in order to ‘track’ the parameters (are those ‘model parameters’ or the ADVI parameters \mu and \rho?) of the fit.
How can I use tracker["mean"] and tracker["std"] as start and start_std of pymc.fit? Does that even make sense?
In How to initialize ADVI it is suggested to use the posterior mean of the model parameters to initialize the fit. I would have expected that I need to supply starting parameters for \mu and \rho instead?
Arguments start and start_std require a dictionary but tracker["mean"][-1] is an array, i.e. I am missing the correct keys, unless I really need to specify the model parameters. In that case should I use the posterior standard deviation to initialise start_std?
So essentially I am confused as to which parameters I need to supply for the initialisation? The one of the model or the one of the ADVI (mean field) approximation? And if I need to specify the parameters for the approximation, \mu and \rho, which are the correct keys for the start and start_std dictionaries?