I am using ADVI to find optimal parameters using the below code
with mod: inference = pm.ADVI() tracker = pm.callbacks.Tracker( mean= inference.approx.mean.eval, # callable that returns mean std= inference.approx.std.eval # callable that returns std ) approx = pm.fit(n= 20000, method=inference, callbacks=[tracker],obj_optimizer=pm.adam(learning_rate=0.0005), obj_n_mc=2)
-log(loss) looks like
20,000 iterations took 24 hours to run on a 16 CPU machine, the final parameters are close to the ground truth values though. I wonder if I can have a better learning rate, obj_n_mc and number of iterations? The loss looks too noisy and 24 hours is a lot. I tried increasing the learning rate but did not help.