I am now trying to use Normalizing Flow for variational inference. And I am facing these difficulties when using pm.NFVI in Pymc3:
1). How could I set initial values for the parameters of Flow (initial value for Flows)?
My model is:
formula = 'scale-planar*8-loc' with basic: inference = pm.variational.inference.NFVI(formula)
I found there are similar topics and questions on ADVI, and I also use the recommended
for initialization, but is there any more efficient way to realize that? Considering there are too many parameters for complex Flows combination.
In the “Variational API”, I found the first parameters of pm.NFVI() is the flow parameter, and have this description:
formula or initialized Flow, default is ‘scale-loc’ that is identical to MeanField
So how can I set this “initialized Flow”?
2). What is the initial distribution q_0 defined in Pymc3 for pm.NFVI? Is that a Standard Gaussian like ADVI? Or can I define it with any analytical form by myself?
Previously, I thought the initial distribution q_0 is set to be standard Gaussian like ADVI, but when I use a flow of
"scale-loc" and set
[loc, rho] = [, ], I found the transformed distribution is not a standard Gaussian (if we set
[loc, rho] = [, ], , the transformed distribution q_K should be exactly the same as the initial q_0).
formula = 'scale-loc' with basic: inference = pm.variational.inference.NFVI(formula) inference.approx.params.set_value() inference.approx.params[-1].set_value() nf_ini = inference.approx.sample(5000)['x']
The main reason I use Normalizing Flow is because it might provide a (semi)-analytical posterior for complexed problem (ADVI could just handle simple problems), so the parametric (analytical) form of q_0 matters a lot if I want to trace the posterior from q_0.
3). In Pymc3, is there a provided function to directly give users the posterior’s solution based on Normalizing Flow? Or I have to trace it by myself by calculating Jacobi determinant?
Many thanks for your answers.