Questions about Normalizing Flows and pm.NFVI

Hello everyone,

I am now trying to use Normalizing Flow for variational inference. And I am facing these difficulties when using pm.NFVI in Pymc3:

1). How could I set initial values for the parameters of Flow (initial value for Flows)?
My model is:

formula = 'scale-planar*8-loc'

with basic:
inference = pm.variational.inference.NFVI(formula)

I found there are similar topics and questions on ADVI, and I also use the recommended


for initialization, but is there any more efficient way to realize that? Considering there are too many parameters for complex Flows combination.
In the “Variational API”, I found the first parameters of pm.NFVI() is the flow parameter, and have this description:


formula or initialized Flow, default is ‘scale-loc’ that is identical to MeanField

So how can I set this “initialized Flow”?

2). What is the initial distribution q_0 defined in Pymc3 for pm.NFVI? Is that a Standard Gaussian like ADVI? Or can I define it with any analytical form by myself?
Previously, I thought the initial distribution q_0 is set to be standard Gaussian like ADVI, but when I use a flow of "scale-loc" and set [loc, rho] = [[0], [1]], I found the transformed distribution is not a standard Gaussian (if we set [loc, rho] = [[0], [1]], , the transformed distribution q_K should be exactly the same as the initial q_0).

formula = 'scale-loc'

with basic:
    inference = pm.variational.inference.NFVI(formula)


nf_ini = inference.approx.sample(5000)['x']

The main reason I use Normalizing Flow is because it might provide a (semi)-analytical posterior for complexed problem (ADVI could just handle simple problems), so the parametric (analytical) form of q_0 matters a lot if I want to trace the posterior from q_0.

3). In Pymc3, is there a provided function to directly give users the posterior’s solution based on Normalizing Flow? Or I have to trace it by myself by calculating Jacobi determinant?

Many thanks for your answers.

1 Like