Questions about Normalizing Flows and pm.NFVI

Questions
1

Hello everyone,

I am now trying to use Normalizing Flow for variational inference. And I am facing these difficulties when using pm.NFVI in Pymc3:

1). How could I set initial values for the parameters of Flow (initial value for Flows)?
My model is:

formula = 'scale-planar*8-loc'

with basic:
inference = pm.variational.inference.NFVI(formula)

I found there are similar topics and questions on ADVI, and I also use the recommended

inference.approx.params[0].set_value(my_loc_ini)
inference.approx.params[-1].set_value(my_rho_ini)

for initialization, but is there any more efficient way to realize that? Considering there are too many parameters for complex Flows combination.
In the ā€œVariational APIā€, I found the first parameters of pm.NFVI() is the flow parameter, and have this description:

flowstr|AbstractFlow

formula or initialized Flow, default is ā€˜scale-locā€™ that is identical to MeanField

So how can I set this ā€œinitialized Flowā€?

2). What is the initial distribution q_0 defined in Pymc3 for pm.NFVI? Is that a Standard Gaussian like ADVI? Or can I define it with any analytical form by myself?
Previously, I thought the initial distribution q_0 is set to be standard Gaussian like ADVI, but when I use a flow of "scale-loc" and set [loc, rho] = [[0], [1]], I found the transformed distribution is not a standard Gaussian (if we set [loc, rho] = [[0], [1]], , the transformed distribution q_K should be exactly the same as the initial q_0).

formula = 'scale-loc'

with basic:
    inference = pm.variational.inference.NFVI(formula)

inference.approx.params[0].set_value([0])
inference.approx.params[-1].set_value([1])

nf_ini = inference.approx.sample(5000)['x']

The main reason I use Normalizing Flow is because it might provide a (semi)-analytical posterior for complexed problem (ADVI could just handle simple problems), so the parametric (analytical) form of q_0 matters a lot if I want to trace the posterior from q_0.

3). In Pymc3, is there a provided function to directly give users the posteriorā€™s solution based on Normalizing Flow? Or I have to trace it by myself by calculating Jacobi determinant?

Many thanks for your answers.

1 Like