Multiple chains in Variational inference

Questions
1

  1. I would like to use multiple chains for ADVI. So far, i could not find a single documentation on how to sample multiple chains in ADVI?

  2. If multiple chains are not possible, i would like to append each run of pm.variational.approximations.sample_approx (…) such that i get a final trace of type

“<MultiTrace: ‘n’ chains, ‘x’ iterations, ‘m’ variables>”
from appending ‘n’ samples of
“<MultiTrace: ‘1’ chains, ‘x’ iterations, ‘m’ variables>”

How to append ?

2

For the second question ‘2.’ I found a solution,

Multi_chain_trace =
for i in range(required_chain_number) : #Repeat over required_chain_number #Loop begin
approx_trace = pm.variational.inference.fit(n = no_of_draws_required, method = ‘advi’)
single_chain_trace = approx_trace.sample(no_of_draws_required)
Multi_chain_trace.append(single_chain_trace)
Multi_chain_trace[i]._straces[i] = Multi_chain_trace[i]._straces.pop(0)

#End of Loop

Multi_chain_trace = pm.backends.base.merge_traces(Multi_chain_trace)

pm.traceplot(Multi_chain_trace)

One can append single chains this way.

3

Multiple chains are typically used to check MCMC convergence criteria. These criteria do not make sense for ADVI, as an approximating Gaussian is fit in closed form. The sampling you get from ADVI does not involve MCMC at all – and so there’s no such thing as a “chain” for ADVI – which is why there is no documentation on the subject.

1 Like