Multivariate time series inference

Questions
4

I think there is actually a lot of room for improvement in PyMC3’s treatment of time-series, though I have not been using PyMC3 for very long, I may just not be aware of the best way to proceed.

That being said, one the most powerful tools for modelling multivariate time series is the markovian state space model, so it seems natural to want to have this directly available. I’m working on implementing this model in PyMC3 here: https://github.com/RJTK/pymc3-kalman, where I forked an existing Kalman filter implementation (the likelihood comes from running a forward Kalman filtering pass). Unfortunately, this implementation seems to be extremely slow.

I have also had some success with VAR models implemented just with the pm.Normal distribution and some looping, e.g.

x_target, X_lagged = make_lagged_data(X, p)
x_target = x_target[:, 0, :]

T, n, K = X.shape  # Time, nodes, reps

with pm.Model() as model:
    beta = pm.Uniform("beta", -1, 1, shape=(n, p))
    mu = pm.StudentT("mu", nu=3)
    sd = pm.HalfStudentT("sv", nu=3)

    pred = 0.0
    for tau in range(p):
        for i in range(n):
            pred += beta[i, tau] * X_lagged[tau][:, i]
    pm.Normal("ar", mu=mu + pred, sd=sd,
              observed=x_target)

which is set up for one-step-ahead prediction of a target series from a collection of lagged components. This is fast enough with ADVI, but is surely not the best way to go about building bigger time series models.

Here is a relevant and recent paper on Bayesian multivariate time series: http://www.jmlr.org/papers/volume19/18-009/18-009.pdf

5 Likes