I’m working on examples for SARIMAX now, but they’re not done. The way to do it will be analogous to this example of an ARIMA-GARCH model, except instead of an ARIMA model with GARCH distributed errors, you will have a regression model with ARIMA-distributed errors. The model is:
\begin{align}y_t &= X_t \beta + \eta_t \\ \eta_t &= \sum_{i=1}^p \rho_i \eta_{t-i} + \sum_{j=1}^q \theta_j \varepsilon_{t-j} + \varepsilon_t \end{align}
So inside the scan you need to compute eta = y - X @ beta, then model the mean of eta mu_eta = (rhos * eta_lags).sum() + (thetas * epsilon_lags).sum(). Compute the current step epsilon as epsilon_t = eta - mu_eta, and compute your model’s prediction as y ~ N(X @ beta + mu_eta, sigma)
This is all like what is done in the 2nd example in the GARCH notebook, except there the ARIMA dynamics play the role of the regression, and the GARCH dynamics play the role of the ARIMA dynamics.
Re: state-space, it’s just a mathematical framework for organizing time-series models. Many models can be written in the state-space framework, so it’s like a unifying family. Time-varying parameters can also be cast into a state-space (as latent “states” to be estimated, rather than single parameters), but they aren’t the defining feature.
You can think of an HMM as a type of non-linear state-space model, where one of the states (called the “state”, which is confusing) is discrete, while other state (the emission – the actual data we observe) is a continuous function of the discrete state.