I came across A book called dynamic linear models with r. I’m curious if there are any PyMC3 resources for learning to construct such models? For those unfamiliar dlm is basically time series meets differential equations
Hi @jbuddy_13!
Yes, @brandonwillard wrote a very good blog post showing how to do that with PyMC3 and Theano – spoiler: this should become easier with the coming 4.0.0
And you’re in luck, I just interviewed Brandon on my podcast, and we talked just about that
Hope that helps
My blog post is rather Theanospecific–i.e. there’s no PyMC3 involved.
Fortunately, the RandomVariable
s used in that post are now available in our TheanoPyMC fork, so there’s no need for symbolicpymc
.
We’re working on some major updates to PyMC3 that will replace its current use of Distribution
s with RandomVariable
s. That, along with some updates to the samplers in PyMC3, will make it possible to specify Dynamic Linear Models in a similar–if not the same–way and make use of PyMC3’s sampler suite.
Otherwise, DLMs can be discretized diff. eqs for some stochastic processes, but the usefulness of that relationship–as a description of DLMs–diverges quite rapidly from there.
Hi @brandonwillard, I think I’ll wait for the release you’ve mentioned. I’m not an expert in DLMs (or the theory therein), but would like to play around with high level API and see if it’s relevant in my line of work.
If I did want to do some more reading on the theory behind the subject, any recommendations?
Am I right that the models specified in the linked blog post could be specified and fit using regular old pymc3 (no Theano scan), but with two caveats:

You can only generate the smoothed posterior distributions, not the filtered, and

The sampling might not be as efficient as when using FFBS
I don’t believe any of the filtering or smoothing could be done without a Scan
Op
, unless the models were of a very specific and simple variety (e.g. they had Markov dependencies that could be reduced to vectorized operations).
PyMC3 operates almost exclusively on loglikelihoods, which are absent from that exposition. In order to use PyMC3’s loglikelihood driven samplers, one would need to convert those “samplespace” (i.e. random variable) graphs to “measurespace” (i.e. loglikelihood) graphs.
Here’s a rough draft describing the manual conversion of such graphs into PyMC3compatible Distribution
s. I had started to create some tools to automate the process, but quickly noticed that certain aspects of PyMC3 make that unnecessarily difficult and inconsistent. The correct approach is to make these changes from within PyMC3, and that’s what we’re starting to do.
Maybe that’s what I’m thinking of  a very specific and simple variety. The example I have in mind is a Dynamic Factor model with factor loadings fixed through time, e.g. this example.
Hi @brandonwillard,
Thank you for this great piece of software!
Some time ago, I tried to implement a DLM using the scan function, but without luck.
What is the update status on PyMC3 for implementing DLMs?
For reference, I read your posts Theano Model Graphs  Brandon T. Willard and
Dynamic Linear Models in Theano  Brandon T. Willard
Thanks!
We’re focusing our efforts on PyMC v4 and the Theano fork, Aesara, that powers it. Within the Aesara ecosystem, we have a project called AePPL that is now capable of generating loglikelihoods for arbitrary Scan
Op
s: Add support for `Scan` `Op`s in `joint_logprob` by kc611 · Pull Request #24 · aesaradevs/aeppl · GitHub.
In other words, we’ve just implemented the features from my earlier comment in AePPL, and this functionality will be available in PyMC v4 at some point.
If you’re interested, please, give that new AePPL feature a try and report any issues you have along the way.
AePPL looks great! I am so glad of posting the question. Just in time to try it. I will report any issues. Thanks again!