Just to be clear, the KF is an efficient method of computing the likelihood of a sequence given data, so it doesn’t really have anything to do with NUTS per se, except that NUTS consumes likelihood functions in order to generate posterior samples. But this is also true of anything else, including good old numerical minimization, so that’s why I say it doesn’t have anything to do with NUTS specifically.
You are correct that you can implement basically any model you like in PyMC without resorting to the specific matrix machinery of a linear statespace model – I’d even argue that the code is more readable without it. I’d say main advantage of the KF is automatic marginalization of hidden states and missing data. This is a very non-trivial point in these models that is very easy to get wrong – without the KF, at each step of the sequence you will need to manually compute the marginal distributions on hidden states/missing data given the observed data. There’s a (very rough) an example of how this ends up looking here.
By the way, if you’re interesting in working with dynamic linear models with kalman filtering, there’s a new statespace sub-module in pymc-experimental that provides them. You can see how the API works for structural models here. It’s still a work-in-progress, so if you try it and find any bugs, please report them!