Noncentered Parametrizations

Questions
1

I have a question – why is reparametrizing to a non-centered form not done automatically if the benefits are so large? Having to specify a bunch of dummy variables that I don’t end up using doesn’t seem especially Pythonic. Is there a reason the user has to specify each one by hand and then use pm.Deterministic to scale them?

2

You should not assume that non-centered specifications are universally superior to their centered counterparts. See here for a bit of discussion.

2 Likes
3

Thank you! Do you know how I can tell before I run which will work better? Is there a way I can switch my model to a noncentered parametrization without rewriting the whole thing?

4

I do not. In general, my personal workflow is to build models incrementally, adding desirable complications bit by bit. This allows me to nail down the more fundamental aspects of the model before adding/tweaking the more subtle components. It also allows me to detect things like sampling issues and to more directly associate such problems with specific modeling choices. In many cases, I could sit down a write a huge model immediately, but then it would be much more of a struggle (for me) to correctly diagnose and rectify any “soft” sampling problems (e.g., things run ok, but there are divergences, sampling is super slow, etc.).

It’s possible to programatically construct models or components of models. This function might give you some sense of how this can work and specifically addresses the centered/non-centered issue. You could certainly get away with something simpler, but even then you would have to decide whether such leg work would be worth it in your typical use cases.

5

ISTR Michael Betancourt arguing that non-centered would often work better when data are scarce, and centered when data are plentiful, but I am away from my computer and can’t look it up. I am quite likely garbling or over simplifying…

@brandonwillard might have a way to automate the transformation using symbolic PyMC3.

1 Like
6

Here is Betancourt’s essay on the topic, esp Section 3.2 which compares centered and non-centered. It is as @rpgoldman says, Betancourt argues that the “well-informed” parameters are harder to sample in the non-centered parameterization.

In an example lower down in the essay he uses a heuristic threshold of 25 observations, below which he assigns parameters to the non-centered parameterization (and above, to the centered). Of course, the specific threshold would depend on the data, the likelihood, etc, but it seems like “mixed parameterizations” can be helpful.

1 Like