I was reading through Rethinking 2 today, and came across this paragraph, which is an interesting analogy for the difference between the first approach and the second:
“In an apocryphal telling of Hindu cosmology, it is said that the
Earth rests on the back of a great elephant, who in turn stands on the back of a massive turtle.
When asked upon what the turtle stands, a guru is said to reply, “it’s turtles all the way down.”
Statistical models don’t contain turtles, but they do contain parameters. And parameters
support inference. Upon what do parameters themselves stand? Sometimes, in some of
the most powerful models, it’s parameters all the way down. What this means is that any
particular parameter can be usefully regarded as a placeholder for a missing model. Given
some model of how the parameter gets its value, it is simple enough to embed the new model
inside the old one. This results in a model with multiple levels of uncertainty, each feeding
into the next—a multilevel model.”
And then later: “We will be interested in multilevel models primarily because they help us deal with over-fitting”