Correlated errors like this are called a Moving Average (MA) process. It’s not that the authors “want” the errors to correlate. They are positing a particular stochastic structure to the time series being studied.
Consider quarterly oil well production. An i.i.d. normally distributed error might be a freak well fire. As a result, the well’s production at time t will be reduced. In addition, at t+1, clean-up and repair might not be done, so we might expect additional disturbance, proportional to the amount of damage done by the original fire shock (\theta \in (0, 1)). After those two quarters, though, the effect of the fire is totally gone.
In contrast, if we modeled the oil well production as an autoregressive process, like y_t = \rho y_{t-1} + \epsilon_t, the fire shock will last far into the future. To see this, repeatedly substitute the AR process into itself. After \tau quarters, the effect of the fire will be \rho^\tau \epsilon_t, which might stay quite large for quite some time, depending on the magnitude of \rho.
As for the model, what specific errors are you getting? I strongly recommend you use NUTS instead of Metropolis. Metropolis is “easier to work with” in the sense that it complains less, but when NUTS complains it is telling you something is wrong with your model. Metropolis, on the other hand, is happy to silently return incorrect results (biased/incomplete posterior exploration).