Block Random Walk Metropolis Hastings

Questions v5
1

Hi,

I’m trying to replicate someone else’s work with PyMC before switching to more modern sampling.
They use a block random-walk metropolis hastings algorithm. Is this possible to do with PyMC at all?

For a simple example, consider a model that is a composition of an AR(1) and an MA(1):

loglik(theta) = omega * loglik_ar(theta1) + (1 - omega) * loglik_ma(theta2).

Where theta = (rho, beta, sigma, omega), theta1 = (rho, sigma), theta2 = (beta, sigma).

I’d like to sample with RWMH with the hessian at the mode as the proposal varcov and 3 blocks of paramters: [rho, beta], [sigma] and [omega].

I realize this is a somewhat trivial example. There are more complex models involved where the blocks have more meaning.

The exercise I’m doing is whether or how much it is to be gained from switching to e.g. NUTS.

Edit: In addition, is there a way to target/tune for a specific acceptance rate?

Maybe I’m looking in the wrong places, but couldn’t find that much info about Metropolis samplers in the docs.

1 Like
2

Edit: In addition, is there a way to target/tune for a specific acceptance rate?

You can do this for HMC / NUTS, but as far as I can tell it’s not an option for the RW M-H. This block in the code shows how the tuning is done; essentially the tuning is done so that the acceptance rate falls between 0.2 and 0.5.

With regard to the block sampling, you can try something like this which gives each block a separate MH step:

import pymc as pm

with pm.Model() as model:
    x = pm.Normal('x', shape=5)
    y = pm.Normal('y', mu=x, shape=5)
    z = pm.Normal('z', mu=y, shape=5)

    steps = [pm.Metropolis([x,y]), pm.Metropolis([z])]
    idata= pm.sample(step=steps)
1 Like
3

Awesome, thanks for you input!

At least the blocks are super-simple, after using over the years lot of custom scripts that samples I gotta say this library is a dream to work with.

2 Likes