Metropolis scaling: different for each RV?

Questions
1

Metropolis supports an initial scaling factor, which is quite useful. (ht: @twiecki for suggesting the scaling factor in his answer to another question)

Is it possible to have different initial scaling for each RV? I see from the sampling documentation that scaling can take an array instead of a scalar. But my naive experiments with providing scaling arrays only led to runtime errors. Is the format of this scaling array described somewhere?

2

It appears that there might be a problem with this argument. In the code block below, I try to use scaling for two RVs which have shape[2,3] and [4]. If the scaling argument has the first shape, then it fails when it tries to broadcast against the second and vice versa. To me, it looks like it only works if you have the same dimension for each RV here. See below.

import pymc3 as pm
import numpy as np

with pm.Model() as model:
    x = pm.Normal('x',shape=[2,3])
    y = pm.Normal('y',shape=4)
    step = pm.Metropolis(scaling=np.ones(4))
    trace = pm.sample(step=step,chains=1)
1 Like
3

Aha! The shape of the scaling array should be aligned to the shape of the RVs, suggesting there is no way to use a scaling array to provide different scaling for each RV.

But separately I realized I can adjust use compound steps for that purpose, to provide different scaling to different RVs.

4

Yeah, currently the multi-trace backend assumes the scaling is a scalar for each RV, so while the initialization might work (you can supply a flatten array as scaling), sampling is broken.

1 Like