Metropolis accept prob: which RV corresponds to which index?

The MultiTrace created by sampling using Metropolis supports accept, a useful statistic for looking at how the acceptable probability changes over the sampling for each RV. accept is a numpy array with the shape (steps, RVs), so when my model of 6 RVs samples for 10,000 steps, accept has 10000 rows and 6 columns.

But which RV corresponds to which column? The order of the RVs in the columns seems to be different from the order of the RVs defined in the model. Does the MultiTrace know the correspondance?

You can find out the mapping using the array_to_dict method:

logp = model.logp_dlogp_function()
logp.array_to_dict(array_slice)
1 Like

Thank you Junpeng.

By “array_slice”, do you mean a single sample? For my model a single sample of accept is a six-element array, i.e. of shape (6,). logp.array_to_dict() seems to expect an array of shape (402,).

Am I missing something?

I misunderstood your question :sweat_smile: array_slice is suppose to be a 1d array of all your free parameter. But in your case you are actually looking for something different - the MH acceptance ratio for each random variables.
There is no good way to read out this information as it is not currently saved in the trace, the best you can do is to read from the log.info like

CompoundStep
>Metropolis: [s]
>Metropolis: [x]
>Metropolis: [p]

which means the ordering will be s, x, p. To be on the safe side you can specific the ordering in compound step, see https://docs.pymc.io/notebooks/sampling_compound_step.html for more information

1 Like

Thank you for your help.

I am confused. The trace of the MH sampling of my model has four stats.

image

Each of those stats has the same shape: (10000, 6). I interpret that shape to be a row for each sample, and a column for each random variable.

accept seems to be the acceptance probability, and accepted seems to be whether the newly proposed value is actually accepted. At least it is the case that the True values for accepted are associated with larger values for accept.

But which column is associated with which random variable? They appear not to be in the same order as the RVs are defined in the model. At least the second RV in the model rarely takes a new value:

image

But the second column of accepted has quite a few new values.

image

The second from last column of accepted may the same as the second variable, as that column has only 53 new values, and eyeballing the above graph, there may be 53 changes in the line. If true, that would suggest that the order of the columns is the reverse of the order of RV definition.

But there must be a simpler way of determining which column of the stats is associated with which RV, a simpler way than eyeball comparison of the traceplot graphs to the accepted data.

Or maybe I am misunderstanding something, and jumping to conclusions about the MH stats.

(For folks in the future wondering about the answer to this question.)

By trial and error, I now understand which RV corresponds to which index in the trace stats. During sampling, the RVs are listed, e.g.
image

The first RV listed is the first (i.e. zero index) of the trace stats. The second RV listed in the second of trace stats. And so on.

So for this model, the array of acceptance probabilities for the RV skill_realizing is found via trace.accept[:,3]