DEMetropolisZ sampler rejecting 100% of samples in high-dimensional(?) black-box likelihood

1

I’m trying to fit between 40 and 64 parameters in a physical model, using the DEMetropolisZ sampler with a black-box likelihood (negative root-mean squared error). Most of the parameters have strong, log-normal priors. DEMetropolisZ runs without error but when examining the posteriors, it appears that the initial value is propagated throughout the chain (i.e., 0% of the samples are accepted). If I fix all but 8 of the free parameters, then I get good mixtures, i.e., >0% of the samples are accepted).

In reading related topics in this forum, there is reference to “high-dimensional” problems causing problems for samplers. I don’t know if 40 free parameters qualifies as “high-dimensional" but is this a known issue? What should I try instead?

The model looks something like the following:

import pymc as pm
import pytensor.tensor as pt

with pm.Model as model:
    alpha = external_parameters_list[0] # A scalar, example of fixed parameter
    beta = pm.LogNormal('beta', **external_prior_dict['beta'])
    ... # 39 more free parameters

    params_list.append(alpha, beta)
    params_tensor = pt.as_tensor_variable(params_list)
    pm.Potential('likelihood', black_box_likelihood(params_tensor))

The DEMetropolisZ sampler is initialized with its defaults (3 chains, tuning the “scaling” hyperparameter).

2

you could try sample_smc

3

it’s new and improved!

4

Great, thanks! As a point of reference, I was able to get DEMetropolisZ to accept samples with as many as 24 free parameters, but not 32 free parameters.

I can confirm that the SMC sampler works as a drop-in replacement for DEMetropolisZ with 8 free parameters. I have a job running with 64 parameters and I believe it will work because Beta is increasing. It is hard to know how long the job will take because in the progress bar(s), the clock never changes:

Chain 0 ⠸ -:--:-- / 0:21:53 Stage: 0 Beta: 1.000
Chain 1 ⠸ -:--:-- / 0:21:53 Stage: 0 Beta: 1.000
Chain 2 ⠸ -:--:-- / 0:21:53 Stage: 0 Beta: 1.000

That is, -:--:-- never updates. Not sure if I should be seeing something there.

It also appears that the multiple chains are not running in parallel, though multi-job chains will finish (they just run serially), whereas DEMetropolisZ was able to run in parallel. I’m going to play around with the cores, blas_cores, and mp_ctx options to see if I can get the chains to run in parallel. It might not matter in the end because multiple chains are only used for convergence diagnostics in SMC(?) and not for learning as in DEMetropolisZ.