Does PyMC work for ABC SMC?

Questions
,
1

Hi everyone, I’m new here and new to Bayesian modeling. I’m working on something that involves approximate bayesian inference with a physics engine (the physics engine is the simulator). Is it possible to use PyMC to perform ABC SMC with an external (i.e. not written in PyMC) simulator?

Thanks!

1 Like
2

Yes, you can take a look at Sequential Monte Carlo - Approximate Bayesian Computation — PyMC3 3.11.4 documentation which the simulator is written in numpy (any Python Callable would work similarly)

3 Likes
3

And here is the pm.Simulator() documentation that I can never seem to find.

1 Like
4

Thank you @junpenglao. I saw this, originally could not get it to work, and just updated to v3.11.4 and it now works. I’m able to get my model running in the ABC SMC framework as described in that article. Unfortunately, it seems to hang after the first stage:

Initializing SMC sampler...
Sampling 2 chains in 1 job
/Users/lollipop/miniconda3/envs/pymc311/lib/python3.9/site-packages/theano/tensor/elemwise.py:826: RuntimeWarning: divide by zero encountered in log1p
  variables = ufunc(*ufunc_args, **ufunc_kwargs)
Stage:   0 Beta: 0.006
INFO:pymc3:Stage:   0 Beta: 0.006

After this first stage, which takes around 2-5 minutes on my MacBook Pro M1 Max, it hasn’t nudged after >20 minutes. I’m new to PyMC and there is very little documentation on ABC with PyMC (as @cluhmann alluded to with the pm.Simulator(), so excuse my initial lack of detail.

Is there a way to get a verbose output from either pm.Simulator or pm.sample_smc? Or, is there a check or a routine I can perform to see what’s going on?

Thanks again!

5

I think the Simulator is just struggling a lot. You can try to use a more forgiving epsilon / distance metric perhaps?

2 Likes
6

More forgiving epsilon would be, in this case, setting epsilon=<1? So we would be, in effect, dampening the divergence?

7

The other way. epsilon>1 makes the distance metric less peaked

1 Like
8

Thanks. I’ll report back with results. In the meantime, can I ask if I can use pm.Simulator() with other sampling methods (e.g. pm.sample)? I’m having a hard time understanding the control flow of these model builds and exactly what pieces are composable and what pieces aren’t (e.g. if I can only use pm.Simulator with pm.sample_smc). If I need to make a new post asking this specific question, let me know! :slight_smile:

9

You can use Simulator with Metropolis + pm.sample in the next release v4 (we have a beta out already).

However, Simulators/ABC have very little empirical support outside of SMC, so I wouldn’t endorse it without good reasons :slight_smile:

3 Likes
10

Hello everyone,

I am new here and also new to Bayesian modeling. I am working on a project that needs to use approximate Bayesian computation. I have seen many examples in the documentation and elsewhere using SMC-ABC in PyMC3, but none for the latest release.

I tried to implement a simple example using SMC with pm.sample_mc running in PyMC 4.0.0b6, but without success. From the new release, I could not understand whether this is my lack of knowledge of this framework or if it’s not an implemented feature.

Is SMC-ABC implemented in PyMC 4.0.0?

I thought my question would be relevant to this thread, but if necessary I can make a new post asking specifically about SMC in v4.

Thanks!

1 Like
11

Yup. If I recall correctly, there was quite a bit of work on it last summer in preparation for v4. However, the relevant example notebooks (e.g., this one) have not yet been updated. But doing so is on the to do list. I might suggest opening a new topic and maybe we can grab one of the relevant developers to provide some guidance.

12

@amanda.azevedo ABC is implemented in PyMC 4 whenever you use a Simulator in your Model. You can just call sample_smc without any special arguments and it will work out of the box. Note that the way we define Simulators changed slightly: pymc.Simulator — PyMC dev documentation

13

Thanks @cluhmann, I will open a new topic on this. :slight_smile:

@ricardoV94 thanks for the link. However, I am not able to run even the example on this page. I will then describe the error in the new topic.

14

I have an example of ABC in PyMC(3) here:

The Simulator class is not used; instead, a custom Theano operator for calculating the likelihood from the black-box model is defined.

1 Like