Theano.shared in parallel loop

Questions
1

I have some models that I’m fitting in parallel and I’m running into some problems with the theano compile lock. The general form of my model is the same for each parallel run, I just change some input data. I would like to define one my variables that change as a theano.shared so that I don’t have the compile lock problem. Can I do something like:

X = theano.shared(X0)
...some pymc3 model....

for X_new in X_collection:
    X.set_value(X_new)
    with model:
        pm.sample(model=model,...)

and parallelize the loop and have it be thread safe?

2

You can certainly do this, but I am not sure if you parallelize the for loop it would be safe - how are you planning to parallel it?

3

@junpenglao I was planning on parallelizing the for loop so that I would be modifying X and submitting it in each thread. My hope was to avoid having several simultaneous theano locks that way. Would that work?

4

I have never tried - maybe @aseyboldt and @hvasbath has some experience.

5

I am not sure but this is exactly how you would run again into the compile lock, as then each model you have will be built in the initialsiation of pm.sample-depending on how you finally run your parallel code.
I would suggest you initialise your sampler first ergo the model/thenao function is compiled once.
Then you can run the sampling in parallel with sample, but hand over the step (initialised sampler object). This forks the theano graph and then you do the set_value() in the initialisation of the child process. I have done something like this here:

The update_weights there basically does a shared.set_value() after it calculated the new weights.
The init_chain_hypers is then used here when it is parallelized: https://github.com/hvasbath/beat/blob/master/src/models/base.py#L236

Good luck!

6

@hvasbath Sorry for my delay in responding. I had to shift gears to another project.

I’ll give that a shot. Thanks!