Hi,
I was wondering if this line should also be parallelized, as here. In the end we compute in parallel when the model computation is expensive, so each call to likelihood_logp is a good candidate (even though in metrop_kernel there is likely more than one step, so it is more expensive and worth parallelizing).
David.
@aloctavodia any idea?
Theano already does some magic to compute the likelihood, so I guess it will only work for very expensive likelihoods. I am on vacation right now, but I could run some experiments when I come back.
SMC is useful especially for models which don’t take advantage of Theano, because it uses Metropolis under the hood. For expensive models, computing likelihood_logp in parallel is always a good choice, otherwise I don’t see why it was done for metrop_kernel in the first place.
To be clear I think your suggestion is a good one, sorry for not mentioning earlier.
Theano not only computes gradients it also perform certain optimizations and runs some calculations in parallel. With the current version of SMC the overhead of setting parallel calculation is low and in general using parallel=true is the best option. According to my tests only for very cheap likelihoods parallel=true is (slightly) slower than parallel=false and thus the default is to use multithreading. I am also exploring the advantages of using dask to run over a cluster.
Btw, if you want to submit a PR go ahead. It will be very welcome, if not I will try your suggestion in a week or so, after I come back from vacations.
Thanks @aloctavodia, I submitted a PR.