I want to use NUTS to sample part A of the model, and update the remaining part B of the model manually and deterministically. Part B contains categorical distributions (https://github.com/pymc-devs/pymc3/issues/1902).
Log-likelihood of A and log-likelihood of B share some intermediate calculations C. I plan to define custom theano operations for A, B, and C to avoid evaluating C for twice (Custom operation for likelihood and cache value of likelihood).
I will call the theano operation for log-likelihood of A as α.
I am thinking about the following:
- store the parameters of B in α
- update parameters of B within the perform method of α.
But that means α is not a pure function. The following two cases give different results:
- Evaluating α at point x first and then at point y
- Evaluating α at point y first and then at point x
Would that cause any trouble if I use NUTS?
Within each step of NUTS, how many times would NUTS call the perform method for the same point? Can I still avoid evaluating C for the same point twice?
Does NUTS use the same instance of α across all chains?