Posterior sample from an approximation with Minibatches?

I see. Yes I would say this is probably more a modelling problem. One thing you should be aware of is that Minibatch by design samples randomly with replacement, which means that you can have the same data point being selected more than once in one .eval() call during one training step. This is not a very big deal in some model, but for a spatial model and GP it would create problem - You can get around with it by using more informative prior (as you have much less information in one minibatch, your model is more likely to become degenerative).

In your case, the problem is shifts = pm.Normal('shifts', mu, tau=tau, shape=batchsize, total_size=nobs). Notice here that they are the same for each realization of the minibatch, which is not the same model when you are only using a subset of data.