Hi @marcodena The LOO implemented in PyMC3 is actually PSIS-LOO (Pareto smoothed importance sampling-LOO) which is an approximation to the exact LOO that you are trying to compute. PSIS-LOO (and WAIC) are both approximation for estimating pointwise out-of-sample prediction accuracy. Under certain circumstances these information criteria should be proportional to the RMSE, I think the proportionality only holds for normal models.
For the comparison you want to perform i think the right approach is what you are doing, I am not sure there is a shortcut here, maybe do you want to check this paper.