Hi,
I am not sure I understand the question nor the goal you had in mind. Please correct me if I’m wrong as I’ll assume answers to some of these questions and these assumption may be completely wrong.
I have the feeling you mean root mean square error (or 1-rmse or some related quantity) when you mean accuracy, and that you expect az.loo to use this quantity. However, this is not the case. I’d recommend reading section 2 of “Understanding predictive information criteria for Bayesian models” by Gelman et al.
Regarding RMSE and AROC, az.loo does not work with them, and in addition, I think that PSIS approximation cannot be used in order to estimate the result of cross-validation.