Hm, actually, the same paper mentions that probabilistic matrix factorization usually sucks for sparse or imbalanced data, and that I should probably try Bayesian matrix factorization. So if there isn’t an obvious bug/suggestion out there, it might just be a bad model 