The SHAP library has nice plotting functions to visualize the contribution of various features for a particular predicted value, such as waterfall plots:
Is there a Bayesian version of these types of plots? When a stakeholder asks a question like “How much does Feature X contribute to this prediction?”, what is the proper Bayesian way to communicate feature importance & contribution?
Is the Bayesian approach to simply plot the posterior distributions? Or is there a better way of communicating feature importance?
It’s a very interesting question that I spent some time thinking about but I don’t think there is a simple/easy answer to this.
Shap values are a weighted average of lots of different conditional distributions. As such, in theory, a probabilistic modelling approach is very well suited to this. That said, I don’t think the approach taken by the shap library is compatible with PyMC or other PPLs.
You’d need to write your own algorithm for computing shap values relying on MCMC for giving you the relevant conditional expectations. Moreover the approach, in its pure, form would likely require you to give priors to every feature that you want to compute attributions for (this is because the fundamental “unit” of shap values is the difference between a model output with and without a feature; the “without a feature” part is done by marginalising over a distribution so you’d need to have that distribution)
If I am not mistaken (but I could be and would need to refer back to the literature) shap values of linear models simplify to the linear coefficients and that’s something that could be done more readily in PyMC without complex methodology. But for non-linear models that no longer holds.
