Is it possible to use WAIC to calculate log predictive accuracy of pymc3 models? For a linear regression model, I defined both X and Y as shared variables. Then once the model is trained using the train dataset, changed the value of the shared_X and shared_Y with test datasets.Then calculate the WA…

Sure, WAIC is one way to estimate the expected log point-wise predictive density, which in turn asymptotically equal to LOO. Just keep in mind that predictive accuracy is in principle impossible to calculate, thus the usual quantification of predictive accuracy we have (psis-LOO, WAIC, DIC and AIC)…

Using WAIC to compute Log Predictive Accuracy

Questions

narendramukherjee May 11, 2018, 6:02pm 4

In addition, there’s a really nice discussion of DIC, AIC and WAIC in McElreath’s Statistical Rethinking book, in case you have it handy.

1 Like

Topic		Replies	Views
Unexpected Results when using WAIC to Compute Log Predictive Accuracy Questions	3	688	May 13, 2018
Calculating model comparison using log pointwise predictive density Questions	15	3598	April 5, 2018
Calculating WAIC for models with multiple likelihood functions Questions	20	4626	October 26, 2020
Can logp be positive when computing WAIC Questions	2	610	May 17, 2018
Is model with high or low WAIC and LOO better version agnostic arviz	1	1638	August 12, 2021

Using WAIC to compute Log Predictive Accuracy

Related topics