Hi @jessegrabowski!
Thanks for your guidance. I’ve followed up with with some investigation into the posterior predictive. I’d be curious if you have any further suggestions.
Visualizing the ppc it does look like all the chains are giving consistent results.
But it’s hard to get a sense of how they are making predictions for individual waves here so I tried visualizing a subset of the predictions for some individual waves:
The columns represent three different waves and the rows represent the different chains. These actually look pretty fine to me by eye. But something that I noticed is in general the model seems to be finding most of the true frequencies that I initially generated. There are a small number of exceptions:
This is a scatter plot of the mean predicted frequency (across all chains) against the true frequency for all the waves. I’ve marked a few in orange that are particularly bad, and I looked at the posterior predictive for those:
So these clearly are doing quite a bit worse. But interestingly for the slower oscillating waves it does seem like one of the chains is getting it right. It’s not obvious to me how the other solutions are equiprobable. But this is zooming in on particular problem cases and not accounting for the more global fit.
Does this provide any additional information that is helpful troubleshooting? Thanks again for your input.