How do I "Score" My Model

Your ppc plot shows clear signs of overdispersion (link to a blogpost of my own). It could be due to the choice of priors, for example the prior on the scale parameter having too much of its weight on too large values or due to model miss specification.

You should not look at the number of samples but at the number of effective samples (ess). In most cases, an effective sample size of ~400 is enough, depending on the accuracy you want on you estimates (or if you want accurate estimates of tail properties) you may need more. The MCSE will give you an idea about where you are at. Both these quantities can be used to assess convergence of the MCMC. Take a look at this PyMCon talk to get both an overview on model convergence and more details on the strange words and acronyms I have just written:

The tune parameter regulates the number of warmup/tuning iterations during which the sampler parameters are tuned in order to be able to sample from the posterior efficiently and accurately. This blog post covers that in more detail with diagrams and intuitive explanations in case you want to learn more about tuning.

Here I am a bit lost about both what you are doing and about what you want to do. If you are using sample_posterior_predictive, is should internally take care of any equation you have like the y = logistic... because you have had to use it when creating and sampling the model. It is hard to see what is happening only with parts of the code.

I believe the reason behind part of the confusion could be behind these sentences, but this is a guess:

You may have set their priors to these distributions, but this does not mean that their posterior distributions are the same type of distribution with different parameters, a parameter whose prior distribution is an exponential distribution can perfectly have a normal distribution as posterior distribution. You can use plots like az.plot_posterior or az.plot_density to explore the posterior distributions visually.

1 Like