Meaning, if I don’t know whether my model is working the way I want it to, and I use it to generate synthetic data, then I don’t really understand that data any more than I understand my model. If I then fit my model to that data and it “works”, it’s unclear what exactly I have demonstrated/accomplished. In contrast, if I can code up exactly what I am thinking in numpy, etc. then I can be more confident that my data accurately reflected the generative process I wish to model. Building that same logic into a PyMC model (or trying to) can then evaluated by fitting the (maybe correct, maybe incorrect) model to the data (from the known generative process). If it fits well and recovers the parameter true/known values, that gives you some indication that your PyMC model reflects what you intended it to.
2 Likes