Dear Jungpen,
I had tried this before - still the outcome differed in v3.0 and v3.1. I also tried to init nuts with “nuts” and all the other options. Adaptive step size seems to be independent of initialization.
Thanks for clarifying the geometry issue again.
Dear Adrian,
your prior philosophy sounds very reasonable, iff one really conducts the prior sensitivity analysis you suggest and demonstrate in your notebook. However, it is a modeling-perspective that is fundamentally different from what “average” people are used to who were primed on frequentist statistics. What I (and reviewers potentially) want to see as a posterior is in fact only the data distribution. Hence, I initially turned to the “weakly informative prior” idea: specifying a very unspecific distribution, possibly weighting the prior low and practically only setting the “family”, ie. expected shape.
I’m still relatively inexperienced with pymc, but is there a way to weigh the prior distribution? Is that even necessary with mcmc sampling? (the “nu” parameter in R’s MCMCglmm package implies it is, yet that toolbox is completely weird and uninstructive in its prior definition and I would not trust it).
Both prior sensitivity analysis and weakly informative priors are valid and complementary approaches. I think the first is less common because, I have the impression, it is less well documented and understood. Maybe you can provide some tutorial on it? I did not search if there is one, but your workbook seems an excellent basis!
The discussion also has to do with human perception bias and good scientific philosophy. Take my four example animals: we observed that, at least in some cases, individuals would obviously run faster. If we had fed this expectancy to the model, we might have influenced the outcome. It turns that non-hierarchical approaches (we actually tested your OLS suggestion before) confirm this expectation, which I would now judge as a false positive. Those models are just wrong. The character we observe (locomotor speed) is inherently hierarchical: Individuals’ locomotor capacities are most appropriately described as a sample from a population distribution of capacities; whereas the measured speed on each run is then drawn from an individual distribution in a replicative manner. Taking the appropriate hierarchical model of reality allows you to visualize the probabilities on all levels (cf. Iknayan 2014), which is an enormous leap forward compared to box plots and hypothesis tests. Considering what Thomas described here (robust regression), bayesian models which are hierarchical and have an accurate “fat tail” prior are just excellent for many applications in Biology (the “science of exceptions”).
Best,
Falk
Iknayan, K. J., Tingley, M. W., Furnas, B. J., and Beissinger, S. R. (2014). Detecting diversity: emerging methods to estimate species diversity. Trends in ecology & evolution, 29(2):97–106.