I would like to use an LKJ prior that is not centered at zero. Any ideas how to achieve this?
How do you mean? The LKJ prior is a prior for correlation matrices. Could you give some more context what you’re trying to achieve?
Thanks for picking this up.
I am setting up a Hierarchical Bayesian Model and at each of its units I need to sample the correlation matrix. Since the correlation coefficients derived by the LKJ prior are always centered around zero, I cannot pass information on the center of the prior distribution of one correlation coefficient from the “global” level to every unit.
Does this description explain the situation?
I could be wrong, but I don’t think the LKJ distribution allows for anything other than controlling the strength of the regularization of correlations towards zero.
If you need to make use of this kind of information you will have to try using other prior parameterizations for your correlation matrices (e.g., Wishart). However these tend to suffer from speed or stability issues 
As far as I know, the Wishart distribution cannot to be applied in PyMC models. Are there any other options for my generating correlation matrices per unit in a Hierarchical Bayesian Model?
If you want to go into super-experimental territory, you could create a matrix from Fisher von-Mises distributed vectors
A = [v_1 | v_2 | \dots | v_k]
and set \Sigma = A^TA. Give each v_i its own hierarchical model.
Unfortunately, you’d have to implement the vMF distribution; which is of interest to me but I’ve never quite had the impetus…
Sounds like this is the most promising way forward. Since I’m fairly new to PyMC, could you please recommend a guide to how creating my own distribution could be accomplished?
In general, it’s as simple as implementing the methods logp(pt) and a random(params). However, given the support of the distribution, it’s not quite so simple. As with the Dirichlet distribution, a transform would need to be implemented from \mathbb{R}^{n} \longleftrightarrow \mathcal{S}^{n} \subsetneq \mathbb{R}^{n+1}. The Dirichlet uses the stick-breaking transform. Sterographic projection may work for the sphere; or failing that the standard spherical coordinates with r=1 (though there may be issues with backward due to inverse trig domains).
For random, you can “borrow” the implementation from libDirectional, which I believe is due to Kurtz & Hanebeck.
Thanks a lot for your advice. Even though it appears to be quite some work for a starter in PyMC, I will give this a try.