Hi, Junpeng – I was just rereading this, and I realized that PyMC3 is itself confusing your two kinds of transformation, or at least I think so.
So the bounding transformation fits your model, because it transforms to an unbounded underlying RV (IIUC).
But there are also transforms like ordered which I do not believe fits your model (or does it?)
Is there a good source of information on what the transformation kwarg should and should not be used for?
well, ordered transformation should also fit with our initial usecase - mapping constrained parameter (e.g., sorted vector) to unconstrained parameter. I think what happen here is kind of a scope creep of the kwarg, which we should ideally have constrain that apply to RVs, and transformation that apply to distributions.
For example, in Stan this is more transparent that constrain is for RVs, and unrelated to the underlying distribution:
parameters {
real<lower=0> sigma;
ordered[K-1] c;
}
model {
sigma ~ Normal # <== actually a halfnormal
}
}
@junpenglao Thanks. That’s helpful. What’s still confusing me is that the Transform class has both forward and backward methods. So shouldn’t a forward sampler like sample_posterior_predictive be able to use the random method for its parent transformed variable and then apply the forward transform on the resulting value?
I think the problem is mostly a documentation one – it’s not clear what role the forward and backward methods are intended to play, based on their docstrings.
It’s even a little confusing what is the “transformed” variable and what is “the variable that was transformed.” If you are a programmer, and you say transform=sorted then it seems like the variable in your program is the transformed variable, and the upstream variable, with the canonical support is the untransformed variable. Correct? You apply the transform forward to get the sorted variable from the unsorted one.
So I am still groping to try to see why it’s hard to do forward sampling with transformed RVs. Thanks for your patience.
oh yeah i constantly confuse between forward and inverse (does not help that in TFP the meaning is reverse).
If we can distinguish this two use case of transform, it would be useful to be able to use the forward method for sample predictive
Alright, I wrote a colab to demonstrate how a separate API of 1) transform distribution, and 2) transforming bounded Variables should solve this (the example is in TFP): https://colab.research.google.com/drive/1Swagke96szpczDD7Iflf958B7CNRwjXV?usp=sharing
Hi all,
Any chance of v4 allowing forward sampling that depends on transforms? (I’m looking at ordered transform → OrderedLogistic)
No, because those don’t (usually) have a “closed form” expression with which to generate random draws directly. You can always use NUTS to do prior predictive sampling by just omitting the “observed” from the model.
Just want to document the case:
I have just trapped in all the traps described by @rpgoldman when trying to use an ordered transform and sampling from the prior predictive. It took some time of inspection to notice that it’s not applied when sampling the pp. (There being forward and backward methods in the source code added to the confusion.). I should note that I’m still on PyMC v4 and not sure if the behavior has changed in v5.
No, the behavior is the same. Transforms are still a property of logp-based sampling (mcmc) and not forward sampling (prior and posterior predictive)