Specify reference group in softmax regression

Questions v5
1

Hello,

Question

In softmax regression sometimes it is usefull to use a reference group and report the odds-ratios of the other groups in relation to the reference group for a given predictor. How can I specify reference group in PYMC v5.15.0?

Current implementation

Currently I’m using the following code, where for each predictor I’m getting as many coefficients as groups/categories.

Nfeatures = features.size # 115 features
Nclasses = np.unique(yObserved).size # 3 groups recoded as 0,1,2

coords = {
        'features': xObservedScaled.columns.tolist(),
        'observations': xObservedScaled.index.tolist()
        }
with pm.Model() as model:
    for k in coords.keys():
        model.add_coord(k, coords[k], mutable=True)
    alpha = pm.Normal('alpha', mu=0, sigma=3, shape=Nclasses)
    beta = pm.Laplace('beta', mu=0, b=2, shape=(Nfeatures,Nclasses), dims=('features', 'classes'))
    X = pm.MutableData("X", xObservedScaled, dims=('observations', 'features'))
    mu = alpha + pm.math.dot(X, beta)
    theta = pm.Deterministic('theta', pt.special.softmax(mu, axis=1))
    yhat = pm.Categorical('yhat', p=theta, observed=yObserved, dims=('observations'))

The above code returns coefficients for each predictor in the form:

[xi, 0]
[xi, 1]
[xi, 2]

Since there is no reference group, I assume the coefficient should be interpreted as the increase/decrease in the log(odds ratio) of being in a given group, in relation of being in any other group, for every unit change of the “xi” predictor.

How can I specify reference group in the code above?
Thank you in advance.

2

Usually you hardcode the intercepts of the reference group to zero.

There’s also alternative like using the ZeroSumNormal for the intercept priors that don’t require picking an arbitrary reference group