I’m trying to implement a logistic regression model with One vs Rest(OvR) stratergy for multi-class classificaiton using PyMC3. I found the softmax regression code from this git-hub issue thread and tried to change it to a OvR model. This is my code.
with pm.Model() as self.logistic_model: #feature count is X.shape alpha = pm.Normal('alpha', mu=0, sd=20, shape=class_count) beta = pm.Normal('beta', mu=0, sd=20, shape=(feature_count, class_count)) #link the RV with target variable (y) mu = alpha[y] + tt.dot(X, beta[y]) # sigmoid function instead of softmax p = pm.Deterministic('p', tt.nnet.sigmoid(mu)) y = pm.Categorical('y', p=p, observed=y) trace = pm.sample(1000, step=pm.Metropolis())
As shown by the hierarchical linear regression model tutorial, I tried to link the target variable (y) and random variables. But, after that change the dot product between X and beta fails due to shape mismatch.
I can’t figure out a way to sample separate random variable from the random matrices for the each class. Any help, or suggestion will be appreciated.