Prior choice for discrete features - logistic regression

Questions version agnostic
,
4

@AlexAndorra thanks for the suggestions. I bought the book and I’ll take a look at the chapter [and, with time, the whole book]

@cluhmann, I’m note sure I understand how to use binomial regression for this application, and I can’t really think on how to implement the n trials idea in my specific problem.

In my specific case, my target variable is either 0 or 1 (benign or malicious commandline), in Binomial regression their target variable can be any number between 0-n

The model I did so far below, usually with mu_beta = 0, sigma = 1 and constant = 0 (same as in the binomial regression notebook). I have little samples and almost 2k predictors, they can be ranked into usually benign to usually malicious with help from experts.

It’s funny that it seems to classify better with mu_beta = -1 and sigma = 1 but I don’t know why.

In any case, my question was more related on a different distribution than pm.Normal for my betas. Or how should I think about my betas to find a more suitable distribution.

 with pm.Model() as model:
        X = pm.MutableData('X', X_train)
        y = pm.MutableData('y', y_train)

        beta = pm.Normal('beta', mu=mu_beta, sigma=sigma_beta, shape=len(X_train.columns))
        
        if constant:
            constant = pm.Normal('constant', mu=mu_constant, sigma=sigma_constant)
        else:
            constant = 0
            
        p = pm.Deterministic('p', pm.math.sigmoid(constant+X@beta))

        observed = pm.Bernoulli('obs', p, observed=y)
        idata = pm.sample(chains = chains)
        idata_prior = pm.sample_prior_predictive(samples=50)
    
        with model:
            pm.set_data({'X':X_test, 'y':np.zeros_like(y_test)})
            y_pred = pm.sample_posterior_predictive(idata)