# Help to model a difficult logistic regression model with minibatches

**URL:** <https://discourse.pymc.io/t/help-to-model-a-difficult-logistic-regression-model-with-minibatches/1503>\
**Category:** Questions\
**Created:** [July 13, 2018, 8:03pm UTC](https://discourse.pymc.io/t/help-to-model-a-difficult-logistic-regression-model-with-minibatches/1503 "2018-07-13T20:03:22Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![marcodena](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/marcodena/32/41_2.png) [@marcodena](https://discourse.pymc.io/u/marcodena)\
**Post date:** [July 13, 2018, 8:03pm UTC](https://discourse.pymc.io/t/help-to-model-a-difficult-logistic-regression-model-with-minibatches/1503/1 "2018-07-13T20:03:22Z")

</div>

Dear community,  
I’m trying to model a problem that might be difficult to sample. I have a matrix filled with 0 and 1 and I have to model the relationship of each pixel with all the others, following [this paper](https://arxiv.org/abs/1209.3699).

This is the input matrix (1000 x 1000 matrix)  
 ![47](https://canada1.discourse-cdn.com/flex036/uploads/pymc3/original/1X/b96d77cb8eedc66dc36eba646917cea290f7c782.png)

This is the equation that models the pixels. I have to find gamma.

![49](https://canada1.discourse-cdn.com/flex036/uploads/pymc3/original/1X/54be011d9bf4431dc9e20aedf6232b871f98448c.png)

Since the distance matrix is too big for a PC (1000^2 x 1000^2 distance matrix) I thought it might be easier to use minibatches and compute just part of the matrix at every step.

```
M0 = np.load( open('ema/rybski_L_1024.npz', 'rb') )['arr_0'].astype('float32')
m = tt.as_tensor_variable(M0.reshape(-1, 1))
data = pm.Minibatch(M0.reshape(-1, 1), 1)
BB = pm.Minibatch(B, 1)

with pm.Model() as pooled_model:

    y = pm.Bound(pm.HalfNormal, lower=0.001, upper=6)('gamma', sd=2)
    # Euclidian distance
    d = tt.sqrt((BB **2).sum(1).reshape((BB.shape[0], 1)) + (B** 2).sum(1).reshape((1, B.shape[0])) - 2 * BB.dot(B.T))
    # Normalization to avoid overflows
    d = d/d.sum()
    
    dgamma = tt.power(d, -y)
    # Sometimes overflows and nans
    dgamma = tt.switch(tt.isnan(dgamma), 0., dgamma)
    dgamma = tt.switch(tt.isinf(dgamma), 0., dgamma)
    
    qi = eu.dot(m) / eu.sum() 

    # expected parameter
    p = pm.math.sigmoid(qi/qi.max())
    
    qi_obs = pm.Bernoulli('qi_obs', p=p, observed=data, total_size=len(M0.ravel()))

    pooled_trace = pm.sample(3000, njobs=1, tune=6000, progressbar=True, init="advi")

```

Unfortunately, this occurs to “NaN occurred in optimization.” in the newest pymc, while in the oldest it does not converge so much. Am I modeling the problem wrong? Can you help me, please?

Thank you

---

<div class="post-metadata">

**Author:** ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)\
**Post date:** [July 14, 2018, 6:43am UTC](https://discourse.pymc.io/t/help-to-model-a-difficult-logistic-regression-model-with-minibatches/1503/2 "2018-07-14T06:43:34Z")

</div>

Minibatch might not work well here as you can have some batch contain very little data, as the observed is mostly sparse. Moreover, since the formulation is summing over all pair of pixels, and minibatch would only summing a few pixels (ie pixels in that batch), wouldnt you get different model?

The formulation reminds me a lot of Mean field theory and Ising Model - would a similar decomposition works here also?

---

<div class="post-metadata">

**Author:** ![marcodena](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/marcodena/32/41_2.png) [@marcodena](https://discourse.pymc.io/u/marcodena)\
**Post date:** [July 14, 2018, 6:18pm UTC](https://discourse.pymc.io/t/help-to-model-a-difficult-logistic-regression-model-with-minibatches/1503/3 "2018-07-14T18:18:59Z")

</div>

My pymc formulation sums the batch over all pixels, so the model is still the same. However, yes, I might have problems of sparsity…

Mhh I am not familiar with these models, let me check and understand! Thank you!