so this is where I’ve gotten with it. please note that matrix is a scipy csc_matrix
### TRYING A SPARSE MATRIX IMPLEMENTATION
print('[LOGGER]: STARTING PYMC3')
start = time.time()
#shape of betas
n_betas = matrix.get_shape()[1]
target_array = np.array(target_list)
with pm.Model() as py_lr2:
intercept = pm.Normal('c', 0, 100)
betas = pm.Normal('betas', 0, 10, shape=(n_betas, 1))
sd = pm.HalfNormal('sigma', 5.)
mu = theano.sparse.basic.dot(matrix, betas) + intercept
obs = pm.Normal('y', mu=mu, sigma=sd, observed=target_array)
trace = sample(500, cores=2,progressbar=True) # draw 500 posterior samples using NUTS
plt.figure(figsize=(7, 7))
traceplot(trace)
plt.tight_layout()
plt.show()
end = time.time()
print("[LOGGER]: FINISHED PYMC3...total run took {} minutes to run".format((end-start)/60))
This throws the following traceback …
ERROR retrieving error_storage.Was the error set in the c code? [<class 'MemoryError'>, ((222713, 222713), dtype('float64')), None]
Traceback (most recent call last):
File "rpm_modeling.py", line 177, in <module>
obs = pm.Normal('y', mu=mu, sigma=sd, observed=target_array)
File "C:\Users\cory.jez\AppData\Local\Continuum\anaconda3\envs\rpm-env\lib\site-packages\pymc3\distributions\distribution.py", line 83, in __new__
return model.Var(name, dist, data, total_size, dims=dims)
File "C:\Users\cory.jez\AppData\Local\Continuum\anaconda3\envs\rpm-env\lib\site-packages\pymc3\model.py", line 1112, in Var
var = ObservedRV(
File "C:\Users\cory.jez\AppData\Local\Continuum\anaconda3\envs\rpm-env\lib\site-packages\pymc3\model.py", line 1740, in __init__
self.logp_elemwiset = distribution.logp(data)
File "C:\Users\cory.jez\AppData\Local\Continuum\anaconda3\envs\rpm-env\lib\site-packages\pymc3\distributions\continuous.py", line 537, in logp
return bound((-tau * (value - mu)**2 + tt.log(tau / np.pi / 2.)) / 2.,
File "C:\Users\cory.jez\AppData\Local\Continuum\anaconda3\envs\rpm-env\lib\site-packages\theano\tensor\var.py", line 150, in __sub__
return theano.tensor.basic.sub(self, other)
File "C:\Users\cory.jez\AppData\Local\Continuum\anaconda3\envs\rpm-env\lib\site-packages\theano\gof\op.py", line 674, in __call__
required = thunk()
File "C:\Users\cory.jez\AppData\Local\Continuum\anaconda3\envs\rpm-env\lib\site-packages\theano\gof\op.py", line 862, in rval
thunk()
File "C:\Users\cory.jez\AppData\Local\Continuum\anaconda3\envs\rpm-env\lib\site-packages\theano\gof\cc.py", line 1731, in __call__
exc_value = exc_type(_exc_value)
TypeError: __init__() missing 1 required positional argument: 'dtype'
What I can’t tell is if this is memory related (ERROR retrieving error_storage) or something in my syntax related to the dtype argument. Any thoughts would be great, thanks.