Sparse Gradient

Questions
3

Hi @ckrapu, thank you very much for taking a look at this.

I saw the Github issue but felt like mine is slightly different. For me the error above is triggered even if I explicitly set theano.config.compute_test_value = 'off' and I profiled the memory use of the script for N = 10_000 to see that the memory apparently remains occupied throughout the MAP optimization:

image
image1692×1000 93.3 KB

My version are:
Python: 3.9
Scipy 1.8.0
Theano: 1.1.2
PyMC: 3.11.4
Ubuntu: 18.04.5 LTS
and I removed my ~/.theanorc.

However, I did try the script on a 2019 MacBook Pro now and indeed, the script finished successfully. The peak memory usage for N=10_000 was only 561.73 MiB and for N=1_000_000 it was only 1234.56 MiB. Additionally, looking at the traceback above shows that config.compute_test_value is evaluated

File ~/.conda/envs/pymc3_nv1/lib/python3.9/site-packages/theano/graph/op.py:253, in Op.__call__(self, *inputs, **kwargs)
    250 node = self.make_node(*inputs, **kwargs)
    252 if config.compute_test_value != "off":
--> 253     compute_test_value(node)
    255 if self.default_output is not None:
    256     rval = node.outputs[self.default_output]

and plugging in some debugging code reveals that config.compute_test_value == 'raise' at this time.

Why does the script use so much more memory on linux?