I cant really run your code without gridding, maybe you can save the output matrix so others can reproduce your notebook easier.
On a highlevel, it is indeed likely related to sparse matrix. I remembered we discussed it here on this discourse a while ago - not sure what is the solution at the time.