An error message about 'can't pickle fortran objects'


#1

Hallo
I got a question…
I had my model run before, but today I got the error message.
The only thing I could possible change is that I upgraded the package…I am not sure if it could be related. Does someone have some information about this issue? Thanks a lot.

File “/home/user/Documents/PyMC3/BNN_V1.py”, line 189, in construct_nn_L1
train_trace = pm.sample(draws =5000, tune = 3000)
File “/home/user/anaconda3/lib/python3.6/site-packages/pymc3/sampling.py”, line 442, in sample
trace = _mp_sample(**sample_args)
File “/home/user/anaconda3/lib/python3.6/site-packages/pymc3/sampling.py”, line 982, in _mp_sample
traces = Parallel(n_jobs=cores, mmap_mode=None)(jobs)
File “/home/user/anaconda3/lib/python3.6/site-packages/joblib/parallel.py”, line 789, in call
self.retrieve()
File “/home/user/anaconda3/lib/python3.6/site-packages/joblib/parallel.py”, line 699, in retrieve
self._output.extend(job.get(timeout=self.timeout))
File “/home/user/anaconda3/lib/python3.6/multiprocessing/pool.py”, line 644, in get
raise self._value
File “/home/user/anaconda3/lib/python3.6/multiprocessing/pool.py”, line 424, in _handle_tasks
put(task)
File “/home/user/anaconda3/lib/python3.6/site-packages/joblib/pool.py”, line 371, in send
CustomizablePickler(buffer, self._reducers).dump(obj)
TypeError: can’t pickle fortran objects


#2

Can you try setting cores=1 in pm.sample?


#3

Then it works. But why is that before it is not a problem ? :thinking:


#4

Then this is not related to PyMC3 - it is a joblib problem.
Try upgrading joblib, please report back if you still seeing this problem.


#5

Hallo,
I got this error again even though yesterday it still worked. I already got up-to-date joblib and I also tried with core = 1 but the error msg still showed up.

The message looks like this

Thanks a lot in advance. :slight_smile:


#6

The error is not related to njobs - you are saving the model and trace using pickle right? Not sure why this error occurs but could you try pickle them separately?


#7

yes, that’s what I tried to do.

I have my code now like this

def save_model(train_trace, neural_network):
with open ( ‘file.pkl’, ‘wb’) as buff:
pickle.dump ({‘model’: neural_network}, buff)
pickle.dump ({‘trace’: train_trace}, buff)

But I am not sure if I am doing it right…:confused:


#8

what about saving the trace and model separately like:

with open('model.pkl', 'wb') as buff1:
    pickle.dump({‘model’: neural_network}, buff1)
with open('trace.pkl', 'wb') as buff2:
    pickle.dump({‘trace’: train_trace}, buff2)

Just trying to understand whether the error is from pickling the model or the trace.


#9

Hm…I still got the same error message…:confused: but it’s in the trace part.
image

Is it possible that I changed something in the system when trying doing something else with I/O under python? Or is it possible that I can just make it back to previous status?


#10

This is quite strange as there should be any fortran objects in the trace… and I have never see this error myself before so I am not quite sure how to solve it. Maybe a workaround is to set the trace backend to HDF5 and save the trace directly to the disk.


#11

The model is secretly stored in the trace object, but I am not sure why. You can access it with, for example, trace._straces[0].model. I’ll add this to my list of things to look at when I have more time…


#12

Thanks a lot for all the help.
After trying deleting the theano cache (.theano folder under home directory) and reinstalling pymc3 from master, it still didn’t work…But the same code worked in my another laptop. So I just reinstalled the whole packages including python. And it works now. I guess I changed something behind when I was trying some code related to I/O, but didn’t notice that.