Hallo
I got a question…
I had my model run before, but today I got the error message.
The only thing I could possible change is that I upgraded the package…I am not sure if it could be related. Does someone have some information about this issue? Thanks a lot.
File “/home/user/Documents/PyMC3/BNN_V1.py”, line 189, in construct_nn_L1
train_trace = pm.sample(draws =5000, tune = 3000)
File “/home/user/anaconda3/lib/python3.6/site-packages/pymc3/sampling.py”, line 442, in sample
trace = _mp_sample(**sample_args)
File “/home/user/anaconda3/lib/python3.6/site-packages/pymc3/sampling.py”, line 982, in _mp_sample
traces = Parallel(n_jobs=cores, mmap_mode=None)(jobs)
File “/home/user/anaconda3/lib/python3.6/site-packages/joblib/parallel.py”, line 789, in call
self.retrieve()
File “/home/user/anaconda3/lib/python3.6/site-packages/joblib/parallel.py”, line 699, in retrieve
self._output.extend(job.get(timeout=self.timeout))
File “/home/user/anaconda3/lib/python3.6/multiprocessing/pool.py”, line 644, in get
raise self._value
File “/home/user/anaconda3/lib/python3.6/multiprocessing/pool.py”, line 424, in _handle_tasks
put(task)
File “/home/user/anaconda3/lib/python3.6/site-packages/joblib/pool.py”, line 371, in send
CustomizablePickler(buffer, self._reducers).dump(obj)
TypeError: can’t pickle fortran objects
Hallo,
I got this error again even though yesterday it still worked. I already got up-to-date joblib and I also tried with core = 1 but the error msg still showed up.
The error is not related to njobs - you are saving the model and trace using pickle right? Not sure why this error occurs but could you try pickle them separately?
what about saving the trace and model separately like:
with open('model.pkl', 'wb') as buff1:
pickle.dump({‘model’: neural_network}, buff1)
with open('trace.pkl', 'wb') as buff2:
pickle.dump({‘trace’: train_trace}, buff2)
Just trying to understand whether the error is from pickling the model or the trace.
Hm…I still got the same error message… but it’s in the trace part.
Is it possible that I changed something in the system when trying doing something else with I/O under python? Or is it possible that I can just make it back to previous status?
This is quite strange as there should be any fortran objects in the trace… and I have never see this error myself before so I am not quite sure how to solve it. Maybe a workaround is to set the trace backend to HDF5 and save the trace directly to the disk.
The model is secretly stored in the trace object, but I am not sure why. You can access it with, for example, trace._straces[0].model. I’ll add this to my list of things to look at when I have more time…
Thanks a lot for all the help.
After trying deleting the theano cache (.theano folder under home directory) and reinstalling pymc3 from master, it still didn’t work…But the same code worked in my another laptop. So I just reinstalled the whole packages including python. And it works now. I guess I changed something behind when I was trying some code related to I/O, but didn’t notice that.
yes, I am using WinOS. It seems like there are much problems with win OS compare to other OS.
Hopefully we can find a solution to allow it run in WinOS.
I only notice this issue (core=1) problem when I try to solve gaussian problems.
I did not encounter multiple cores issue when I solve other distribution problems.
Seconding this as another WinOS user having issues with the pymc3 install, even after creating a new environment and following the instructions on Git step-by-step.
Seems like WinOS users have trouble 1. using multiprocessing and 2. linking Theano with the proper BLAS setup (e.g. defaults to Numpy C-API implementation)
A separate but related thread that sites the same “can’t pickle fortran objects” issue links to a website explaining how users can install Ubuntu in their WinOS environment and use pymc3 that way.
That might work for some people, but ideally a fix for WinOS would be better.
Hello everyone, Is there any update on this?
I am facing a similar issue to @brianleblanc and @GMCobraz_T while trying to fit a Gaussian Process with core>1 in an anaconda environment.