For some reason, I’m getting memory errors when running the following code on a simple regression model: posterior_samples_regression = pm.sample_posterior_predictive(trace, model = model, samples=500) This has worked in previous models. The full error code and model are below. Error Code: 1…

What are the shapes of train_X and train_y?

It was (40384, 54). I ran it with 100 samples and it worked. Then I ran it with 500 samples again and it worked after failing a few times. A bug maybe?

Ok, this means that the output of the posterior predictive should be a (500, 40384) array of float64. The current implementation does not preallocate the memory for that, makes a list and then converts it to an array, so it would take up roughly twice that size. That would explain the intermittency.…

The full array should just take up 25Gb, so it should fit. It doesn’t because of the preallocation issue, where you end up using almost twice the memory. It should be easy to add some preallocation logic after making the very first draw (which let’s you infer the outputs shape). Would you be interes…

I apologize but I’m not sure what a ‘PR’ is.

It is an acronym for pull request. It is a term used in git to make a contribution to the pymc3 code base. If you feel adventurous, you can try to read some tutorials online on how to use git and github. If not, I’ll try to make one when I get the chance

Would you want me to push my model code up and error?

Memory Error with posterior_predictive_sample

Questions

jordan.howell2 March 10, 2019, 11:46pm 4

Sorry. That was X-test. X-train was (127534, 54). Y-train was (127534,1)

Topic		Replies	Views
Pm.sample_posterior_predictive hanging on certain architecture v5	3	73	October 30, 2024
MemeoryError when sampling Questions	2	559	July 12, 2018
Memory issues with creating simple regression model Questions	4	1975	June 17, 2019
Understanding shape of values returned by sample_posterior_predictive Questions	3	950	January 26, 2020
Load_trace and sample_posterior_predictive do not work Questions	1	610	September 2, 2019

Memory Error with posterior_predictive_sample

Related topics