Dealing With Missing Data

[junpenglao]

related discussion:

Large Scale Factor Analysis with minibatch ADVI

Hi, I’m new to PyMC3 and am using it in my master dissertation. I have a dataset of roughly 1.3 million datapoints, each of 20 dimensions, and 40% of the data matrix are missing. My goal is to reduce its dimensionality for further analysis. The way I deal with missing values is simple - assign a 20 dimensional Bernoulli mask, that either returns the “true” value (when entry is observed) or a placeholder (when the entry is missing). To make it run on ADVI, I have integrated away the Bernoulli la…