How to only save some of the variables while sampling?

Questions v5
1

Hi all,

Due to the memory limit and my large dataset, I want to track (save into idata) only some of the variables in my model. Is that possible while calling pm.smaple? Thanks!

Yangkang

2

You don’t need to use Deterministics or Constant/MutableData (in which case such variables will not be saved) but other model variables are always saved.

@OriolAbril may know a trick

1 Like
3

Thanks! I’m aware of these, but still wanna drop some of the intermediate variables :slight_smile:

4

I don’t know any trick, as far as I know it is not possible to do this. I can’t 100% guarantee it is not possible but I would be surprised if it is

5

I’m going to bump this. It seems like a great feature to have or trick to find. Is there any way that I can create the distribution I want in PyTensor so that the variable is sampled and the logpdf evaluated, but the value itself is not stored?

One idea we had was to get the full sampled posterior points, discard some of them, and then resume sampling at the previous final points. For this we would need to carry forward some state information (the current tuning), although I suppose we could retune if it came to that.

Would this work?
Opher

6

The original issue is solved via the new argument var_names in pm.sample. The issue of resuming sampling with a partial subset of the draws already sampled is something very different and not supported

2 Likes
7

Perfect. Thanks!

Opher