Hierarchical Neural Network with Minibatch ADVI

Questions
#1

I have been playing around with Hierarchical Neural Networks as discussed in this blog post

In the post, he samples using NUTS, which is proving to not really be tractable for my dataset, so I’d like to try using Minibatch ADVI to compare performance. However, I get a shape error when I create the activations.

I can tell the problem is because Minibatch is Minibatching (indexing? Doesn’t matter) the wrong dimension. For example, my data is shape (N_GROUPS, N_DATA_IN_GROUP, N_FEATURES) as in the blog post. When I use Minibatch, it ends up as (BATCH_SIZE, N_DATA_IN_GROUP, N_FEATURES), which causes the shape error. Is there a way to specify to Minibatch which dimension it should slice from? Or, is there some other way around this problem?

Cheers!

#2

Okay, I think I answered my own question.

For anyone else having a similar issue, here’s what I did:

pm.Minibatch(x, [None, BATCH_SIZE])

instead of:

pm.Minibatch(x, BATCH_SIZE)

will draw batches from the second dimension instead of the first. Hopefully this will help someone else

1 Like