The new minibatch interface

I am currently using the new minibatch interface and I am wandering what a typical example of using update_shared_f would be. As far as I understand it, it’s a function that loads an intermediate minibatch to the GPU memory, supposed to be used for a data set that does not fit entirely into GPU memory. Is that a correct interpretation?

If yes, there is no scheduler for swapping this intermediate batch included in Minibatch and the swapping has to be done by hand by calling update_shared. How would that work in practice? Would I have to stop the training and re-start it with new intermediate minibatch? Or should I use custom callbacks for that? It seems to be partially explained in the docstring but the provided code is far from a working example.

You can call this update from callbacks having iteration number

And Yes, this is for large datasets

A follow up question: what is the use case for in_memory_size?

This takes a slice of data before transfer to GPU. Use case is when you can’t hold all dataset there.

But how does it relate to batch_size? Does batch_size have to be smaller then in_memory_size? If batch_size=10 and in_memory_size=100 how often is the GPU memory swapped?

This works in the following way:

  1. store data on GPU in_memory_size is related
  2. create random slice form stored data of size batch_size
  3. return random slice

GPU storage avoids from/to cpu io. Batch size is not necessary smaller, as it utilizes slices but it for sure is not what you often need