When iterating over a devicedataloader, are all the batches moved every time I iterate

def __iter__(self):
    """Yield a batch of data after moving it to device"""
    for b in self.dl: 
        yield to_device(b, self.device)

In this function, it seems to me that everytime we iterate over the dataset, we move the batches to the device. Isn’t that inefficient? And isn’t calling the to_device function enough?

Hey @kukuquack, we are moving each batch of the data loader to the GPU at a time so that we don’t go out of memory while moving the data to GPU. I don’t think this is inefficient, as we are breaking the function of moving the data to the GPU in parts instead of moving it as a whole.

But If I iterate over the dataloader more than once, will each time the data be transferred?

Because it pushes the data to the gpu every time we iterate.