🎓 A1Learner
🔍

PyTorch Shuffling and Batches

Shuffling mixes the row order before an epoch starts. With shuffle=True, the DataLoader serves the same rows in a new order each epoch.

No shuffle, same order

These rows carry their own index as the target, so the serving order is easy to see. Without shuffle, the order is plain counting order.

loader = DataLoader(data, batch_size=5)
order = []
for xb, yb in loader:
    order.extend(int(v) for v in yb.tolist())
print(order)

Shuffle with a seed

A seeded generator drives the shuffle. The same seed gives the same mixed order on every run.

generator = torch.Generator().manual_seed(123)
loader = DataLoader(data, batch_size=5, shuffle=True, generator=generator)
order = []
for xb, yb in loader:
    order.extend(int(v) for v in yb.tolist())
print(order)

Drop the short last batch

Ten rows in batches of 3 leave a batch of 1. drop_last=True drops that short batch.

generator = torch.Generator().manual_seed(123)
loader = DataLoader(data, batch_size=3, shuffle=True, generator=generator, drop_last=True)
sizes = []
for xb, yb in loader:
    sizes.append(len(yb))
print(sizes)
print(sum(sizes))

Example

import torch
from torch.utils.data import TensorDataset, DataLoader

features = torch.arange(10, dtype=torch.float32).reshape(-1, 1)
targets = torch.arange(10, dtype=torch.float32)
data = TensorDataset(features, targets)

generator = torch.Generator().manual_seed(123)
loader = DataLoader(data, batch_size=3, shuffle=True, generator=generator)

for epoch in range(1, 3):
    order = []
    for xb, yb in loader:
        order.extend(int(v) for v in yb.tolist())
    print(f"Epoch {epoch}: {order}")

Key points

  • shuffle=True: the DataLoader mixes the row order at the start of each epoch
  • Generator: a seeded random source that makes a shuffle repeatable
  • drop_last=True: a short final batch is dropped, so every batch stays full
  • Use a fixed seed to make a shuffled run repeatable
  • Shuffling changes the order of the rows, never the rows themselves

In this example, each target equals its row number, so the printed lists show the serving order. The loader shuffles with a generator seeded at 123. Epoch 1 serves the rows in one mixed order. Epoch 2 serves the same rows in a different mixed order. Running the program again prints the same two orders, because the seed is fixed.

Try it Yourself

Edit the code, then press Run.

Running is simulated · Python Playground

Exercise

Why does a training loader shuffle the rows each epoch?