PyTorch Padding and Stride

Padding adds a border of zeros around an image before convolution. Stride is the step size the kernel takes between stops.

A zero border

F.pad grows a tiny image with a ring of zeros. The real pixels stay in the middle.

import torch.nn.functional as F

x = torch.ones(1, 1, 2, 2)
padded = F.pad(x, (1, 1, 1, 1))
print(padded.shape)
print(padded[0, 0])

Padding keeps the size

Padding 1 lets the kernel centre on every border pixel of a 6 by 6 image. The output stays 6 by 6.

conv = nn.Conv2d(1, 1, kernel_size=3, padding=1)
print(conv(torch.rand(1, 1, 6, 6)).shape)

Stride shrinks the map

Stride 2 moves the kernel two cells at a time. The same image now gives a 2 by 2 output.

conv = nn.Conv2d(1, 1, kernel_size=3, stride=2)
print(conv(torch.rand(1, 1, 6, 6)).shape)

Example

import torch
from torch import nn

torch.manual_seed(0)

image = torch.rand(1, 1, 6, 6)

cases = [
    ("padding 0, stride 1", 0, 1),
    ("padding 1, stride 1", 1, 1),
    ("padding 0, stride 2", 0, 2),
    ("padding 1, stride 2", 1, 2),
]

for name, padding, stride in cases:
    conv = nn.Conv2d(1, 1, kernel_size=3, padding=padding, stride=stride)
    out = conv(image)
    print(name, "->", tuple(out.shape))

Key points

Padding: a border of zeros that lets the kernel centre on edge pixels
Use padding=1 with a 3 by 3 kernel to keep the output size
Stride: how many cells the kernel moves at each stop
A larger stride gives a smaller output map
Padding guards the border, and stride sets the step between stops

In this example, the seed is fixed and one random 6 by 6 image feeds four Conv2d layers. Each layer uses a 3 by 3 kernel, and only padding and stride change. With no padding and stride 1, the output shrinks to 4 by 4. Padding 1 restores the full 6 by 6 size. Stride 2 cuts the stops down, giving 2 by 2 without padding and 3 by 3 with it. The four printed shapes show both controls at work.

Note

nn.Conv2d pads with zeros, and its default stride is 1

Try it Yourself

Edit the code, then press Run.

Running is simulated · Python Playground

import torch
from torch import nn

torch.manual_seed(0)

image = torch.rand(1, 1, 6, 6)

cases = [
    ("padding 0, stride 1", 0, 1),
    ("padding 1, stride 1", 1, 1),
    ("padding 0, stride 2", 0, 2),
    ("padding 1, stride 2", 1, 2),
]

for name, padding, stride in cases:
    conv = nn.Conv2d(1, 1, kernel_size=3, padding=padding, stride=stride)
    out = conv(image)
    print(name, "->", tuple(out.shape))

Exercise

What does stride=2 do to a conv layer?

It moves the kernel two cells at each stop It adds two rows of padding It doubles the number of kernels It makes each kernel two cells wide