🎓 A1Learner
🔍

PyTorch Convolution by Hand

Convolution by hand slides a kernel over an image and adds up each overlap. A kernel is a small grid of weights that scores one pattern.

One overlap

Multiply each pair and add the products. This mixed patch scores zero with an edge kernel.

patch = [[1, 2, 0], [0, 1, 2], [2, 0, 1]]
kernel = [[1, 0, -1], [1, 0, -1], [1, 0, -1]]
total = sum(patch[a][b] * kernel[a][b] for a in range(3) for b in range(3))
print(total)

How many stops

A 3 by 3 kernel on a 5 by 5 image fits in three positions per side. The output grid holds nine cells.

side = 5 - 3 + 1
print(side)
print(side * side)

A clean edge

A patch that is dark on the left and bright on the right scores high. The kernel rewards exactly that left-to-right change.

patch = [[0, 0, 1], [0, 0, 1], [0, 0, 1]]
kernel = [[-1, 0, 1], [-1, 0, 1], [-1, 0, 1]]
total = sum(patch[a][b] * kernel[a][b] for a in range(3) for b in range(3))
print(total)

Example

image = [
    [0, 0, 1, 1, 1],
    [0, 0, 1, 1, 1],
    [0, 0, 1, 1, 1],
    [0, 0, 1, 1, 1],
    [0, 0, 1, 1, 1],
]

kernel = [
    [-1, 0, 1],
    [-1, 0, 1],
    [-1, 0, 1],
]

output = []
for i in range(3):
    row = []
    for j in range(3):
        total = 0
        for a in range(3):
            for b in range(3):
                total += image[i + a][j + b] * kernel[a][b]
        row.append(total)
    output.append(row)

print("Image rows:", len(image))
print("Kernel size: 3 x 3")
for row in output:
    print(row)

Key points

  • Overlap: the patch of image under the kernel at one stop
  • Use element-wise multiply and add to score one position
  • Each stop of the kernel gives one output cell
  • A 5 by 5 image with a 3 by 3 kernel gives a 3 by 3 output
  • The by-hand sum is the full operation, with no library hidden inside it

In this example, the image is five rows of zeros and ones with a bright right side. The kernel holds minus one on the left, zero in the middle, and one on the right. Four nested loops walk each stop of the kernel and add up each overlap. Stops that cross the edge score 3, because each row adds one bright cell. Stops inside the bright side score 0, because both sides match. The edge stands out as a column of high scores.

Note

PyTorch does not flip the kernel, so this sum matches nn.Conv2d exactly

Try it Yourself

Edit the code, then press Run.

Running is simulated · Python Playground

Exercise

A 6 by 6 image meets a 3 by 3 kernel with no padding and stride 1. How large is the output?