🎓 A1Learner
🔍

PyTorch What the Model Learned

A decision boundary is the line where a model's answer changes from one class to another. Asking the model about a grid of points draws that boundary as a map.

Build a grid of points

Two loops walk the x and y values. Each pair becomes one grid point.

grid_points = []
for y in [1.0, 0.0, -1.0]:
    for x in [-1.0, 0.0, 1.0]:
        grid_points.append([x, y])
grid = torch.tensor(grid_points)
print(grid.shape)
print(grid[0])

The model answers the grid

One forward pass answers every grid point at once. argmax turns the scores into class numbers.

with torch.no_grad():
    answers = model(grid).argmax(dim=1)

Print answers as a map

The flat answers are cut into rows of the grid's width. Each row prints as one line of characters.

answers = [0, 0, 1, 1, 0, 1]
width = 3
for row in range(2):
    line = "".join(str(answers[row * width + col])
                   for col in range(width))
    print(line)

Example

import torch
from torch import nn
from torch.utils.data import DataLoader, TensorDataset

torch.manual_seed(0)

per_class = 60
centres = [torch.tensor([-3.0, -2.0]), torch.tensor([3.0, -2.0]),
           torch.tensor([0.0, 3.0])]
blobs = [torch.randn(per_class, 2) * 0.8 + centre for centre in centres]
points = torch.cat(blobs)
labels = torch.cat([torch.full((per_class,), i, dtype=torch.long)
                     for i in range(3)])

dataset = TensorDataset(points, labels)
loader = DataLoader(dataset, batch_size=32, shuffle=True)

torch.manual_seed(1)
model = nn.Sequential(
    nn.Linear(2, 16),
    nn.ReLU(),
    nn.Linear(16, 3),
)
loss_fn = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.05)

for epoch in range(30):
    for batch_points, batch_labels in loader:
        scores = model(batch_points)
        loss = loss_fn(scores, batch_labels)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

model.eval()
width, height = 30, 15
grid_points = []
for row in range(height):
    y = 4.5 - row * (9.0 / (height - 1))
    for col in range(width):
        x = -5.5 + col * (11.0 / (width - 1))
        grid_points.append([x, y])
grid_tensor = torch.tensor(grid_points)

with torch.no_grad():
    answers = model(grid_tensor).argmax(dim=1).tolist()

print("What the model predicts at each grid point:")
for row in range(height):
    line = "".join(str(answers[row * width + col]) for col in range(width))
    print(line)

Key points

  • Use a grid of evenly spaced points to cover the whole data area
  • Use the trained model under no_grad to answer every grid point
  • Each map cell prints the class predicted at that spot
  • The boundary is the dividing line the model itself learned

In this example, the three-blob classifier trains exactly as in the earlier lesson. Then a grid of 30 by 15 points is built across the data area. The top row of the grid holds the highest y values. The trained model answers all 450 grid points in one pass under no_grad. Each answer prints as one character, row by row. Class 2 fills the top of the map. Class 0 holds the bottom left, and class 1 holds the bottom right. The edges between the regions are the decision boundaries.

Note

The map needs no plotting library, because characters carry the picture

Try it Yourself

Edit the code, then press Run.

Running is simulated · Python Playground

Exercise

What does a character in the boundary map stand for?