🎓 A1Learner
🔍

PyTorch Training a Regressor

Training a regressor fits a model that predicts a number for each input. The loss falls during training and settles near the noise floor.

Noisy line data

The targets follow 3x - 2, plus noise. Seed 0 is set before the noise is drawn, so the data is repeatable.

torch.manual_seed(0)
x = torch.linspace(-2, 2, 40).reshape(40, 1)
noise = 0.3 * torch.randn(40, 1)
y = 3.0 * x - 2.0 + noise
print("Input shape:", x.shape)
print("First target:", round(y[0].item(), 2))
print("Last target:", round(y[-1].item(), 2))

The training loop

A single nn.Linear layer is the whole model. The loop is the five lines from the last lesson.

for step in range(300):
    pred = model(x)
    loss = loss_fn(pred, y)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

Read the learned line

The weight and bias are the learned line. Printing them reads the fit directly.

print("Learned weight:", round(model.weight.item(), 2))
print("Learned bias:", round(model.bias.item(), 2))

Example

import torch
from torch import nn

torch.manual_seed(0)
x = torch.linspace(-2, 2, 40).reshape(40, 1)
noise = 0.3 * torch.randn(40, 1)
y = 3.0 * x - 2.0 + noise

torch.manual_seed(0)
model = nn.Linear(1, 1)
loss_fn = nn.MSELoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.05)

for step in range(301):
    pred = model(x)
    loss = loss_fn(pred, y)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    if step % 100 == 0:
        print("Step", step, "loss:", round(loss.item(), 4))

print("Learned weight:", round(model.weight.item(), 2))
print("Learned bias:", round(model.bias.item(), 2))
print("True weight: 3.0")
print("True bias: -2.0")

Key points

  • Regressor: a model that predicts a number
  • Noise floor: the lowest loss the random noise in the data allows
  • Full batch: every step trains on all rows at once
  • Use torch.linspace to spread inputs evenly over a range
  • Use torch.randn to add small random noise to exact targets
  • Training fits the line under the points, not the noise on them
  • A learned weight near the true weight shows the fit found the pattern

In this example, seed 0 is set before the data is made. The inputs are 40 points spread from -2 to 2. The targets follow 3x - 2 plus noise of size 0.3. Seed 0 is set again before the model is built. The model is one Linear layer with Adam at rate 0.05. The loop runs 301 steps on the full batch. The loss prints every 100 steps. It falls from 19.5956 to 0.0803 and then stops falling. That flat level is the noise floor. The noise size 0.3 sets a floor near 0.09. The learned weight and bias print beside the true values. Both learned numbers sit close to the truth.

Try it Yourself

Edit the code, then press Run.

Running is simulated · Python Playground

Exercise

Why does the loss stop falling at about 0.08 instead of reaching zero?