PyTorch Metrics Beyond Accuracy
Precision and recall are scores that split correct predictions by the kind of mistake. They expose failures that a single accuracy number hides.
The accuracy trap
A model that calls every patient healthy scores well on paper. It finds nobody who is sick.
y_true = torch.tensor([1] * 10 + [0] * 90)
y_pred = torch.zeros(100, dtype=torch.long)
accuracy = (y_pred == y_true).float().mean().item()
recall = ((y_pred == 1) & (y_true == 1)).sum().item() / 10
print(f"Accuracy: {accuracy:.2f}")
print(f"Recall: {recall:.2f}")Count true positives
Two masks pick the rows where prediction and truth are both 1. The sum counts them.
y_true = torch.tensor([1, 0, 1, 1, 0])
y_pred = torch.tensor([1, 0, 0, 1, 1])
tp = ((y_pred == 1) & (y_true == 1)).sum().item()
print("True positives:", tp)Example
import torch
# 100 patients: 10 are sick (1), 90 are healthy (0)
y_true = torch.tensor([1] * 10 + [0] * 90)
# The model flags 4 patients: 2 sick and 2 healthy
y_pred = torch.tensor([1, 1] + [0] * 8 + [1, 1] + [0] * 88)
tp = ((y_pred == 1) & (y_true == 1)).sum().item()
fp = ((y_pred == 1) & (y_true == 0)).sum().item()
fn = ((y_pred == 0) & (y_true == 1)).sum().item()
tn = ((y_pred == 0) & (y_true == 0)).sum().item()
accuracy = (tp + tn) / len(y_true)
precision = tp / (tp + fp)
recall = tp / (tp + fn)
print(f"True positives: {tp}")
print(f"False positives: {fp}")
print(f"False negatives: {fn}")
print(f"True negatives: {tn}")
print(f"Accuracy: {accuracy:.2f}")
print(f"Precision: {precision:.2f}")
print(f"Recall: {recall:.2f}")
print(f"Sick patients missed: {fn} of {tp + fn}")Key points
- True positive: a sick case the model flags as sick
- False positive: a healthy case the model flags as sick
- False negative: a sick case the model calls healthy
- Precision: the share of flagged cases that are truly positive
- Recall: the share of truly positive cases the model flags
- Accuracy: the share of all predictions that are correct
In this example, the test set holds 100 patients. Ten are sick and ninety are healthy. The model flags four patients, and only two of them are sick. Four mask counts split the outcome into true positives, false positives, false negatives, and true negatives. Accuracy divides the 90 correct calls by 100 and prints 0.90. Precision divides the 2 true positives by the 4 flagged cases and prints 0.50. Recall divides the 2 found cases by the 10 sick patients and prints 0.20. The last line states the cost plainly, with 8 of the 10 sick patients missed.
Try it Yourself
Edit the code, then press Run.
Running is simulated · Python Playground
Exercise
A screening test must miss as few sick patients as possible. Which score matters most?