🎓 A1Learner
🔍

PyTorch In-Place Operations

In-place operations change a tensor itself instead of returning a new tensor. Their names end with a trailing underscore, like add_ and mul_.

The normal way copies

An ordinary operation returns a new tensor. The original stays unchanged.

prices = torch.tensor([100.0, 50.0, 20.0])
print(prices + 10.0)
print(prices)

add_ changes the tensor

The underscore version writes into the same tensor.

prices.add_(10.0)
print(prices)

Two names see the change

A second name for the same tensor sees in-place edits too.

same = prices
prices.mul_(2.0)
print(same)

Example

import torch

prices = torch.tensor([100.0, 50.0, 20.0])
sale_copy = prices * 0.9
print("Start:", prices)
print("Copy:", sale_copy)
prices.mul_(0.9)
print("After in place:", prices)

Key points

  • In-place: the result is written into the same tensor, not a new one
  • A trailing underscore marks an in-place operation, like add_ or mul_
  • Use add_(x) to add x to every cell of the tensor itself
  • Every name that points at the tensor sees an in-place change

In this example, prices starts as three prices. Multiplying by 0.9 the normal way makes a new tensor, sale_copy, and prices stays at the start values. Then mul_ applies the same discount in place, so prices itself holds the sale prices. The last line prints the changed original.

Note

In-place edits can upset autograd, which tracks operations to work out gradients later in this course.

Try it Yourself

Edit the code, then press Run.

Running is simulated · Python Playground

Exercise

prices holds three prices. What does prices.add_(5.0) do?