🎓 A1Learner
🔍

Rust Dedup & Retain

dedup and retain are two Vec tools that clean a list in place. dedup removes repeated items that sit side by side.

dedup only removes repeats that touch

dedup compares neighbours. A repeat that is not side by side survives:

let mut v = vec![1, 2, 2, 1];
v.dedup();
println!("{:?}", v);

Sort first to remove every duplicate

Once the repeats stand together, one dedup pass clears them all:

let mut v = vec![3, 1, 3, 2, 1];
v.sort();
v.dedup();
println!("{:?}", v);

dedup_by_key: judge by one part

Two neighbours count as repeats when a chosen part matches — here, the tens band a score falls in:

let mut scores = vec![10, 15, 21, 30, 35];
scores.dedup_by_key(|s| *s / 10);
println!("{:?}", scores);

dedup_by: your own sameness rule

The closure receives the later item first, then the one before it. Return true to drop the later one:

let mut words = vec!["Rust", "rust", "Go", "go", "Go"];
words.dedup_by(|later, earlier| later.eq_ignore_ascii_case(earlier));
println!("{:?}", words);

retain: keep only what passes the test

retain walks the Vec once and drops every item the closure rejects:

let mut v = vec![1, 2, 3, 4, 5, 6];
v.retain(|&x| x % 2 == 0);
println!("{:?}", v);

retain_mut: fix items while you filter

retain_mut hands the closure a &mut, so the test may also change the item it keeps:

let mut v = vec![1, 2, 3, 4];
v.retain_mut(|x| {
    if *x <= 3 { *x *= 10; true } else { false }
});
println!("{:?}", v);

Example

fn main() {
    let mut guests = vec!["cy", "ana", "bo", "ana", "cy", "cy", "di"];
    guests.sort();
    guests.dedup();
    println!("unique guests = {:?}", guests);
    guests.retain(|name| *name != "bo");
    println!("after dropout = {:?}", guests);
    let mut scores = vec![10, 15, 21, 30, 35];
    scores.dedup_by_key(|s| *s / 10);
    println!("one per band = {:?}", scores);
}

Key points

  • dedup removes only repeats that sit side by side — it never scans the whole list for a match
  • Plain dedup compares with ==.
  • dedup_by_key compares one chosen part of each item.
  • dedup_by compares neighbours.
  • It uses a rule of your own
  • Use retain to keep the items your closure says true to, in their original order, in one pass
  • Both tools work in place: nothing is copied into a new Vec

In this example, the guest list arrives messy: ana twice, cy three times. sort() lines the names up. They read ana, ana, bo, cy, cy, cy, di. So every repeat now touches its twin. dedup() removes them. It leaves one of each. retain then drops "bo", the guest who cancelled, and keeps the rest in order. Last, dedup_by_key thins the scores. The key is the tens band. 10 and 15 are both band 1. 30 and 35 are both band 3. So only the first score of each band survives. The survivors are 10, 21, 30.

Note

For a full duplicate purge, sort() then dedup() is the standard pair.

If the original order must be kept instead, walk the values through a HashSet (Rust HashSet) and keep first sightings.

Try it Yourself

Edit the code, then press Run.

Running is simulated · Rust Playground

Exercise

What does vec![1, 2, 2, 1] become after dedup()?