🎓 A1Learner
🔍

Rust Deadlines & Retries

A deadline is a moment by which a whole job must finish. A chained job shares one deadline across all its steps.

A timeout answers Ok or Err

timeout races the future against the clock, and the clock wins here:

let result = timeout(Duration::from_millis(10), sleep(Duration::from_secs(1))).await;
println!("finished in time = {}", result.is_ok());

One deadline, many steps

Both steps answer to the same Instant. Step one fits inside the 30 ms budget. Step two is still running when the deadline passes. So its timeout fires:

let deadline = Instant::now() + Duration::from_millis(30);
let step1 = timeout_at(deadline, sleep(Duration::from_millis(10))).await;
println!("step1 in time = {}", step1.is_ok());
let step2 = timeout_at(deadline, sleep(Duration::from_secs(5))).await;
println!("step2 in time = {}", step2.is_ok());

Wait until a moment, not for a span

sleep_until sleeps up to an Instant — the waiting half of a deadline:

let deadline = Instant::now() + Duration::from_millis(20);
sleep_until(deadline).await;
println!("deadline reached = {}", Instant::now() >= deadline);

The retry loop shape

A counter for the attempts, a wait that doubles after each failure, and a hard stop. The closing program runs this exact shape:

let mut wait = Duration::from_millis(1);
loop {
    match timeout(Duration::from_millis(50), flaky_download()).await {
        Ok(Ok(text)) => { /* success: use text, break */ }
        Ok(Err(e)) => { /* failed fast: maybe retry */ }
        Err(_) => { /* timed out: maybe retry */ }
    }
    sleep(wait).await;
    wait *= 2; // backoff
}

Example

// Cargo.toml: tokio = { version = "1", features = ["full"] }
use std::sync::atomic::{AtomicUsize, Ordering};
use tokio::time::{sleep, timeout, Duration};

static ATTEMPTS: AtomicUsize = AtomicUsize::new(0);

async fn flaky_download() -> Result<String, String> {
    let n = ATTEMPTS.fetch_add(1, Ordering::SeqCst) + 1;
    if n < 3 {
        Err(format!("network glitch #{n}"))
    } else {
        Ok(String::from("file contents"))
    }
}

#[tokio::main]
async fn main() {
    let mut wait = Duration::from_millis(1);
    let mut attempt = 1;
    loop {
        match timeout(Duration::from_millis(50), flaky_download()).await {
            Ok(Ok(text)) => {
                println!("attempt {attempt}: success - {text}");
                break;
            }
            Ok(Err(e)) => println!("attempt {attempt}: failed ({e})"),
            Err(_) => println!("attempt {attempt}: timed out"),
        }
        if attempt == 5 {
            println!("gave up after {attempt} attempts");
            break;
        }
        sleep(wait).await;
        wait *= 2; // backoff: each wait doubles
        attempt += 1;
    }
}

Key points

  • Give every network call both a timeout and a retry budget.
  • An unbounded wait leaks a task.
  • An unbounded retry leaks patience
  • Retry only work that may safely run twice — reads and repeatable requests.

In this example, flaky_download pretends to be an unreliable network. A shared counter makes it fail on its first two calls. It makes it succeed on the third. The retry loop gives each attempt a 50 ms timeout. Attempt 1 fails fast with glitch #1, so the loop sleeps 1 ms and tries again. Attempt 2 fails with glitch #2. The wait doubles to 2 ms. Attempt 3 succeeds, prints the file contents, and breaks out. If all five attempts had failed, the loop would have printed its "gave up" line instead of retrying forever. The timeout arm never fires here. The failures are instant. But it stands guard over a download that hangs instead of failing.

Note

Never blindly retry work that charges money, sends messages, or appends records.

Try it Yourself

Edit the code, then press Run.

Running is simulated · Rust Playground

Exercise

A job has a 200 ms budget for three network steps in a row. What should each step get?