🎓 A1Learner
🔍

Rust Text Encodings

Percent-encoding rewrites each unsafe byte as % plus two hex digits, so any text can travel safely inside a link. A URL cannot hold spaces, & or accented letters directly. Base64 and hex are two more ways to carry the same bytes as text.

Encode text for a URL

Every byte outside the safe set becomes % plus two hex digits.

let encoded = percent_encoding::utf8_percent_encode("hello world", percent_encoding::NON_ALPHANUMERIC);
println!("{encoded}");

Accented letters become UTF-8 bytes

é is two bytes in UTF-8 (C3 A9), so it grows to two escapes.

let encoded = percent_encoding::utf8_percent_encode("café", percent_encoding::NON_ALPHANUMERIC);
println!("{encoded}");

Decode it back

percent_decode_str undoes the encoding, and decode_utf8 turns the bytes back into text.

let decoded = percent_encoding::percent_decode_str("caf%C3%A9").decode_utf8().unwrap();
println!("{decoded}");

Form data uses + for spaces

HTML forms write a space as +. The & still becomes an escape.

let encoded: String = url::form_urlencoded::byte_serialize("a b&c".as_bytes()).collect();
println!("{encoded}");

Base64 turns bytes into text

STANDARD is the classic Base64 alphabet. Padding = appears at the end.

use base64::{engine::general_purpose, Engine as _};

println!("{}", general_purpose::STANDARD.encode("hello"));

Decode it back to bytes

decode returns the raw bytes — turn them into text only if you know they are text.

let bytes = general_purpose::STANDARD.decode("aGVsbG8=").unwrap();
println!("{}", String::from_utf8(bytes).unwrap());

Hex writes two digits per byte

Shorter to eyeball than Base64, twice as long as the raw bytes.

println!("{}", hex::encode("Rust"));

URL-safe Base64 avoids + and /

The byte pair FB FF becomes -_8 instead of +/8=.

println!("{}", general_purpose::URL_SAFE_NO_PAD.encode([0xfb, 0xff]));

Hex and Base64 of the same word

The closing program shows both encodings of the same word. It decodes the Base64 copy back to text. It also confirms the hex copy round-trips to the original bytes.

use base64::{engine::general_purpose::STANDARD, Engine as _};

fn main() {
    let data = "Ferris".as_bytes();
    let as_hex = hex::encode(data);
    let as_b64 = STANDARD.encode(data);
    println!("hex = {as_hex}");
    println!("base64 = {as_b64}");
    let back = STANDARD.decode(&as_b64).unwrap();
    println!("back = {}", String::from_utf8(back).unwrap());
    println!("round trip ok = {}", hex::decode(&as_hex).unwrap() == data);
}

Example

fn main() {
    let mut query = url::form_urlencoded::Serializer::new(String::new());
    query.append_pair("q", "rust & c++");
    query.append_pair("page", "2");
    let qs = query.finish();
    println!("query = {qs}");
    println!("url = https://example.com/search?{qs}");
    let parsed: Vec<(String, String)> = url::form_urlencoded::parse(qs.as_bytes())
        .map(|(k, v)| (k.into_owned(), v.into_owned()))
        .collect();
    println!("q decoded = {}", parsed[0].1);
}

Key points

  • Add percent-encoding = "2" and url = "2" to Cargo.toml.
  • The first encodes and decodes raw bytes.
  • The second builds real query strings.
  • Use percent_encoding::utf8_percent_encode(text, NON_ALPHANUMERIC) to encode: it returns a value you can print, append to a String, or collect into one.
  • Use percent_encoding::percent_decode_str(text).decode_utf8() to reverse it.
  • It returns a Result, because a mangled string may not be valid UTF-8.
  • Use url::form_urlencoded::byte_serialize for form data: it follows the HTML form rule where a space becomes +, unlike %20 in plain percent-encoding.
  • Use url::form_urlencoded::Serializer to build a whole query string: append_pair("q", "...") adds one encoded key-value pair, and finish() returns the finished string.
  • In short, each unsafe byte becomes a % escape inside the text.
  • Encoding "https://example.com/search" would hide the : and / that make it a link.
  • Add hex = "0.4" and base64 = "0.22" to Cargo.toml.
  • Both encode (bytes to text) and decode (text back to bytes).
  • Use base64::engine::general_purpose::STANDARD with encode and decode.
  • The Engine trait must be in scope as (Engine as _).
  • These methods come from the trait.
  • Use general_purpose::URL_SAFE_NO_PAD when the text will travel in a URL.
  • Standard Base64's + and / are dangerous inside links.
  • The padding = is often dropped too.
  • Use hex::encode for a human-readable view of bytes.
  • It writes two hex digits per byte.
  • Use hex::decode to turn them back.
  • Both directions can fail, so decode returns a Result.
  • In short, both forms carry the same bytes as plain text.
  • Anyone can decode Base64 or hex, so they hide nothing.
  • They only make bytes safe to carry as text.

The closing program builds a real search query with two pairs. It prints the full URL. Then it parses the query back. It shows the original search text restored.

Note

Only encode the pieces of a URL, never the whole thing.

These are encodings, not encryption.

Try it Yourself

Edit the code, then press Run.

Running is simulated · Rust Playground

Exercise

Base64 and hex are called encodings, not encryption, because…