Go Rate Limiting

Rate limiting caps how many jobs run at the same time. A channel full of tokens sets the cap.

Take a token, run, return it

The channel starts with two tokens. One job takes a token, runs, and puts the token back.

package main

import "fmt"

func main() {
	tokens := make(chan struct{}, 2)
	tokens <- struct{}{}
	tokens <- struct{}{}
	fmt.Println("Tokens before:", len(tokens))
	<-tokens
	fmt.Println("Job ran")
	fmt.Println("Tokens during:", len(tokens))
	tokens <- struct{}{}
	fmt.Println("Tokens after:", len(tokens))
}

Four jobs, two tokens

Four jobs share two tokens. Each job takes a token, writes its slot, and returns the token.

package main

import (
	"fmt"
	"sync"
)

func main() {
	tokens := make(chan struct{}, 2)
	tokens <- struct{}{}
	tokens <- struct{}{}
	results := make([]int, 4)
	var wg sync.WaitGroup
	for i := 0; i < 4; i++ {
		wg.Add(1)
		go func(id int) {
			defer wg.Done()
			<-tokens
			results[id] = (id + 1) * 10
			tokens <- struct{}{}
		}(i)
	}
	wg.Wait()
	for i, r := range results {
		fmt.Printf("Result %d: %d\n", i, r)
	}
}

The burst uses both tokens at once

The first two jobs take both tokens and hold them. The third job waits until a token comes back.

package main

import (
	"fmt"
	"sync"
)

func main() {
	tokens := make(chan struct{}, 2)
	tokens <- struct{}{}
	tokens <- struct{}{}
	results := make([]int, 3)
	started := make(chan int, 3)
	hold := make(chan struct{})
	var wg sync.WaitGroup
	run := func(id int) {
		defer wg.Done()
		<-tokens
		results[id] = (id + 1) * 10
		started <- id
		<-hold
		tokens <- struct{}{}
	}
	wg.Add(1)
	go run(0)
	<-started
	wg.Add(1)
	go run(1)
	<-started
	fmt.Println("Started first:", results[0], results[1])
	fmt.Println("Tokens left:", len(tokens))
	wg.Add(1)
	go run(2)
	fmt.Println("Job 2 waits for a token")
	close(hold)
	wg.Wait()
	fmt.Println("Results:", results[0], results[1], results[2])
	fmt.Println("Tokens back:", len(tokens))
}

Example

package main

import (
	"fmt"
	"sync"
)

func main() {
	tokens := make(chan struct{}, 2)
	tokens <- struct{}{}
	tokens <- struct{}{}
	results := make([]int, 5)
	var wg sync.WaitGroup
	for i := 0; i < 5; i++ {
		wg.Add(1)
		go func(id int) {
			defer wg.Done()
			<-tokens
			results[id] = (id + 1) * 10
			tokens <- struct{}{}
		}(i)
	}
	wg.Wait()
	for i, r := range results {
		fmt.Printf("Result %d: %d\n", i, r)
	}
	fmt.Println("Tokens back:", len(tokens))
}

Key points

Rate limiting caps how many jobs run at once.
A token is one pass taken from a channel.
A job takes a token before it runs and returns the token after.
A channel with two tokens lets at most two jobs run at once.
Jobs write results to indexed slots, so main prints in job order.
The len function reads how many tokens sit in the channel.

In this example, five jobs share a channel that holds two tokens. A job takes a token before it writes its slot and returns the token after. Main joins all five jobs and prints the slots in job order. The last line reads the channel length and shows both tokens back.

Try it Yourself

Edit the code, then press Run.

Running is simulated · Go Playground

package main

import (
	"fmt"
	"sync"
)

func main() {
	tokens := make(chan struct{}, 2)
	tokens <- struct{}{}
	tokens <- struct{}{}
	results := make([]int, 5)
	var wg sync.WaitGroup
	for i := 0; i < 5; i++ {
		wg.Add(1)
		go func(id int) {
			defer wg.Done()
			<-tokens
			results[id] = (id + 1) * 10
			tokens <- struct{}{}
		}(i)
	}
	wg.Wait()
	for i, r := range results {
		fmt.Printf("Result %d: %d\n", i, r)
	}
	fmt.Println("Tokens back:", len(tokens))
}

Exercise

A token channel holds two tokens and five jobs want to run. How many jobs can hold a token at the same time?

Five Two Three None