Go Worker Pools
A worker pool is a fixed group of goroutines that share one channel of jobs. Each worker takes jobs until the channel closes.
Two workers double four jobs
Each job carries the index of its input. A worker doubles the input and writes the result into the slot with the same index.
package main
import (
"fmt"
"sync"
)
func doubleWork(jobs <-chan int, inputs []int, results []int, wg *sync.WaitGroup) {
defer wg.Done()
for job := range jobs {
results[job] = inputs[job] * 2
}
}
func main() {
inputs := []int{3, 5, 7, 9}
jobs := make(chan int)
results := make([]int, 4)
var wg sync.WaitGroup
for worker := 0; worker < 2; worker++ {
wg.Add(1)
go doubleWork(jobs, inputs, results, &wg)
}
for job := 0; job < 4; job++ {
jobs <- job
}
close(jobs)
wg.Wait()
fmt.Println(results)
}Three workers square five jobs
Three workers share the same jobs channel here. Five jobs go in, and the results still land in job order.
package main
import (
"fmt"
"sync"
)
func squareWork(jobs <-chan int, inputs []int, results []int, wg *sync.WaitGroup) {
defer wg.Done()
for job := range jobs {
results[job] = inputs[job] * inputs[job]
}
}
func main() {
inputs := []int{2, 4, 6, 8, 10}
jobs := make(chan int)
results := make([]int, 5)
var wg sync.WaitGroup
for worker := 0; worker < 3; worker++ {
wg.Add(1)
go squareWork(jobs, inputs, results, &wg)
}
for job := 0; job < 5; job++ {
jobs <- job
}
close(jobs)
wg.Wait()
fmt.Println(results)
}Closing the jobs channel stops the pool
Closing the jobs channel ends each worker range loop. Wait returns after both workers finish, and the final lines print.
package main
import (
"fmt"
"sync"
)
func countWork(id int, jobs <-chan int, counts []int, wg *sync.WaitGroup) {
defer wg.Done()
for range jobs {
counts[id]++
}
}
func main() {
jobs := make(chan int)
counts := make([]int, 2)
var wg sync.WaitGroup
for worker := 0; worker < 2; worker++ {
wg.Add(1)
go countWork(worker, jobs, counts, &wg)
}
for job := 1; job <= 4; job++ {
jobs <- job * 10
}
close(jobs)
wg.Wait()
total := counts[0] + counts[1]
fmt.Println("Jobs done:", total)
fmt.Println("Pool stopped")
}Example
package main
import (
"fmt"
"sync"
)
func squareWork(jobs <-chan int, inputs []int, results []int, wg *sync.WaitGroup) {
defer wg.Done()
for job := range jobs {
results[job] = inputs[job] * inputs[job]
}
}
func main() {
inputs := []int{1, 2, 3, 4, 5, 6}
jobs := make(chan int)
results := make([]int, 6)
var wg sync.WaitGroup
for worker := 0; worker < 3; worker++ {
wg.Add(1)
go squareWork(jobs, inputs, results, &wg)
}
for job := 0; job < 6; job++ {
jobs <- job
}
close(jobs)
wg.Wait()
fmt.Println(results)
}Key points
- A worker pool shares one jobs channel among a fixed group of workers.
- A job is one unit of work sent on the jobs channel.
- Workers write each result into the slot with the job index.
- Main prints results after Wait, in slot order.
- Closing the jobs channel ends every worker after the last job.
- The pool size is the number of worker goroutines running at once.
In this example, three workers share one jobs channel. Main sends the six job indexes and then closes the channel. Each worker squares its input and writes the result into the slot with the job index. Main waits for the workers and prints the slots, so the squares appear in input order.
Note
Workers pass results back through shared slots or channels, and main does the printing.
Try it Yourself
Edit the code, then press Run.
Running is simulated · Go Playground
Exercise
How does main keep the pool results in job order?