🎓 A1Learner
🔍

Python Multiprocessing

A process is a separate running program with its own memory. Python can run several processes at once.

Start one process

Process works like Thread. You give it a target and call start().

from multiprocessing import Process

def greet():
    print("Hello from a process")

if __name__ == "__main__":
    p = Process(target=greet)
    p.start()
    p.join()

Memory stays separate

The child changes its own copy of value. The parent copy stays at 10.

from multiprocessing import Process

value = 10

def change():
    global value
    value = 99
    print("Child sees:", value)

if __name__ == "__main__":
    p = Process(target=change)
    p.start()
    p.join()
    print("Parent sees:", value)

Pool maps work in order

Pool.map sends each number to a worker process. It collects results in input order.

from multiprocessing import Pool

def square(n):
    return n * n

if __name__ == "__main__":
    with Pool(2) as pool:
        results = pool.map(square, [1, 2, 3])
    print(results)

The main guard

The main guard is the line if __name__ == "__main__":. Code under it runs only when you run the file directly.

if __name__ == "__main__":
    print("Started directly")

Example

from multiprocessing import Pool

def square(n):
    return n * n

if __name__ == "__main__":
    numbers = [1, 2, 3, 4, 5, 6]
    with Pool(2) as pool:
        results = pool.map(square, numbers)
    print("Numbers:", numbers)
    print("Squares:", results)

Key points

  • Process: a separate running program with its own memory.
  • Use Process to start one extra process.
  • It mirrors Thread from the threading lessons.
  • Use Pool to share CPU work across a small team of processes.
  • Use Pool.map to apply a function to each item in a list.
  • Pool.map returns results in the input order.
  • Pickle: Python turns a value into bytes so it can travel between processes.
  • Arguments and results between processes must pickle.

In this example, square is a small function that returns n times n. The main guard holds all the start-up code. Pool(2) creates two worker processes. The map call sends each number to a worker. Each result travels back by pickle. The results list keeps the input order. The program prints the numbers and then the squares.

Note

Without the main guard, a child process can import your file and start the work again.

Try it Yourself

Edit the code, then press Run.

Running is simulated · Python Playground

Exercise

Why must multiprocessing code sit under the if __name__ == "__main__": guard?