Processes vs. Threads vs. Async: How WhatsApp Held 2 Million Connections on One Server

Processes vs. Threads vs. Async: How WhatsApp Held 2 Million Connections on One Server

In 2012, WhatsApp engineer Rick Reed told an Erlang conference that the company's servers were holding more than 2 million simultaneous connections each. A test the team described reached over 2.5 million connections on a single machine, up from roughly 200,000 at the start of their tuning work, with CPU use above 85% across 24 logical cores. A few years later, with servers busier, they ran closer to 1 million connections per server across about 150 chat servers handling around 150 million peak connections. Behind it was one idea: every connected phone was handled by its own tiny Erlang process, which holds only kilobytes of state while idle.

Try that with the obvious approach, one operating system thread per connection, and the math falls apart. A Linux thread typically reserves around 8 MB of virtual address space for its stack (Java and Windows default to about 1 MB). Two million threads would reserve roughly 16 TB of address space, and that's before counting the kernel's own bookkeeping and the cost of scheduling them all. The stack space is virtual, not RAM, but the model still doesn't scale to that level.

Compare that with Go, where a new goroutine starts with a stack of about 2 KB, which is why a Go program can run hundreds of thousands of them. And Python has been changing the rules too: since Python 3.14, released in October 2025, there's an officially supported free-threaded build that runs without the Global Interpreter Lock, though it's still optional.

These three examples all answer the same question: how does a program do many things at once? The three classic answers are processes, threads, and async (event loops and lightweight tasks). They trade memory, speed, safety, and complexity in different ways, and operating systems courses often make them harder to remember than they should be. This guide explains each one with working examples, the real numbers behind the trade-offs, and a simple way to choose.

If operating systems concepts keep slipping away, the best fix is experiments, and our Linux & Shell and Backend Engineer courses are built around hands-on practice.

This post originally appeared on the Ciphemic Academia blog.

The Short Version

A restaurant analogy makes the difference stick:

• Process: a separate restaurant. It has its own kitchen, staff, and ingredients. Nothing is shared, so a fire in one doesn't burn the other, but opening a new restaurant is expensive and sending a message between them takes effort.
• Thread: extra cooks in the same kitchen. They share ingredients and equipment, so they're quick to add and cooperate easily, but they can collide over the same pan, and one disaster can ruin the whole kitchen.
• Async: one cook who never stands idle. When a dish needs ten minutes in the oven, the cook starts another dish instead of waiting. It's incredibly efficient for waiting-heavy work, but if the cook gets stuck on one long task, everything stops.

If you want one default: use processes for CPU-heavy work that must use many cores, threads for modest amounts of blocking I/O, and async (or lightweight tasks) when you need to handle huge numbers of concurrent connections. Most real systems combine them.

What All Three Are Actually Trying to Solve

Before the differences, the shared problems:

• Using waiting time: programs spend huge amounts of time waiting for disks, databases, and networks. Doing other work during that wait is the biggest win
• Using multiple cores: a modern machine has many cores, and a single sequential program leaves most of them idle
• Staying responsive: a program shouldn't freeze because one task is slow
• Isolating failures: sometimes one task crashing shouldn't take everything down

Two terms often get mixed up here. Concurrency is about structuring a program to handle many things in progress at once. Parallelism is about actually running things at the same instant on multiple cores. Async gives you concurrency without parallelism, while processes and threads can give you both.

Processes

What it actually is: a running program with its own private memory space, managed by the operating system. Processes don't share memory by default, so they communicate through explicit channels such as pipes, sockets, or message queues. Each one is scheduled independently, which means a process can run on its own core.

A small example (four worker processes in Python):

from multiprocessing import Pool

def square(n):
    return n * n

if __name__ == "__main__":
    with Pool(4) as pool:        # 4 separate processes, 4 separate memory spaces
        print(pool.map(square, range(10)))

Where it shines:

• Isolation: a crash or memory leak in one process doesn't corrupt another, which is why browsers like Chrome run tabs in separate processes
• True parallelism for CPU-bound work in any language, including Python in its default build, because each process has its own interpreter
• A simple mental model: with no shared memory, whole categories of race conditions disappear
• Easy to restart: servers like Gunicorn and nginx manage pools of worker processes and replace ones that die

Where it struggles:

• Heavier to create and run: each process has its own memory space and operating system bookkeeping, so you can't have as many as you can threads or async tasks
• Communication costs: data has to be serialized and sent between processes, which is slower and more awkward than sharing memory
• Memory duplication: the same data can end up copied in several processes, depending on how you share it

Who this suits: CPU-heavy tasks, workloads that need fault isolation, and security-sensitive designs where one component shouldn't be able to touch another's memory.

Threads

What it actually is: multiple paths of execution inside a single process, sharing the same memory. Because they share memory, threads are cheaper to create than processes and can communicate by reading and writing shared data. The operating system schedules them across cores.

A small example (three downloads overlapping while each waits on the network):

from concurrent.futures import ThreadPoolExecutor
import requests

urls = ["https://example.com/a", "https://example.com/b", "https://example.com/c"]

def fetch(url):
    return requests.get(url, timeout=5).status_code

with ThreadPoolExecutor(max_workers=3) as pool:
    print(list(pool.map(fetch, urls)))   # waits overlap instead of running one by one

The classic danger (shared state without a lock, then with one):

import threading

counter = 0
lock = threading.Lock()

def unsafe_increment():
    global counter
    for _ in range(100_000):
        counter += 1        # not atomic: threads can read the same value and lose updates

def safe_increment():
    global counter
    for _ in range(100_000):
        with lock:          # only one thread updates at a time
            counter += 1

Where it shines:

• Lighter than processes: faster to start, and threads share memory directly, so passing data is easy
• Simple code for blocking I/O: each thread can just wait on its own request, which is easy to read
• Real multi-core parallelism in languages and runtimes without a global lock, such as Java, C++, and Rust
• A natural fit for moderate concurrency, such as a few dozen simultaneous requests

Where it struggles:

• Shared mutable state breeds bugs: race conditions, deadlocks, and lost updates are notoriously hard to reproduce and debug
• Each thread has real weight: a stack reservation (about 1 MB on Java and Windows, about 8 MB virtual on Linux), kernel bookkeeping, and scheduling overhead, which is why thread-per-connection fails at massive scale
• Python's Global Interpreter Lock: in the default build, threads can't run Python bytecode in parallel, so CPU-bound threads don't speed up. Python 3.14 added an officially supported free-threaded build that removes this limit. It's optional, and the single-threaded cost is reported at roughly 5 to 10%, so check what your libraries support

Who this suits: applications with moderate concurrency, I/O-bound work where simple blocking code is easiest to maintain, and CPU-bound work in runtimes that allow true thread parallelism.

Async and Lightweight Tasks

What it actually is: instead of one thread per task, a small number of threads run an event loop that juggles many tasks. A task runs until it hits something slow, such as a network call, then yields control so another task can run. In Python and JavaScript this looks like async and await. Some runtimes go further: Go's goroutines and Erlang's processes are lightweight tasks scheduled by the runtime across a few real threads.

A small example (many requests in flight on one thread, using Python's asyncio):

import asyncio
import aiohttp

async def fetch(session, url):
    async with session.get(url) as resp:
        return resp.status

async def main(urls):
    async with aiohttp.ClientSession() as session:
        results = await asyncio.gather(*(fetch(session, u) for u in urls))
        print(results)

asyncio.run(main(["https://example.com/a", "https://example.com/b"]))

Where it shines:

• Tiny cost per task: a Go goroutine starts at about 2 KB of stack, and Erlang processes are similarly small, which is how WhatsApp reached millions of connections per server
• Built for waiting-heavy work: servers like nginx and Node.js use event loops because most connections are idle most of the time
• Fewer locking problems: because tasks only switch at known points, you avoid many data races, although you can still create logical ones
• Cheap context switches: switching between tasks in the runtime is generally far cheaper than the operating system switching threads, though exact figures vary by benchmark

Where it struggles:

• CPU-bound work blocks everyone: one task that computes for a second freezes every other task on that loop
• Async spreads through your code: once a function is async, its callers usually need to be too, and every library you use has to cooperate with the event loop
• One loop, one core: a single event loop doesn't use multiple cores unless you run several loops or combine it with processes
• Harder debugging: stack traces and error handling in concurrent tasks can be less intuitive
• Easy to accidentally block: a stray synchronous call inside async code, such as a blocking database driver or time.sleep, silently stalls the whole loop

Who this suits: servers that handle huge numbers of mostly waiting connections, such as chat, streaming, gateways, and API fan-out.

Side-by-Side Comparison

How Real Systems Combine Them

Production systems rarely pick just one:

• nginx runs a small number of worker processes, typically one per core, and each runs an event loop handling thousands of connections
• Gunicorn runs several worker processes, and each can use threads or an async worker
• Go schedules huge numbers of goroutines onto a handful of operating system threads
• Node.js runs JavaScript on one main thread with an event loop, while using background threads for some I/O and offering worker threads for CPU-heavy tasks
• Java 21 added virtual threads, lightweight threads managed by the JVM, a sign that the industry keeps moving toward cheaper tasks that still read like simple blocking code
• Redis executes commands on a single thread driven by an event loop, which our Redis vs. Memcached guide compares with Memcached's multi-threaded approach

A useful picture:

Process      = isolated program, own memory            (heavy, safe)
Thread       = worker inside a process, shared memory  (lighter, risky)
Async task   = tiny job on an event loop               (cheapest, never block it)

How This Fits a Career Path

• Backend engineers: choosing between worker processes, threads, and async is a daily design decision, and it explains behavior like why one slow endpoint can freeze a whole server; our Python vs. Go for backend services guide shows how language choice changes the answer
• DevOps and platform engineers: you'll tune worker counts, connection limits, and timeouts, and read the symptoms when concurrency goes wrong; our Layer 4 vs. Layer 7 load balancing post covers how connections reach those servers
• Computer science students: "what's the difference between a process and a thread?" is one of the most common operating systems questions in exams and interviews, and a clear, example-backed answer stands out
• System design interviews: be ready to explain how a server handles ten thousand connections, and why thread-per-connection stops working

How to Choose Without Overthinking It

1. Ask whether the work is CPU-bound or I/O-bound. CPU-bound work needs multiple cores, so use processes, or threads in a runtime without a global lock. I/O-bound work mostly waits, so threads or async both help.
2. Count the concurrent tasks. Dozens: threads are fine. Thousands or more: async or lightweight tasks.
3. Decide how much isolation you need. If one failure shouldn't affect others, separate processes give the strongest boundary.
4. Check your runtime. In default Python, CPU-bound threads don't help. Go and Erlang make lightweight tasks easy, and Java 21 offers virtual threads.
5. Combine when needed. A common pattern is an async server that hands CPU-heavy jobs to a pool of processes or threads.
6. Prefer high-level tools, such as pools, executors, queues, and async frameworks, over hand-rolled locks.
7. Measure first. Profile where your time actually goes before adding concurrency, because concurrency adds complexity even when it helps.

A note on honesty: the WhatsApp numbers come from conference talks in 2012 to 2014 about a heavily tuned Erlang and FreeBSD stack with custom patches, and many of those connections were idle, so they show what lightweight processes can hold, not what typical code achieves. A goroutine's roughly 2 KB starting stack is a default that grows as needed, and recent Go versions can adapt the initial size. Linux thread stacks are virtual reservations, so the 8 MB figure is not 8 MB of RAM per thread. Context-switch timings vary widely by hardware and benchmark, so treat any specific microsecond figure as an order of magnitude. Free-threaded Python is evolving quickly, so check the current release notes, including those for Python 3.15, before relying on it.

Common Mistakes When Learning Concurrency

• Expecting threads to speed up CPU-bound Python. In the default build, they won't. Use processes, or try the free-threaded build if your libraries support it.
• Blocking the event loop. Heavy computation, time.sleep, or a synchronous database call inside async code freezes every task on that loop.
• Sharing mutable state without protection. Use locks, queues, or message passing, and keep shared state as small as possible.
• Creating unlimited threads or processes. Use pools with sensible limits, since unbounded concurrency can exhaust memory or file descriptors.
• Assuming async means parallel. Async overlaps waiting. It doesn't run code on multiple cores by itself.
• Forgetting timeouts. A task waiting forever on a slow service can pile up and take the whole system down.
• Ignoring errors in background tasks. An exception inside a task can disappear silently if nothing awaits or checks it.
• Locking in inconsistent order. Two threads taking the same locks in different orders can deadlock.
• Adding concurrency before measuring. Many slow programs are slow because of one bad query, not a lack of parallelism.

Frequently Asked Questions

Is async faster than threads?

Not inherently. Async is cheaper per task, so it handles far more concurrent connections, but for a small number of tasks, threads can be just as fast and simpler to write. The difference shows up at scale.

What is the GIL, and does it still matter?

The Global Interpreter Lock lets only one thread run Python bytecode at a time in the default CPython build. Python 3.14 introduced an officially supported free-threaded build without it, but it's optional and library support is still catching up, so for many projects it still matters.

What's the simplest way to explain process versus thread?

A process is an independent program with its own memory. A thread is a worker inside a process that shares that memory. Processes are safer and heavier, and threads are lighter and riskier.

What's the difference between concurrency and parallelism?

Concurrency is handling many tasks that are in progress at once, even on one core. Parallelism is running multiple tasks at the same instant on multiple cores.

Does Node.js use threads?

Your JavaScript runs on one main thread with an event loop, which is why a long computation blocks everything. Node uses background threads internally for some operations and offers worker threads for CPU-heavy work.

How do interviewers evaluate this topic?

Usually by asking you to explain the differences and then apply them: how would you build a server for ten thousand connections, or speed up a CPU-heavy job? Naming the trade-offs and the failure modes of each shows real understanding.

Pick the Model That Matches the Work

WhatsApp didn't hold millions of connections by picking the most popular tool. It picked the model that matched the work: huge numbers of mostly idle connections, each costing almost nothing. Match your concurrency model to your workload, keep shared state small, and measure before you optimize. Explore the Linux & Shell and Backend Engineer courses to practice these ideas with real experiments instead of memorizing definitions.