What is an overlay network, and when is it needed?
Quick Answer
An overlay network extends Docker's virtual networking across multiple separate physical/virtual hosts, letting containers running on different machines communicate with each other as if they were on the same local network — using an encapsulation protocol (VXLAN) to tunnel traffic between hosts. It's needed specifically for multi-host container deployments (most commonly, Docker Swarm mode), since a plain bridge network is inherently confined to a single host and has no concept of reaching a container running somewhere else.
Detailed Answer
Why bridge networks alone don't work across multiple hosts
A bridge network (default or user-defined) is fundamentally a local, single-host construct — it's a virtual network interface (docker0 or a custom bridge) that only exists on one machine. A container on Host A has no native way to reach a container on Host B using ordinary bridge networking. There's no shared virtual network spanning both machines — only two entirely separate, unrelated bridge networks that happen to exist on each host independently.
What an overlay network adds: a virtual network spanning multiple hosts
# On a Docker Swarm manager node
docker network create -d overlay my-overlay-network
docker service create --name web --network my-overlay-network --replicas 3 nginx
An overlay network creates a single virtual network that genuinely spans every host participating in the same Swarm cluster. A container — technically, a Swarm "task" — running on any node can reach another by name, exactly like on a single-host bridge network, even though they're physically running on entirely different machines.
The mechanism: VXLAN encapsulation
At the implementation level, overlay networking works by encapsulating container-to-container traffic inside VXLAN (Virtual Extensible LAN) packets. These packets are then routed across the hosts' normal underlying network — their real, physical or cloud network connectivity — and de-encapsulated on arrival at the destination host. This is conceptually similar to how some Kubernetes CNI plugins (like Flannel's VXLAN mode — see that stack's CNI question) achieve the same goal of making pods across many nodes appear to share one flat network. The underlying encapsulation technique and the problem it solves are directly analogous.
Host A: container "web-1" sends traffic to container "web-2" (on Host B)
→ VXLAN-encapsulated, sent over Host A's and Host B's REAL network connection
→ de-encapsulated on arrival at Host B
→ delivered to "web-2" as if it had arrived over a normal local bridge network
When you actually need this
Overlay networking is specifically relevant when running Docker Swarm mode (Docker's own built-in, simpler multi-host orchestration — see the production topic's Swarm-vs-Kubernetes question) across more than one node. It's the networking layer that makes Swarm's multi-host service deployment and scaling actually work, letting a service's replicas spread across several machines while still being able to reach each other, and be reached, consistently.
If you're running plain Docker (not Swarm) on a single host, or if you're using Kubernetes rather than Swarm for multi-host orchestration, you generally won't interact with Docker's own overlay network driver directly. Kubernetes has its own separate, CNI-plugin-based networking model (see that stack) that solves the same multi-host connectivity problem in its own way, independent of Docker's overlay network feature specifically.