What is Docker Compose, and what problem does it solve?

Docker Compose lets you define an entire multi-container application — every service, its build/image, environment variables, networks, and volumes — declaratively in a single YAML file, and bring the whole thing up or down with one command. It solves the problem of manually running a long sequence of individual `docker run`/`docker network create`/`docker volume create` commands (in the right order, with the right flags, every single time) by capturing the entire application's configuration as a single, version-controllable, repeatable file.

What are the key sections of a compose.yaml file?

`services` defines each container to run, with its image or build instructions, ports, environment variables, and volumes. `networks` optionally defines custom networks beyond the default one Compose creates automatically. `volumes` declares named volumes referenced by one or more services, so their definitions (and driver options, if any) live in one place. Most Compose files only need to explicitly define `services` — `networks` and top-level `volumes` are often just referenced implicitly or with minimal extra configuration.

How does Compose handle environment variables and .env files?

Compose automatically reads a file named `.env` in the same directory as the compose file, making its variables available for substitution (`${VAR_NAME}`) anywhere in the compose file itself. Separately, the `environment:` key sets environment variables *inside* a service's container at runtime, and `env_file:` loads an entire file's worth of variables into a service's container in one step — these are two related but distinct mechanisms: `.env` substitutes values into the compose file's own YAML, while `environment`/`env_file` set what the running container actually sees.

What's the difference between docker compose up and docker compose up -d?

`docker compose up` runs in the foreground, attaching your terminal to every service's combined log output — useful for actively watching logs during development, but it blocks your terminal until you stop it (Ctrl+C, which also stops all the services). `docker compose up -d` (detached mode) starts everything in the background and immediately returns control of your terminal, which is the standard choice for anything beyond active, hands-on local development where you want to keep watching logs directly.

How do you express a dependency between services in Compose, and what are its limitations?

`depends_on` controls **startup order** — Compose starts a dependency's container before the dependent service's container. By default, this only waits for the dependency's container to *start*, not for the application inside it to actually be ready to accept connections — a real and common source of race conditions (e.g., an API container starting before its database has actually finished initializing). Adding `condition: service_healthy` (requiring the dependency to define a `HEALTHCHECK`) closes this gap, making Compose wait for genuine readiness, not just process start.

What are Compose profiles, and when would you use them?

Profiles let you tag services in a single compose file so they're only started when their profile is explicitly activated — services with no profile assigned always start by default, while profile-tagged services are opt-in. This is useful for including optional or situational services (debugging tools, a local monitoring stack, a service only needed for a specific workflow) in the same compose file as your core application, without them cluttering or slowing down every ordinary `docker compose up`.

How does Docker Compose differ from Kubernetes, and when would you outgrow it?

Compose orchestrates multiple containers on a **single host**, with no built-in self-healing across machine failures, no rolling updates, and no horizontal scaling across a cluster. Kubernetes orchestrates containers across a **cluster of many machines**, with automatic rescheduling on node failure, sophisticated rolling updates and rollbacks, and horizontal scaling — solving problems Compose was never designed to address at all. Teams typically outgrow Compose exactly when they need multi-host scale, high availability across machine failures, or production-grade rollout/rollback sophistication that a single Docker host fundamentally cannot provide.

Docker Compose

Defining and running multi-container applications declaratively with a single compose.yaml file.

Difficulty

Open as page

The problem: real applications are rarely just one container

# Manually orchestrating a 3-service application with plain docker commands:
docker network create my-app-network
docker volume create db-data
docker run -d --network my-app-network --name db -v db-data:/var/lib/postgresql/data \
  -e POSTGRES_PASSWORD=secret postgres:16
docker run -d --network my-app-network --name api -e DB_HOST=db \
  -p 3000:3000 myapi:1.0
docker run -d --network my-app-network --name web -p 8080:80 myweb:1.0

Even a modest three-service application requires remembering and correctly sequencing several separate commands, each with several flags. This is error-prone to type out manually every time, hard to share consistently with teammates, and impossible to meaningfully version-control as a single coherent unit.

The Compose solution: one declarative file

# compose.yaml
services:
  db:
    image: postgres:16
    environment:
      POSTGRES_PASSWORD: secret
    volumes:
      - db-data:/var/lib/postgresql/data

  api:
    image: myapi:1.0
    environment:
      DB_HOST: db
    ports:
      - "3000:3000"
    depends_on:
      - db

  web:
    image: myweb:1.0
    ports:
      - "8080:80"

volumes:
  db-data:

docker compose up -d      # brings up ALL three services, the network, and the volume
docker compose down        # tears it all down again

One file captures the entire application's shape: every service, its configuration, and how they relate. Two simple commands then bring the whole thing up or down consistently, every time, on any machine with Docker and Compose installed.

What Compose handles automatically that you'd otherwise do manually

Creates a dedicated network automatically — every service defined in the file can reach every other service by its service name (db, api, web) via DNS, with zero manual network setup (see the networking topic's DNS question). Compose creates a user-defined bridge network for the whole project by default.
Manages startup/shutdown ordering — respects depends_on relationships (with the caveats covered in that question).
Consistent, reproducible environment — the exact same file, committed to version control, produces the exact same multi-container setup for every developer on a team, or in CI. No one needs to remember or manually re-type the individual docker run commands.

What Compose is (and isn't) meant for

Compose is designed and best suited for single-host multi-container applications — local development environments, simple single-server deployments, CI test environments spinning up a full application stack for integration testing. It is explicitly not a multi-host orchestrator. It has no concept of scheduling containers across many machines, no built-in self-healing or rescheduling on node failure, and no rolling-update mechanism comparable to Kubernetes's. (See that stack's Compose-vs-Kubernetes comparison question for exactly where Compose's scope ends and a real orchestrator's begins.)

Why this matters for local development specifically

Compose has become the standard way to "spin up this whole application's dependencies locally." A single docker compose up command that brings up a database, a cache, a message queue, and the application itself, all correctly networked together, is a dramatically better onboarding and local-development experience. This beats expecting every developer to manually install and configure each dependency directly on their own machine.

Related Resources

Docker Compose overview

Open as page

services — the core of every Compose file

services:
  api:
    image: myapi:1.0             # OR: build: ./api  (build from a local Dockerfile instead)
    ports:
      - "3000:3000"
    environment:
      - NODE_ENV=production
    volumes:
      - api-logs:/app/logs
    networks:
      - backend
    depends_on:
      - db
    restart: unless-stopped

Each entry under services corresponds to (roughly) one docker run invocation's worth of configuration — image/build source, ports, environment, volumes, network attachments, startup dependencies, and restart policy, all in one place, per service.

build vs. image — where the container's code comes from

services:
  api:
    build:
      context: ./api
      dockerfile: Dockerfile.prod
    image: myapi:1.0    # optional: tag the resulting build with this name too

image alone pulls a pre-built image from a registry. build instead builds the image locally from a Dockerfile. The two are commonly used together during local development — building from source, then tagging it with a name. A production deployment, on the other hand, more often just references a pre-built image that CI already built and pushed.

networks (top-level) — custom networks beyond the default

services:
  web:
    networks:
      - frontend
  api:
    networks:
      - frontend
      - backend
  db:
    networks:
      - backend

networks:
  frontend:
  backend:

By default, Compose creates a single network shared by every service in the project. This top-level networks section, paired with a networks: list under each service, is only needed when you want the same kind of deliberate multi-network segmentation covered in the plain-Docker networking topic — for example, keeping db unreachable from web directly, only reachable via api.

volumes (top-level) — declaring named volumes

services:
  db:
    volumes:
      - db-data:/var/lib/postgresql/data

volumes:
  db-data:
    driver: local     # optional -- and can specify driver-specific options here too

A named volume referenced by a service (db-data above) should generally also be declared under the top-level volumes: section. This is where you would specify a non-default driver or driver options, if needed. A bare declaration — just db-data: with nothing else — is sufficient for the common case of a plain, Docker-managed local volume.

Environment variables and .env files

services:
  api:
    environment:
      - DB_PASSWORD=${DB_PASSWORD}    # substituted from a .env file or the shell environment
    env_file:
      - .env.api                       # load an entire file of variables at once

Covered in full in the dedicated environment-variables question — Compose supports both inline environment: entries (with variable substitution from a .env file) and loading an entire separate file via env_file:.

Why most real Compose files are dominated by the services section

For the common case of a single-network application with standard, Docker-managed volumes, the networks and volumes top-level sections often need little or no extra configuration beyond a bare declaration. The real substance of a Compose file — and what most of your time authoring one goes into — is correctly configuring each service's image or build, environment, ports, volumes, and dependencies within the services section.

Related Resources

Compose file reference

Open as page

Two distinct mechanisms that are easy to conflate

1. The .env file — substituted into the compose file itself, at parse time

# .env (same directory as compose.yaml)
DB_PASSWORD=supersecret
API_PORT=3000

# compose.yaml
services:
  api:
    ports:
      - "${API_PORT}:3000"
    environment:
      - DB_PASSWORD=${DB_PASSWORD}

Compose automatically reads a file literally named .env in the project directory. It uses its values to fill in any ${VAR_NAME} placeholders within the compose file's own YAML text, before the file is even processed. This happens regardless of whether those placeholders end up inside an environment: block, a ports: mapping, an image tag, or anywhere else in the file.

2. environment: / env_file: — set inside the running container

services:
  api:
    environment:
      - NODE_ENV=production          # set directly, a literal value
      - DB_PASSWORD=${DB_PASSWORD}     # set via .env substitution (mechanism #1, feeding into #2)
    env_file:
      - .env.api                        # load an ENTIRE separate file's variables into the container

environment: and env_file: control what environment variables the running container itself actually sees, equivalent to docker run -e. This is a runtime concern for the application inside the container, distinct from the .env file's job of substituting values into the compose YAML's own text at parse time.

Why the distinction matters in practice

services:
  api:
    image: myapi:${APP_VERSION}     # .env substitution determines WHICH IMAGE TAG to use
    environment:
      - APP_VERSION=${APP_VERSION}   # separately, ALSO expose that same value to the running app

A value from .env can be used purely to parameterize the compose file itself, like choosing an image tag or a host port, without necessarily also being passed into the container's environment. Conversely, a container's environment: block can set variables that have nothing to do with any .env substitution at all, using plain literal values. Confusing "this variable is available for compose-file templating" with "this variable is available inside my running application" is a common source of "why isn't my app seeing this environment variable" confusion.

A dedicated env_file, separate from .env

# .env.api
NODE_ENV=production
LOG_LEVEL=info
DB_CONNECTION_STRING=postgres://...

services:
  api:
    env_file:
      - .env.api

env_file: is specifically for loading a whole file's worth of variables directly into one service's container. This is distinct from the special, automatically-loaded .env file, which is scoped to the whole compose file's variable substitution, not tied to any one particular service.

Never commit real secrets into either mechanism, uncommented

# .gitignore
.env
.env.*

Both .env and any custom env_file are plain, unencrypted text. Genuinely sensitive values, such as database passwords or API keys, should not be committed to version control in either form. The standard practice is to .gitignore them and provide a .env.example template with placeholder, non-sensitive values showing the expected shape. This mirrors the same secrets-hygiene principle covered in the SQL/Databases and Kubernetes stacks.

Related Resources

Compose: Environment variables

Open as page

Foreground mode — attached, blocking

docker compose up
# api-1  | Server listening on port 3000
# db-1   | database system is ready to accept connections
# web-1  | nginx started
# ... (combined, live, color-coded log output from every service, streamed to your terminal)

Your terminal is now attached to the combined output of every running service. This is genuinely useful when actively developing and you want to watch logs from all services simultaneously in one place, with each line prefixed by which service produced it. Until you press Ctrl+C, this terminal is blocked entirely — you cannot run other commands in that same session while docker compose up is running, unless you open a separate terminal. Pressing Ctrl+C sends a stop signal to every service and then returns control of your terminal.

Detached mode — backgrounded, returns immediately

docker compose up -d
# [+] Running 3/3
#  ✔ Container myapp-db-1   Started
#  ✔ Container myapp-api-1  Started
#  ✔ Container myapp-web-1  Started

Starts every service in the background and immediately hands your terminal back — you can continue running other commands in the same session. To view logs afterward, you explicitly ask for them:

docker compose logs -f       # follow combined logs from all services, on demand
docker compose logs -f api    # follow logs from just the "api" service

Why -d is the standard choice for most real usage

For anything beyond a brief moment of active, hands-on local development where you specifically want to watch logs stream by, -d is almost always what you want. It lets your terminal remain free for other work, and it matches how you would run things in CI, staging, or any semi-automated context. You are not going to leave a terminal session permanently attached and blocked in a CI pipeline or on a server. Combined with docker compose logs -f whenever you do actually want to look at output, detached mode gives you the same log visibility on demand, without the terminal-blocking downside of foreground mode.

Bringing things back down

docker compose down          # stops and REMOVES containers, default network (but NOT named volumes)
docker compose down -v        # also removes named volumes -- be careful, this deletes data!
docker compose stop            # just stops containers, without removing them at all

Regardless of whether you started with foreground or detached mode, docker compose down is the standard teardown command. It is worth explicitly noting that plain docker compose down does not delete named volumes by default, so your data survives. docker compose down -v does delete them, and this distinction matters enormously in practice. An accidental -v on a production-adjacent environment can mean genuine, irreversible data loss.

Related Resources

docker compose up reference

Open as page

The basic (and incomplete) version

services:
  db:
    image: postgres:16
  api:
    image: myapi:1.0
    depends_on:
      - db

docker compose up will start db's container before starting api's container. But critically, this only guarantees that db's container process has started — not that PostgreSQL inside it has actually finished its own initialization and is ready to accept connections. Database engines (and many other services) often take a real, non-trivial amount of time after their process starts before they're genuinely ready to serve requests.

The resulting race condition

1. docker compose up starts db's container
2. Compose immediately proceeds to start api's container (db's PROCESS has started)
3. api's application code tries to connect to the database...
4. ...but Postgres inside db is STILL initializing, not yet accepting connections
5. api's connection attempt fails, possibly crashing api on startup

This is a very common, easy-to-hit real-world Compose bug. Everything looks correctly ordered, since depends_on: [db] is right there in the file. But the actual application-level readiness gap between "container started" and "service ready" isn't accounted for at all by plain depends_on.

The fix: condition: service_healthy

services:
  db:
    image: postgres:16
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 5s
      retries: 5
  api:
    image: myapi:1.0
    depends_on:
      db:
        condition: service_healthy

This changes the guarantee meaningfully: api's container now won't even be started until db's own HEALTHCHECK (see the lifecycle topic's question) actually reports healthy — genuinely waiting for readiness, not just process start. This requires the dependency (db) to actually define a HEALTHCHECK in the first place; condition: service_healthy has nothing to check against otherwise.

Other available conditions

depends_on:
  db:
    condition: service_started      # the default -- just waits for the container to start
  migrations:
    condition: service_completed_successfully   # waits for a one-off task container to EXIT successfully

service_completed_successfully is specifically useful for a one-off setup task — for example, running database migrations as a separate service that's expected to run once and exit. A dependent service can wait for this task to fully finish, rather than waiting for a long-running service to become healthy.

What depends_on still doesn't solve, even with service_healthy

Even with a proper healthcheck-gated dependency, Compose's depends_on only governs the initial startup sequence. It says nothing about what happens if db later crashes and restarts while api is already running. api's application code still needs its own resilience — retry logic, connection pooling with reconnection — to handle a dependency becoming temporarily unavailable after both services are already up and running. depends_on is purely a startup-ordering tool, not an ongoing dependency-health guarantee for the whole lifetime of the application.

Related Resources

Compose: Control startup order

Open as page

The problem: not every service should always start

A single compose file for a real project often accumulates services that aren't needed for every single docker compose up — a local monitoring dashboard, a debugging/admin tool, a service that's only relevant for a specific kind of testing. Without profiles, every one of these either has to be commented in and out manually, or has to live in an entirely separate compose file that needs to be merged in manually. Both approaches are more awkward than necessary.

Tagging services with a profile

services:
  api:
    image: myapi:1.0
    # no "profiles" key -- this service ALWAYS starts, regardless of active profiles

  db:
    image: postgres:16
    # also always starts

  pgadmin:
    image: dpage/pgadmin4
    profiles: ["debug"]        # only starts if the "debug" profile is explicitly activated

  load-tester:
    image: my-load-test-tool
    profiles: ["testing"]

Activating profiles

docker compose up -d
# Only "api" and "db" start -- pgadmin and load-tester are skipped entirely

docker compose --profile debug up -d
# Now "api", "db", AND "pgadmin" all start

docker compose --profile debug --profile testing up -d
# Multiple profiles can be activated together

A service with no profiles key at all is always included, regardless of which profiles, if any, are active. Profiles are purely an opt-in mechanism for the services explicitly tagged with one. They do not let you exclude a service that has no profile assigned.

Also settable via an environment variable

export COMPOSE_PROFILES=debug
docker compose up -d      # equivalent to passing --profile debug

Useful for setting a consistent default profile for a given developer's local environment or a specific CI job, without needing to remember to pass the flag on every single command.

Common real-world use cases

Debugging/admin tools — a database GUI (pgAdmin, Adminer), only needed occasionally, not on every developer's routine docker compose up.
Optional local observability — a local Prometheus/Grafana stack for developers who specifically want to inspect metrics locally, without imposing that overhead on everyone by default.
Environment- or workflow-specific services — a service only relevant for running a particular kind of test suite (a mock external API, a load-testing tool), kept in the same file as the core application for convenience, without slowing down or cluttering the default startup.

Why this beats maintaining multiple separate compose files

Before profiles existed, achieving this same "optional service, sometimes included" behavior typically required one of two approaches: multiple separate compose files merged together with -f base.yaml -f debug.yaml, or manually commenting services in and out. Both are more error-prone and harder to keep in sync than a single file with declarative, named profile tags that anyone on the team can discover just by reading the file.

Related Resources

Compose: Using profiles

Open as page

The core scope difference: one host vs. a cluster

Docker Compose fundamentally orchestrates containers on one Docker host — every service defined in a compose file runs on the single machine where docker compose up was executed. Kubernetes (see that stack) orchestrates containers across a cluster of potentially many machines. A scheduler decides which node each workload actually runs on, entirely independent of where you happen to be running kubectl from.

Compose: docker compose up  -> everything runs on THIS one machine

Kubernetes: kubectl apply -f deployment.yaml -> the SCHEDULER decides which of
            potentially dozens/hundreds of cluster nodes each Pod actually runs on

What Compose fundamentally cannot do, by design

No automatic rescheduling on machine failure — if the single host running your Compose application goes down, everything goes down with it. There is no other machine for anything to fail over to, since Compose has no concept of "other machines" at all.
No horizontal scaling across hosts — docker compose up --scale api=5 can run 5 instances of the api service, but all 5 still run on that same single host, sharing its finite CPU, memory, and disk. There is no way to spread replicas across multiple machines the way Kubernetes naturally does.
No sophisticated rolling updates — Compose can restart services with an updated image, but it lacks Kubernetes's fine-grained rolling update controls: maxUnavailable/maxSurge, readiness-probe-gated progression, and automatic rollback on failure (see that stack's Deployment question).
No built-in self-healing beyond simple restart policies — Compose respects restart: policies per service, identical to plain Docker's (see the lifecycle topic). But it has nothing comparable to Kubernetes's continuous reconciliation loops, which actively work to keep desired and actual cluster-wide state in sync.

Where Compose genuinely excels, and remains the right tool

Local development — spinning up an entire application's dependencies (databases, caches, message queues, the application itself) with one command, identically for every developer on a team, is Compose's strongest and most enduring use case.
CI test environments — bringing up a full integration-testing stack for the duration of a CI job, then tearing it down, is a natural fit for Compose's single-host, ephemeral-by-design model.
Simple, single-server production deployments — a small application that genuinely doesn't need multi-host scale or sophisticated availability guarantees can legitimately run in production via Compose on a single well-maintained server. Not every application needs Kubernetes's complexity (see that stack's "how do you decide whether a project needs Kubernetes" question for the fuller version of this judgment call).

The signals that indicate you've outgrown Compose

You need the application to survive a single machine failing entirely, not just a single container crashing.
You need to scale beyond what one machine's resources can provide.
You need genuinely sophisticated, safe rollout strategies (canary, blue-green — see the Kubernetes stack) with automatic health-gated progression and rollback.
You're running enough distinct services/teams that you need the richer scheduling, resource-quota, and multi-tenancy primitives a real orchestrator provides (see that stack's topics).

Why this isn't a value judgment against Compose

Compose solving a narrower problem than Kubernetes isn't a shortcoming. It is a deliberate, appropriate scope. Using Compose for local development, even when production runs on Kubernetes, is extremely common and sensible. Many teams use Compose specifically to closely approximate their production container images and configuration locally, without needing a full local Kubernetes cluster like minikube or kind for everyday development work. The judgment call is about matching the tool's actual scope to the actual problem. Reaching for Kubernetes's cluster-wide orchestration complexity before you have a genuine multi-host, high-availability, or sophisticated-rollout requirement is over-engineering — exactly as covered in the Kubernetes stack's own judgment question on this topic.

Related Resources

Kubernetes: What is Kubernetes?