Performance and Optimization

This question ties together several individually-covered techniques from earlier topics into one consolidated view — a strong answer references each with a concrete "why."

1. Choose a minimal base image

FROM node:20-slim      # instead of the full "node:20"
# or, for even smaller:
FROM node:20-alpine

Recall from the base-image question: full images include a complete OS userland with many tools you likely never use. Slim and Alpine variants strip this down significantly, often by hundreds of megabytes, before your own application code is even added.

2. Multi-stage builds — exclude build-only tooling entirely

FROM golang:1.22 AS builder
WORKDIR /app
COPY . .
RUN go build -o server .

FROM alpine:3.19
COPY --from=builder /app/server /usr/local/bin/server
CMD ["server"]

As covered in that dedicated question, this is often the single most effective size reduction available for any compiled or build-step-requiring language. The final image contains only the compiled artifact, not the entire compiler or toolchain used to produce it.

3. Combine RUN instructions to avoid leaving bloat in an earlier layer

# BAD: cleanup in a LATER layer doesn't shrink the earlier layer that added the bloat
RUN apt-get update
RUN apt-get install -y build-essential
RUN apt-get purge -y build-essential && rm -rf /var/lib/apt/lists/*

# GOOD: install-and-cleanup happens within ONE layer
RUN apt-get update && \
    apt-get install -y --no-install-recommends some-runtime-only-package && \
    rm -rf /var/lib/apt/lists/*

Recall from the layer-ordering question: since layers are immutable once committed, "deleting" something in a later layer only hides it from the merged view. It doesn't actually shrink the earlier layer's stored size. Combining install-then-cleanup into a single RUN ensures the cleanup genuinely reduces that one resulting layer's footprint.

4. Use .dockerignore to avoid unnecessary context

.git
node_modules
*.md
test/

Recall from that question: files excluded via .dockerignore are never even sent as part of the build context. Beyond just speeding up the build, this also prevents accidentally COPYing large, unnecessary directories — an entire .git history, local build artifacts, or test fixtures — into the image at all.

5. Avoid installing unnecessary packages/recommendations

RUN apt-get install -y --no-install-recommends curl

Package managers often install a broader set of "recommended" packages by default, beyond the one you explicitly asked for. Flags like --no-install-recommends (apt), or equivalent minimal-install options in other package managers, avoid this extra, usually-unneeded bloat.

6. Minimize the number of layers where it doesn't hurt caching

Each meaningfully distinct step generally deserves its own layer, for cache-granularity reasons (see the layer-caching question). However, needlessly splitting many tiny, related operations into separate RUN instructions, when they will always change together anyway, just adds layer overhead without any caching benefit. This is a judgment call that balances cache granularity against layer-count overhead.

Why this matters beyond just "the image is smaller"

• Faster pulls, faster deployments — every node or machine that needs to run the image pulls it faster. This speeds up scaling events and deployments at real scale, a difference that compounds significantly across a fleet of many machines.
• Reduced attack surface — fewer installed packages and tools means fewer things a compromised container could exploit or use to escalate further (see the security topic).
• Lower storage costs — across many images, many tags, and many registry replicas, image size differences compound into real infrastructure cost at scale.

docker images and docker history <image> (per-layer size breakdown) are the basic tools for identifying exactly where an image's size is actually coming from. This is the necessary first step before applying any of the techniques above, rather than guessing at where the bloat lives.

This is deliberately the same core technique as the images/builds topic's cache-ordering question. It is worth restating in the performance context, since it is consistently one of the most effective build-speed optimizations available. Interviewers sometimes ask about it from either angle: as a Dockerfile authoring best practice, or as a performance-troubleshooting scenario.

The pattern, once more

FROM python:3.12-slim
WORKDIR /app

# Rarely changes -- comes FIRST
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Changes on nearly every commit -- comes LAST
COPY . .

CMD ["python", "app.py"]

A code-only change (no dependency changes) invalidates only the final COPY . . layer — the potentially slow pip install step stays cached, since its own inputs (requirements.txt) haven't changed.

Measuring the actual impact

time docker build -t myapp .
# First build: 45s (full pip install)

# Change only application code, rebuild:
time docker build -t myapp .
# Second build: 3s (pip install layer cached, only final COPY + nothing after it re-executes)

This kind of before/after measurement is worth actually doing on a real project, since the improvement from correct ordering alone is often dramatic. It can be the difference between a rebuild taking tens of seconds to minutes, versus a couple of seconds, purely from reordering instructions with no other change.

Beyond ordering: separating dependency-resolution from dependency-installation

# Some ecosystems benefit from an even finer-grained split
COPY package.json package-lock.json ./
RUN npm ci                          # only re-runs if the LOCKFILE changes, not on every dependency version bump elsewhere
COPY . .

Copying just the lockfile or manifest, not the whole project, before running the install step is the specific technique that makes this ordering effective. Copying the entire project first would still tie the install layer's cache key to the entire project's content, even if the install step technically comes "before" other application-code-touching steps. This would defeat the purpose.

Using cache mounts for package manager caches (BuildKit feature)

RUN --mount=type=cache,target=/root/.npm \
    npm ci

BuildKit's cache mounts (see the BuildKit question) provide an additional, complementary speed technique. They persist a package manager's own internal download or cache directory across builds, even when the layer itself isn't reused from cache — for example, because the lockfile changed. This means re-downloading already-fetched package versions is avoided, even when a full reinstall is otherwise triggered.

Why this is worth knowing this well for an interview

This single technique — recognizing that Docker's cache invalidates forward from the first changed instruction, and deliberately ordering instructions by "how often does this change" — is one of the most broadly useful pieces of practical Docker knowledge. It is also a common, concrete way interviewers probe whether a candidate has actually authored and iterated on real Dockerfiles, rather than only having read about Docker in the abstract.

docker stats — a quick, live snapshot

docker stats
# CONTAINER   CPU %   MEM USAGE / LIMIT   MEM %   NET I/O          BLOCK I/O
# api          12.34%   340MiB / 512MiB      66.4%   1.2MB / 3.4MB     0B / 12MB
# db            8.21%   890MiB / 1GiB        86.9%   890KB / 1.1MB     45MB / 12MB

docker stats --no-stream               # a single snapshot instead of continuously updating
docker stats api db                     # limit to specific containers
docker stats --format "table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}"   # customize output columns

This is the fastest way to get an immediate read on resource usage without setting up any additional tooling. It is useful for quick, ad-hoc checks — for example, "is this container actually using anywhere near its configured limit right now" — during local development or a quick production investigation.

Why docker stats alone isn't sufficient for real production monitoring

• No history — docker stats only shows the current, live moment. It has no memory of usage 10 minutes ago, let alone last week. This makes it useless for spotting trends, such as a slow memory leak climbing gradually over hours, or for correlating a past incident with resource usage at the time it occurred.
• No alerting — nothing notifies anyone if a container's memory usage climbs dangerously close to its limit; you have to be actively watching the output yourself.
• Doesn't scale across many hosts — checking docker stats one host at a time doesn't work once you're running containers across more than a small handful of machines.

cAdvisor — container-level metrics collection

# A cAdvisor container, commonly run as a sidecar/daemon on every host,
# exposing detailed per-container resource metrics for a scraper (like Prometheus)

cAdvisor (Container Advisor, originally built by Google) runs alongside Docker and exposes detailed, per-container resource metrics, in a format that monitoring systems like Prometheus can scrape and store historically. This is conceptually the same role that Kubernetes's kubelet-embedded cAdvisor plays for metrics-server and Prometheus in that stack (see that stack's metrics-server/Prometheus question). It is the same underlying technology, just deployed standalone for plain Docker rather than as part of a Kubernetes node.

Building a real monitoring stack

cAdvisor (per host) ──scraped by──▶ Prometheus (stores historical time series)
                                            │
                                            ▼
                                     Grafana (dashboards)
                                            │
                                            ▼
                                     Alertmanager (notifies on thresholds)

This mirrors exactly the monitoring architecture covered in the Kubernetes stack's observability topic. The same core idea — a metrics-exposing agent, a time-series database, a dashboarding tool, an alerting layer — applies whether the underlying workloads run on plain Docker hosts or a Kubernetes cluster. Only the specific agent differs (cAdvisor standalone vs. kubelet-embedded).

Diagnosing a specific resource problem

docker stats --no-stream api
# if MEM % is consistently near 100%, likely candidate for OOMKilled -- see that concern
# if CPU % is pegged at/near its --cpus limit, the container is likely being throttled

Correlating docker stats output (or, better, historical Prometheus data) against configured --memory/--cpus limits (see the lifecycle topic) is the standard first step. Use it to diagnose whether a container's observed slowness or instability is actually a resource-constraint issue, before assuming the problem lies elsewhere — in the application code, a dependency, or networking.

What changed, and why it matters

The legacy Docker builder processed a Dockerfile's instructions largely sequentially, one after another. Even independent build stages in a multi-stage build (see that question), which had no actual dependency on each other, were still built one at a time. BuildKit can analyze the full dependency graph of a build and execute genuinely independent parts in parallel, meaningfully speeding up builds with multiple, unrelated stages.

FROM node:20 AS frontend-build
WORKDIR /frontend
COPY frontend/ .
RUN npm run build

FROM golang:1.22 AS backend-build
WORKDIR /backend
COPY backend/ .
RUN go build -o server .

FROM alpine:3.19
COPY --from=frontend-build /frontend/dist /app/static
COPY --from=backend-build /backend/server /app/server

With BuildKit, the frontend-build and backend-build stages have no dependency on each other at all, so they can build simultaneously rather than following the legacy builder's strictly sequential approach. On a multi-core build machine, this can meaningfully cut overall build time for Dockerfiles with genuinely parallel, independent stages.

New capabilities the legacy builder simply didn't have

Build secrets (covered in the security topic's question):

RUN --mount=type=secret,id=npm_token \
    NPM_TOKEN=$(cat /run/secrets/npm_token) npm install

No equivalent existed in the legacy builder — this specific, secure secret-handling mechanism is BuildKit-exclusive.

Cache mounts — persisting a package manager's download cache across builds, even when the surrounding layer itself must rebuild:

RUN --mount=type=cache,target=/root/.npm \
    npm ci

This directory persists across builds independent of Docker's normal layer caching. This is useful specifically when the layer itself must re-execute (for example, when the lockfile changed) but you still want to avoid re-downloading already-fetched package versions from the network.

Multi-architecture builds via buildx (see that question) — building for multiple CPU architectures in a single command and pushing one combined manifest, a capability built directly on BuildKit's architecture.

Smarter, more granular caching

BuildKit's caching model is more sophisticated than the legacy builder's simple sequential layer cache. In some configurations, it can cache and reuse results at a finer granularity. It also supports exporting and importing build cache to and from a remote location. This is useful in CI, where a fresh build environment might otherwise start with no local cache at all. Pulling a previously-exported cache from a registry lets CI builds benefit from caching, even without persistent local disk state between runs.

How to tell if you're using it (and why you almost certainly already are)

docker build -t myapp .
# BuildKit has been the DEFAULT build engine since Docker 23.0 -- most users
# are already using it without any explicit configuration required

Since Docker version 23.0, BuildKit is the default builder for docker build. Most developers today are already benefiting from it without ever having explicitly enabled anything. On genuinely older Docker installations, it historically required setting DOCKER_BUILDKIT=1 explicitly, or using docker buildx build directly. Its presence is mostly transparent day to day. The practical value in knowing it exists comes from reaching for its newer, exclusive capabilities specifically — build secrets, cache mounts — when a build actually needs them.

Breaking "slow startup" into its actual distinct phases

"The container is slow to start" is really several potentially separate problems, and correctly diagnosing which one you actually have determines the right fix entirely:

1. Image pull time     (how long to fetch the image, if not already cached locally)
2. Container creation   (namespace/cgroup setup -- normally near-instant)
3. Process start to first log line   (how long the application takes just to begin executing)
4. First log line to actually ready   (application initialization work)

Phase 1: image pull time

time docker pull myapp:1.0

On a node that's never pulled this image before — a fresh autoscaled node, or a new deployment target — a large image (see the image-size question) can genuinely take a meaningful amount of time to pull over the network. This is exactly why image size reduction (slim/Alpine/distroless base images, multi-stage builds) has a direct, measurable impact on how quickly new instances can actually start serving traffic, especially during a scaling event or a fresh deployment to new infrastructure.

Phase 3 & 4: application-level initialization

docker run --name test myapp:1.0
docker logs -f test
# time from container start to the FIRST log line, and then from first log line
# to whatever log line indicates the application is actually ready

Common real causes of slow application-level startup:

• Loading a large dataset or cache into memory on startup, rather than lazily or incrementally.
• A slow dependency-injection/framework boot sequence — some frameworks do substantial reflection-based scanning or configuration-resolution work at startup that scales with the size of the codebase.
• Running database migrations synchronously as part of application startup, rather than as a separate, explicit deployment step.
• A JIT-compiled or bytecode-interpreted runtime's own warm-up characteristics (some JVM-based applications, for instance, have meaningfully slower cold-start performance than a fully ahead-of-time-compiled binary).

Waiting on a slow-to-become-ready dependency, with poor retry behavior

Application starts, immediately tries to connect to the database
Database isn't ready yet (still initializing)
Application either: crashes immediately (bad), or retries with a poor backoff
  strategy (e.g., retrying every 100ms in a tight loop, adding load without
  meaningfully improving the odds of success), or retries sensibly with
  exponential backoff (good)

An application with no sensible retry or backoff strategy for a dependency that isn't immediately available can appear to have "slow startup." Often the real issue is actually the dependency's own readiness timing, combined with the application's poor handling of that timing. This is exactly the class of problem that HEALTHCHECK plus depends_on: condition: service_healthy in Compose (see that topic), or a startup probe in Kubernetes (see that stack), is designed to address at the orchestration layer. This complements, rather than replaces, sensible retry logic within the application itself.

Diagnostic approach: time each phase separately, don't guess

docker run --name test -d myapp:1.0
docker logs -f test &
# note the wall-clock time between: container start, first log line,
# and whatever log line/healthcheck indicates true readiness

Rather than assuming a single cause, actually measure where the time goes. A large image pull time points at image-size optimization. A long gap between container start and the first log line points at the application's own boot sequence — this is worth profiling directly, using whatever profiling tools the language or runtime provides. A long gap between the first log line and true readiness, correlated with a dependency's own availability, points at dependency-readiness handling rather than the application's own code. Image size, application boot-time work, and dependency-readiness handling are three genuinely different problems. Shrinking an already-small image when the real bottleneck is a slow in-application data load wastes effort without addressing the actual cause.