When docker ps reports a container as “Up 3 hours”, all it means is that the process has not exited. The application inside can be deadlocked, out of database connections or returning 500 on every request, and Docker will keep cheerfully reporting Up. HEALTHCHECK exists to close that blind spot: it teaches Docker to ask the application “can you still serve?” instead of merely checking that the process breathes.

How does HEALTHCHECK work?

A HEALTHCHECK is a command you define, which Docker runs periodically inside the container. The contract is exit codes only: 0 means healthy, 1 means broken. The container state moves between three values:

  • starting: within the start_period window after launch.
  • healthy: the latest check passed.
  • unhealthy: enough consecutive failures to exhaust retries.

A complete Dockerfile for a Node.js API:

FROM node:22-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --omit=dev
COPY . .
EXPOSE 3000

HEALTHCHECK --interval=30s --timeout=5s --retries=3 --start-period=20s \
  CMD wget -qO- http://127.0.0.1:3000/healthz || exit 1

CMD ["node", "server.js"]

And the docker-compose equivalent, including a startup ordering constraint:

services:
  api:
    build: .
    healthcheck:
      test: ['CMD', 'wget', '-qO-', 'http://127.0.0.1:3000/healthz']
      interval: 30s
      timeout: 5s
      retries: 3
      start_period: 20s

  worker:
    build: ./worker
    depends_on:
      api:
        condition: service_healthy   # worker starts only after api is healthy

The quiet superpower of depends_on with condition: service_healthy: it turns the health check into a startup coordination mechanism, ending the era of “sleep 10 and pray” while waiting for another service to come up.

What do interval, timeout, retries and start_period mean?

Parameter Default Meaning Practical guidance
interval 30s Gap between two checks 30s for most services; 10-15s when fast detection matters
timeout 30s Past this, the check counts as failed 3-5s; a health endpoint needing more than 5s has its own problem
retries 3 Consecutive failures before unhealthy 3 is the sweet spot between sensitivity and stability
start_period 0s Grace window after startup, failures not counted Set it to your slowest measured startup, plus margin

The formula worth memorizing: worst-case detection time is roughly interval x retries plus timeout. With 30s x 3 plus 5s, a container that dies right after a successful check can take about 95 seconds to be flagged unhealthy. Shrink the interval if you need faster detection, but do not drop retries to 1: one GC pause or load spike will hand you a false unhealthy.

start_period is the most commonly forgotten parameter. A Spring Boot application can take 40-60 seconds to boot: without a start_period, the container collects its 3 failures and gets branded unhealthy before it even opens the port, the orchestrator kills and recreates it, and you have built an infinite loop.

What are the most common HEALTHCHECK mistakes?

A health endpoint that calls external services. The most damaging and the most widespread mistake. Someone writes /healthz “thoroughly” by pinging the database, Redis and a third-party API. The consequence: when that external dependency slows down or fails, every container calling it turns unhealthy at once, the orchestrator mass-restarts containers that have nothing wrong with them, and a small localized issue becomes a systemwide cascade. More than one public postmortem in this industry features exactly this script. The rule: a basic health check answers only “can this process still serve?”, and any deep readiness check belongs in a separate probe.

Using curl in an image that has no curl. The check fails 100 percent of the time with executable file not found, and a perfectly healthy container is reported unhealthy forever. Alpine ships only BusyBox wget; distroless has no shell at all. The cleanest fix is checking with the runtime already present:

HEALTHCHECK --interval=30s --timeout=5s --retries=3 --start-period=20s \
  CMD node -e "fetch('http://127.0.0.1:3000/healthz').then(r => process.exit(r.ok ? 0 : 1)).catch(() => process.exit(1))"

Forgetting that exit codes other than 0 and 1 break the contract. curl without --fail returns 0 even when the server answers HTTP 500, meaning your check always “passes”. Always use curl --fail or verify the status code yourself.

A check that is too heavy. Health checks run forever on a schedule; an aggregate database query every 30 seconds is self-inflicted load. A check should be light enough to vanish in your performance statistics.

How do you debug an unhealthy container?

When a container gets the unhealthy label, do not guess. Ask Docker directly:

docker inspect --format '{{json .State.Health}}' api | jq

The output holds the last 5 checks with exit codes and full output, which usually names the problem immediately: a missing binary, a timeout, or an HTTP 500 from the app. To reproduce by hand, run docker exec -it api wget -qO- http://127.0.0.1:3000/healthz and read the output with your own eyes.

HEALTHCHECK is done. Who calls you at 2 a.m.?

A HEALTHCHECK only changes container state; it sends no notification anywhere and restarts nothing. On a plain docker-compose server with no orchestrator, an unhealthy container just sits there waiting for someone to happen to run docker ps. The missing piece is a monitoring system that reads that state and raises the alarm: the AgentWatch agent watches Docker containers on three levels, process state, HEALTHCHECK result and an HTTP probe, then alerts over Zalo or email when a container falls out of health. The Free plan covers 2 servers, enough to try it against your compose stack; details on the pricing section.

The recipe for a decent HEALTHCHECK, condensed: keep the check light, ask only the container itself, use a short timeout, 3 retries, a start_period longer than your slowest boot, and make sure a human hears about it when the light turns red.