Metrics & Health Probes

Watching Resources

The Debugging module's triage sequence mentioned docker stats briefly - here's how to actually read it like an operator, plus the companion concept of health probes.

$ docker stats                     # live table: CPU %, mem/limit, net, PIDs
$ docker stats --no-stream         # one snapshot (for scripts)
$ docker top worker                # processes inside a container

Reading docker stats like an operator:

• MEM % climbing steadily toward its limit → leak → future OOM kill (exit 137)
• CPU pinned at (100% × cores) → runaway loop or undersized --cpus
• PIDS climbing → fork leak; --pids-limit is your circuit breaker

For real fleets, the same data flows to Prometheus/Grafana via cAdvisor or the node agent of your platform - docker stats is the on-the-box emergency view.

Health Probes Close the Loop

docker ps will happily show a container as Up even if the app inside has frozen, deadlocked, or lost its database connection - "the process still exists" is not the same fact as "the app still works". A HEALTHCHECK closes that gap by making Docker itself periodically run a small test command and record whether it succeeded:

HEALTHCHECK --interval=10s --timeout=3s --retries=3 \
  CMD wget -qO- http://localhost:8080/health || exit 1

Status appears in docker ps (healthy / unhealthy / starting) and in .State.Health - which load balancers, compose depends_on: condition: service_healthy, and orchestrators act on.

Tip: Point the probe at a real /health endpoint that checks dependencies (DB reachable? queue connected?), not just "HTTP 200 from anything". A health check that can't fail is decoration.