Liveness Is Not Readiness

Different questions need different probes

This distinction was first introduced back in the FastAPI-serving module, and this final module of the track returns to it in full depth, because getting it wrong is one of the most common real-world causes of a serving outage that looks confusing from the outside. A liveness probe asks a narrow question: is the process alive enough to respond to a cheap request at all? A readiness probe asks a broader, more consequential question: should this specific instance actually be receiving real user traffic right now? A process can absolutely be alive - the Python interpreter is running fine - while still not being ready, because its required model, some piece of configuration, disk access, or a downstream service it depends on is currently unavailable.

Liveness checks should be cheap and entirely local to the process itself - never trigger actual model text generation inside one, since slow inference happening during a period of real system overload could cause the liveness check itself to time out and fail, triggering an automated restart at precisely the moment the system is already struggling, making the underlying problem worse rather than better. Readiness checks are allowed a bit more latitude to actually check required inventory or a downstream dependency, but should still use a short timeout and avoid doing genuinely expensive work themselves.

Analogy: A restaurant's lights being switched on proves the building exists and has power - that's liveness. Being actually staffed, stocked with ingredients, and able to take a real order proves it's ready for customers - that's readiness. A lit-up, empty restaurant with no staff is exactly the failure mode a naive health check that only measures liveness would completely miss.

Scenario: A FastAPI process's /readyz endpoint returns "ready" unconditionally, without actually checking whether its required model marker file is present on disk. A load balancer, trusting that response, happily routes real user traffic to this instance - and every single real request that arrives then fails, because the instance was never actually ready to begin with, it just claimed to be.

Design both probes to return small, machine-readable objects with stable, well-defined status fields, rather than free-text messages a caller has to parse loosely. In a typical production deployment, a readiness failure normally results in an instance being quietly removed from load-balancer rotation (traffic simply stops going to it, without disrupting anything else), while a liveness failure is what typically causes an orchestrator to actually restart the process outright - two genuinely different remediation actions, which is exactly why the two checks need to stay separate rather than being merged into one.