Health, Readiness, and Lifespan

Alive is not ready

This distinction was introduced briefly back in the Ollama CLI module, and it deserves a fuller treatment here because it's easy to get wrong in a real API. A liveness endpoint answers one narrow question: is the Python process itself still alive and able to respond to a cheap request at all? A readiness endpoint answers a broader, more useful question: can this instance actually do real work right now - specifically, is Ollama reachable, and does the configured model actually exist and load correctly?

@app.get("/livez")
def livez():
    return {"status": "alive"}

@app.get("/readyz")
async def readyz():
    ok = await ollama_has_model("qwen2.5:0.5b")
    if not ok:
        return JSONResponse(status_code=503, content={"status": "not_ready"})
    return {"status": "ready"}

It's also worth loading expensive shared resources - like confirming the model is available, or opening a database connection - exactly once, during the application's startup-to-shutdown lifespan, rather than repeating that setup work on every single incoming request, which would waste time and CPU on every call. Uvicorn is the actual web server process that runs your FastAPI application and listens for incoming connections. On this platform's small, shared CPU host, stick to a single Uvicorn worker process - spinning up more workers would each copy the application's Python state independently and end up fighting each other over access to the same single Ollama daemon underneath, with no real benefit.

Warning: Never make the liveness check perform actual model generation. If the model is briefly slow under load, an expensive liveness check would fail right when the system is already under stress, potentially triggering an automated restart loop at exactly the worst possible moment.