Define the API Contract

Put a stable boundary around inference

So far, calling the model has meant talking to Ollama's own HTTP API directly. A real application usually wraps that behind its own API instead, and FastAPI is a popular Python framework for building exactly that kind of HTTP API. Its standout feature is using ordinary Python type declarations to automatically validate incoming request data and generate an OpenAPI document - a machine-readable description of every endpoint, its inputs, and its outputs, which other tools can use to build documentation or client code automatically.

from fastapi import FastAPI
from pydantic import BaseModel, constr

app = FastAPI()

class AskRequest(BaseModel):
    question: constr(max_length=2000)
    model: str = "qwen2.5:0.5b"

class AskResponse(BaseModel):
    answer: str
    model: str
    latency_ms: int

Designing this contract well means being deliberate about required fields, maximum lengths, which model names are actually allowed, and exactly what a successful response contains - here, the answer text, which model produced it, how long it took, and optionally source citations if the endpoint uses RAG. It's just as important to get failure right: return a proper client-error status code for invalid input, and a proper service-error status code when inference genuinely can't be performed right now. Never fall into the trap of converting every possible failure into an HTTP 200 "success" response with an error field buried inside the JSON body - that forces every caller to remember to check for that field, and inevitably some of them won't.

Analogy: A restaurant's printed menu is a stable promise to customers, even while the kitchen behind it swaps out equipment, changes suppliers, or updates recipes. Your API's response contract is that menu; Ollama - or whichever backend you're actually calling - is just one specific piece of kitchen equipment behind the scenes.

Tip: Keep your endpoint's response contract independent from Ollama's raw response shape. If you ever swap backends, add a fallback provider, or change how you call the model internally, callers of your API shouldn't need to change a single line of their own code.