The Model Proxy Boundary

One stable endpoint for several backends

A reverse proxy accepts client requests at one address and forwards them to a backend service. A model proxy can expose an OpenAI-compatible chat-completion interface while routing to Ollama, hosted providers, or multiple deployments.

LiteLLM Proxy reads a model list that maps public route names to provider-specific models and base addresses. Applications call a stable alias such as local-primary; routing configuration selects ollama/qwen2.5:0.5b without embedding provider details in every client.

Analogy: A company switchboard keeps one public phone number while directing calls to available teams. Callers do not need every employee's extension.

Scenario: The route points to a seven-billion-parameter tag absent from the offline image. Every client receives failure even though two approved small models are already available.

A proxy is also a security and observability boundary. Authenticate callers, cap request size and time, record the selected route and latency, and never log authorization headers or full sensitive prompts.