One Interface, Different Backends

Normalize the call shape

Every LLM provider - OpenAI, Anthropic, a locally-hosted Ollama, and many others - historically expects its own slightly different request format and returns its own slightly different response shape, which makes switching providers, or supporting more than one at once, a real engineering hassle. LiteLLM is a Python library built to solve exactly that problem: it gives you one common interface, shaped like OpenAI's well-known chat-completion API, that works the same way regardless of which actual provider is answering underneath. Your application code sends the same message structure and gets back a normalized response every time, while a separate configuration layer decides whether that call actually goes to Ollama or somewhere else - the specific service doing the real work is called the backend.

import litellm
response = litellm.completion(
    model="ollama/qwen2.5:0.5b",
    messages=[{"role": "user", "content": "Say READY"}],
)
print(response.choices[0].message.content)

READY

For local Ollama specifically, a model identifier formatted like ollama/qwen2.5:0.5b makes the intended provider explicit right in the model string itself, rather than leaving it to be inferred. It's worth being clear-eyed about what this portability does and doesn't give you: having one consistent call shape across providers does not mean every model behaves identically once you swap it in - prompts, context sizes, and output quality all still need their own testing against each specific model you plan to actually use.

Analogy: A travel adapter standardizes the physical plug shape so your charger fits the wall socket in a different country. It does not make that country's electrical grid, voltage, or reliability identical to the one back home - you've solved the connector problem, not every underlying difference.