Retries, Fallbacks, and Cost

Handle failures by type

Retry only short transient failures, problems likely to disappear on their own. Exponential backoff waits longer after each failure; jitter adds a small random delay so many clients do not retry together. Do not retry bad credentials, invalid input, or text that exceeds the context window. Set a small retry budget so one outage does not multiply the traffic.

A fallback sends work to a second provider when the first fails. It changes cost, privacy, speed, and answer behavior. Record which provider answered and test that path instead of assuming it creates automatic reliability.

Tip: In offline labs, use only the local provider. The same code structure teaches routing without requiring network access or credentials.