Add fallback models

Keywords AI will catch any errors occurring in a request to model and fall back to the list of models you specified in the fallback_models field. This is useful to avoid downtime and ensure that your application is always available.

See all Keywords AI params here.

​

Set up fallbacks

Fallback models can be set up in the UI or in the code. You can add multiple fallback models to the list, we will try each model in the order they are listed.

​

Set up fallbacks from the UI

Go to Settings -> Fallback -> Click on Add fallback models -> Select the models you want to add as fallbacks.

You can drag and drop the models to reorder them. The order of the models in the list is the order in which they will be tried.

​

Set up fallbacks in code

• OpenAI Python SDK
• OpenAI TypeScript SDK
• Standard API
• Other SDKs

from openai import OpenAI

client = OpenAI(
    base_url="https://api.keywordsai.co/api/",
    api_key="YOUR_KEYWORDSAI_API_KEY",
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "user", "content": "Tell me a long story"}
    ],
    extra_body={
        "fallback_models": ["claude-3-5-sonnet-20240620", "gemini-1.5-pro-002"]
    }
)

​

Troubleshooting

• You turned off Fallbacks in the UI
• You uploaded some invalid customer_credentials
• The models you specified in the Fallbacks list are not available
• The provider of the model does not return a response (so we can’t detect if it’s down or not)