Fallback models

Keywords AI will catch any errors occurring in a request to model and fall back to the list of models you specified in the fallback_models field. This is useful to avoid downtime and ensure that your application is always available. See all Keywords AI params here.

Set up fallbacks

Fallback models can be set up in the UI or in the code. You can add multiple fallback models to the list, we will try each model in the order they are listed.

Set up fallbacks from the UI

Go to Settings -> Fallback -> Click on Add fallback models -> Select the models you want to add as fallbacks. You can drag and drop the models to reorder them. The order of the models in the list is the order in which they will be tried.

Set up fallbacks in code

OpenAI Python SDK
OpenAI TypeScript SDK
Standard API
Other SDKs

from openai import OpenAI

client = OpenAI(
    base_url="https://api.keywordsai.co/api/",
    api_key="YOUR_KEYWORDSAI_API_KEY",
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "user", "content": "Tell me a long story"}
    ],
    extra_body={
        "fallback_models": ["claude-3-5-sonnet-20240620", "gemini-1.5-pro-002"]
    }
)

In OpenAI TypeScript SDK, you should add a // @ts-expect-error before the fallback_models field.

import { OpenAI } from "openai";

const client = new OpenAI({
  baseURL: "https://api.keywordsai.co/api",
  apiKey: "YOUR_KEYWORDSAI_API_KEY",
});

const response = await client.chat.completions
  .create({
    messages: [{ role: "user", content: "Say this is a test" }],
    model: "gpt-4o-mini",
    // @ts-expect-error
    fallback_models: ["claude-3-5-sonnet-20240620", "gemini-1.5-pro-002"]
  })
  .asResponse();

console.log(await response.json());

import requests
def demo_call(input, 
              model="gpt-4o-mini",
              token="YOUR_KEYWORDS_AI_API_KEY",
              ):
    headers = {
        'Content-Type': 'application/json',
        'Authorization': f'Bearer {token}',
    }

    data = {
        'model': model,
        'messages': [{'role': 'user', 'content': input}],
        'fallback_models': ["claude-3-5-sonnet-20240620", "gemini-1.5-pro-002"]
    }

    response = requests.post('https://api.keywordsai.co/api/chat/completions', headers=headers, json=data)
    return response

messages = "Say 'Hello World'"
print(demo_call(messages).json())

Troubleshooting

You turned off Fallbacks in the UI
You uploaded some invalid customer_credentials
The models you specified in the Fallbacks list are not available
The provider of the model does not return a response (so we can’t detect if it’s down or not)

Get started

Features

Admin

Security

Resources

Help & Community

Fallback models

Set up fallbacks

Set up fallbacks from the UI

Set up fallbacks in code

Troubleshooting

Get started

Features

Admin

Security

Resources

Help & Community

​Set up fallbacks

​Set up fallbacks from the UI

​Set up fallbacks in code

​Troubleshooting

Set up fallbacks

Set up fallbacks from the UI

Set up fallbacks in code

Troubleshooting