​
Overview

Keywords AI’s AI Gateway is a gateway that lets you interface with 250+ large language models (LLMs) via one unified API. Instead of maintaining separate SDKs or endpoints for OpenAI, Anthropic, Claude, and others, you send your LLM calls to Keywords AI. And we do the rest rerouting, retries, load-balancing, caching, prompt caching, function calling, image uploading, and fallbacks for you.

​
Benefits of LLM gateway:

  • Call over 250 LLMs using the same format.
  • Ensure your LLM applications become more scalable and reliable.
  • Manage LLM costs in a single place.
  • Mange API keys in a single place without exposing them.

​
Considerations:

  • May not be suitable for products with strict latency requirements (50 - 150ms added).
  • May not be ideal for those who do not want to integrate a third-party service into the core of their application.

​
Prerequisites

For all AI gateway users, you have to add your own credentials to activate AI gateway. We will use your credentials to call LLMs on your behalf.
For example, if you want to use OpenAI, you have to add your OpenAI API key to activate AI gateway. We won’t use your credentials for any other purposes.

​
Integrate with your existing AI framework

Keywords AI offers various integration options, including: mainstream LLM frameworks and REST APIs.

​
Supported frameworks

OpenAI SDK

LangChain SDK

Vercel AI SDK

LlamaIndex SDK

Google GenAI

Anthropic

​
Examples

If you are not using any LLM frameworks, you can use the standard API call to connect 250+ LLMs. 
import requests
def demo_call(input, 
              model="gpt-4o-mini",
              token="YOUR_KEYWORDS_AI_API_KEY"
              ):
    headers = {
        'Content-Type': 'application/json',
        'Authorization': f'Bearer {token}',
    }

    data = {
        'model': model,
        'messages': [{'role': 'user', 'content': input}],
    }

    response = requests.post('https://api.keywordsai.co/api/chat/completions', headers=headers, json=data)
    return response

messages = "Say 'Hello World'"
print(demo_call(messages).json())

​
Parameters

We support all OpenAI parameters. which is the standard format for LLMs. You can check out important OpenAI parameters in this page. You can also learn more about OpenAI parameters here. For Keywords AI parameters, you should use them when you want to achieve specific goals. For example, you can use fallback_models to specify fallback models when the primary model is down. You can check out all Keywords AI parameters in this page. The most used feature of the LLM gateway is the ability to call 200+ LLMs with a single API format. You can switch between models with only 1 line of code change.