Quickstart

Keywords AI offers 2 easy ways to leverage the platform: the async LLM logging API and our AI gateway.

​

LLM logging

Keywords AI provides an Async Logging API that allows you to log your LLM requests and responses asynchronously, which offers complete observability of your LLM applications and won’t disrupt your application’s performance.

​

1. Get API key

Sign in to Keywords AI platform, and get the API key from the API keys page.

​

2. Call the Logging API

import requests

url = "https://api.keywordsai.co/api/request-logs/create/"
payload = {
    "model": "claude-3-5-sonnet-20240620",
    "prompt_messages": [
        {
            "role": "user",
            "content": "Hi"
        },
    ],
    "completion_message": {
        "role": "assistant",
        "content": "Hi, how can I assist you today?"
    },
}
headers = {
    "Authorization": "Bearer YOUR_KEYWORDS_AI_API_KEY",
    "Content-Type": "application/json"
}

response = requests.request("POST", url, headers=headers, json=payload)

​

AI gateway

An AI gateway is a simple “middleman” between you and Large Language Models (LLMs). It handles things like:

• Routing: Sending requests to different models or versions
• Fallback models: If the primary model is down, the proxy will route to a fallback model
• Load balancing: Distributing requests across multiple models
• Caching: Caching responses to improve latency and reduce costs

​

1. Get API key

Sign in to Keywords AI platform, and get the API key from the API keys page.

​

2. Add credentials

You have to add your own credentials to activate AI gateway otherwise your LLM calls will cause errors. We will use your credentials to call LLMs on your behalf.

We won’t use your credentials for any other purposes and no extra charges will be applied.

Go to the Providers page to add your credentials.

​

3. Integrate AI gateway

Keywords AI offers various integration options, including: mainstream LLM frameworks and REST APIs.

import requests
def demo_call(input, 
              model="gpt-4o-mini", # also try "claude-3-5-sonnet-20240620"
              token="YOUR_KEYWORDS_AI_API_KEY"
              ):
    headers = {
        'Content-Type': 'application/json',
        'Authorization': f'Bearer {token}',
    }

    data = {
        'model': model,
        'messages': [{'role': 'user', 'content': input}],
    }

    response = requests.post('https://api.keywordsai.co/api/chat/completions', headers=headers, json=data)
    return response

messages = "Say 'Hello World'"
print(demo_call(messages).json())

import requests
def demo_call(input, 
              model="gpt-4o-mini", # also try "claude-3-5-sonnet-20240620"
              token="YOUR_KEYWORDS_AI_API_KEY"
              ):
    headers = {
        'Content-Type': 'application/json',
        'Authorization': f'Bearer {token}',
    }

    data = {
        'model': model,
        'messages': [{'role': 'user', 'content': input}],
    }

    response = requests.post('https://api.keywordsai.co/api/chat/completions', headers=headers, json=data)
    return response

messages = "Say 'Hello World'"
print(demo_call(messages).json())

from openai import OpenAI

client = OpenAI(
    base_url="https://api.keywordsai.co/api/",
    api_key="YOUR_KEYWORDSAI_API_KEY",
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role":"user", "content":"Tell me a long story"}],
)

​

Next steps

Congratulations! You have integrated LLM proxy into your codebase! When you make any API call, you will see the LLM log on the platform.