Keywords AI offers 2 easy ways to leverage the platform: the async LLM logging API and our AI gateway.

LLM logging

Track your LLM usage and performance with 0 latency impact.

AI gateway

Call 250+ LLMs with a single API and get complete observability.

​
Choose Your Integration: Logging, Gateway, or Tracing?

​
LLM logging

Keywords AI provides an Async Logging API that allows you to log your LLM requests and responses asynchronously, which offers complete observability of your LLM applications and won’t disrupt your application’s performance.

​
1. Get API key

Sign in to Keywords AI platform, and get the API key from the API keys page.
Create API key placeholderCreate API key placeholder

​
2. Call the Logging API

import requests

url = "https://api.keywordsai.co/api/request-logs/create/"
payload = {
    "model": "claude-3-5-sonnet-20240620",
    "prompt_messages": [
        {
            "role": "user",
            "content": "Hi"
        },
    ],
    "completion_message": {
        "role": "assistant",
        "content": "Hi, how can I assist you today?"
    },
}
headers = {
    "Authorization": "Bearer YOUR_KEYWORDS_AI_API_KEY",
    "Content-Type": "application/json"
}

response = requests.request("POST", url, headers=headers, json=payload)

LLM observability

Learn more features of LLM observability.

Other LLM frameworks

Learn how to integrate Keywords AI with other LLM frameworks.

​
AI gateway

An AI gateway is a simple “middleman” between you and Large Language Models (LLMs). It handles things like:
  • Routing: Sending requests to different models or versions
  • Fallback models: If the primary model is down, the proxy will route to a fallback model
  • Load balancing: Distributing requests across multiple models
  • Caching: Caching responses to improve latency and reduce costs

​
1. Get API key

Sign in to Keywords AI platform, and get the API key from the API keys page.
Create API key placeholderCreate API key placeholder

​
2. Add credentials

You have to add your own credentials to activate AI gateway otherwise your LLM calls will cause errors. We will use your credentials to call LLMs on your behalf. We won’t use your credentials for any other purposes and no extra charges will be applied. Go to the Providers page to add your credentials.
Providers pageProviders page
Learn how to add credentials to a specific provider here.

​
3. Integrate AI gateway

Keywords AI offers various integration options, including: mainstream LLM frameworks and REST APIs.
If you are using OpenAI SDK, it’s pretty straightforward to integrate AI gateway into your codebase.
  • set the base_url to https://api.keywordsai.co/api
  • set the api_key to your Keywords AI API key Go to this page to learn details of OpenAI SDK Integration.
from openai import OpenAI

client = OpenAI(
    base_url="https://api.keywordsai.co/api/",
    api_key="YOUR_KEYWORDSAI_API_KEY",
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role":"user", "content":"Tell me a long story"}],
)

AI gateway

Learn more about AI gateway.

Other LLM frameworks

Learn how to integrate Keywords AI with other LLM frameworks.

​
Next steps

Congratulations! You have integrated LLM proxy into your codebase! When you make any API call, you will see the LLM log on the platform.

Call 300+ LLMs

Complete LLM observability

Prompt engineering