Authorization

Docs

Documentation

API reference

Changelog ↗

Get started

Discord

Vercel SDK

Welcome to Keywords AI, the leading LLM monitoring platform.

What is Keywords AI

Log your LLM requests and responses asynchronously

Async logging

Call 200+ LLMs using the same input/output format with 2 lines of code

LLM proxy

OpenAI SDK

Anthropic

LangChain

LlamaIndex

You can log your custom model's outputs through our logging endpoint

Custom model

Supported models

Use your own API keys through Keywords AI

Overview

Use your own Azure OpenAI credits through Keywords AI

Azure OpenAI

Use your own OpenAI credits through Keywords AI

OpenAI

Use your own Anthropic credits through Keywords AI

Use your own Vertex AI credits through Keywords AI

Vertex AI

Use your own AWS credits through Keywords AI

AWS Bedrock

Use your own Groq credits through Keywords AI

Groq

Use your own Fireworks credits through Keywords AI

Fireworks

Use your own Together AI credits through Keywords AI

Together AI

Use your own Perplexity AI credits through Keywords AI

Perplexity AI

Use your own OpenRouter credits through Keywords AI

OpenRouter

PostHog is an open-source product analytics platform, you can monitor your product traffic and user sessions with PostHog. Now you can connect PostHog with Keywords AI to monitor your LLM performance and get observability.

PostHog

BAML is a domain-specific language to write and test LLM functions. Use BAML with Keywords AI to get complete LLM observability

BAML

Mem0 provides a memory layer for LLM applications, enabling them to remember and learn from user interactions over time.

Mem0

The best LLM dashboard to monitor your API usage and spending.

Usage dashboard

Monitor and debug your LLM requests with the Keywords AI logs. You can see details of every LLM request and filter, export, and share logs with your team.

LLM logs

Update logs

Disable logs

Pass your customer's identifier to monitor the user's data in the Keywords AI platform.

Customer

Custom metadata

Threads allow you to group chatlogs together as a single conversation thread.

Threads

Subscribe to system status and get notified via email when an LLM outage is detected.

Alerts

Receive notifications when certain events occur.

Webhooks

Call 200+ LLMs with a single OpenAI compatible format

Integrate 200+ LLMs

API keys management

Fallbacks

Increase your LLM rate limits with our load balancing feature.

Load balancing

Reduce latency and save LLM costs by caching LLM prompts and responses.

Caches

LLM caches

When an LLM call fails, our system detects the error and retries the request to prevent failovers.

Retries

Create/update users to have better user observability on Keywords AI

Create a user

The users page helps you keep track of your users. You can tag each of your API calls and we will keep track of the usage of each of your users.

User data

Keep track of your users' usage and set limit on their budget

User budget

Evaluate your prompts with the help of LLM

LLM-as-a-judge

Iterating and versioning prompts as a team.

Prompts

The best LLM playground to test and compare models

LLM playground

A spreadsheet-like interface built for multi-config prompt testing

Easily manage and organize test cases. Import a CSV file and edit it like a Google Sheet.

Testsets

How to create your own golden datasets from logs.

Datasets

Async logging (recommended)

LLM Proxy

Batch logging

Log list

Prompt

Prompt version

Text to speech

Speech to text

Embeddings

User creation

User list

User detail

User update

Evaluations

Model list

400: Bad request

401: Unauthorized

402: Payment required

403: Forbidden

404: Not found

408: Request timeout

429: Rate limit exceeded

500: Internal server error

Discover the importance of LLM monitoring in AI. Learn how it ensures accuracy, performance, and reliability while reducing costs. Keywords AI offers top-tier LLM monitoring tools.

What is LLM monitoring?

Retry failed request due to busy upstream provider

Get started

Proxy integrations

Integrations

Features

Error handling

Resources

Vercel SDK