LLM proxy

​

Benefits of LLM proxy:

• Call over 200 LLMs using the same format.
• Gain comprehensive observability of your LLM applications.
• Ensure your LLM applications become more scalable and reliable.

​

Considerations:

• May not be suitable for applications with strict latency requirements (100 - 300ms added).
• May not be ideal for those who do not want to integrate a third-party service into the core of their application.

​

Steps:

​

Create an account

Go to Keywords AI’s sign up page to create an account. You can use your email or Google account to sign up.

​

Create an API key

To create an API key, visit the API key page

• Name your API key for future reference
• Pick an environment (test or production)
• Click the View code button to copy the code into your codebase

The API key will show up only once, Be careful and save it somewhere safe!

​

Add credentials

For the Free plan users, you have to use your own credentials to call LLMs. See how to add your credentials here.

​

Try Keywords AI with the OpenAI SDK

Install the OpenAI SDK

pip install openai 

​

Try Keywords AI with a standard API call

• URL change: Modify the API endpoint URL in your code from OpenAI’s URL to the Keywords AI endpoint URL: https://keywordsai.co/api/chat/completions.
• API key: Replace the OpenAI API key with your Keywords AI API key.
• Parameters: See supported parameters in the API reference.

​

Response

The results should be printed in the console.

{
  "id": "chatcmpl-8Ygj0WAGNhHBFjatPCefcPeNi12ct",
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": { "content": "Hello World", "role": "assistant" }
    }
  ],
  "created": 1703230636,
  "model": "gpt-3.5-turbo",
  "object": "chat.completion",
  "system_fingerprint": null,
  "usage": { "completion_tokens": 2, "prompt_tokens": 12, "total_tokens": 14 },
  "_response_ms": 653.2679999999999
}

Congratulations! You have finished the “Hello World” tutorial for Keywords AI API