The user-level rate limit helps you control the LLM usage of each user.
This feature is only available for AI gateway users.
The user-level rate limit helps you control the LLM usage of each user. You can set a rate limit of requests per minute for each user and we will block the API calls that exceed the limit.
"customer_params":{"customer_identifier":"xxxx",// The user you want to set the rate limit for"rate_limit":100// The rate limit of the user, requests per minute},
from openai import OpenAIclient = OpenAI( base_url="https://api.keywordsai.co/api/", api_key=YOUR_KEYWORDSAI_API_KEY,)response = client.chat.completions.create( model="gpt-4o-mini", messages=[{"role":"user","content":"Tell me a long story"}], extra_body={"customer_params":{"customer_identifier":"xxxx","rate_limit":100}})
We also support user-level rate limit in other SDKs or languages, please check out our integration section for more information.