Usage dashboard

Keywords AI’s dashboard page assists developers in identifying and fixing code or LLM-related issues.

You can see aggregated metrics such as the number of requests over time, average generation time, time to first token (TTFT), total cost, average number of tokens, and user activity.

​

Graphs

​

LLM metrics

LLM graphs aggregate your API calls and trace them through time, making it easy to visually keep track of your API usage.

​

Users

User graphs provide an overview of the user information being sent to Keywords AI.

You can tag each of your API calls with a customer_identifier and we will keep track of the usage of each of your users. See examples here: User data

​

Organization metrics

You could monitor your organization’s:

• API usage and spending (see how to manage your API keys here)
• Top users (see how to tag your users here)
• Top models

​

Evaluation

Evaluation graphs provide an overview of the performance of LLM output based on Keywords AI’s built-in metrics and users’ customized evaluation metrics.

See how to turn on evaluations and create your evaluation metrics here: Evaluation