Although LLMs are powerful, they are not perfect. They can make mistakes, and sometimes the mistakes are hard to detect. Human review is a way to ensure the quality of the LLM output and the accuracy of the AI evaluators.

​
Set up human evaluators

1

Create a new evaluator

You can set up an evaluator in Evaluators. Click the + New evaluator button, and select Human.

2

Configure the evaluator

Name the evaluator, and add a description. Then you can choose the type of the evaluation metric. We currently support Boolean and Numerical metrics.

3

Annotate LLM logs in Logs

You can annotate LLM logs in the side panel Scores ofLogs.