In this guide, we will show you how to add an LLM evaluator in the code.
completion_message
: The completion message from the LLM.eval_params
: The parameters for the evaluator.evaluators
: The parameter under the eval_params
.evaluator_slug
: The parameter under the evaluators
, this is the slug of the evaluator you want to run.ideal_output
can be also said as the expected output. You can add it to let the evaluator know what is the expected/right output.
ideal_output
to the description of the evaluator in the UI. Otherwise, the evaluator will not use the ideal_output
from the code.ideal_output
under the eval_inputs
as shown in the example code below:
eval_params
: The parameters for the evaluator.evaluators
: The parameter under the eval_params
.evaluator_slug
: The parameter under the evaluators
, this is the slug of the evaluator you want to run.ideal_output
can be also said as the expected output. You can add it to let the evaluator know what is the expected/right output.
ideal_output
to the description of the evaluator in the UI. Otherwise, the evaluator will not use the ideal_output
from the code.ideal_output
under the eval_inputs
as shown in the example code below:
extra_body
.