Skip to main content
GET
/
api
/
evaluators
/
{evaluator_id}
/
Get Evaluator
curl --request GET \
  --url https://api.keywordsai.co/api/evaluators/{evaluator_id}/ \
  --header 'Authorization: Bearer <token>'
Retrieves detailed information about a specific evaluator, including full configuration.

​
Authentication

All endpoints require API key authentication: 
Authorization: Bearer YOUR_API_KEY

​
Path Parameters

ParameterTypeDescription
evaluator_idstringThe unique ID of the evaluator

​
Examples

import requests

evaluator_id = "0f4325f9-55ef-4c20-8abe-376694419947"
url = f"https://api.keywordsai.co/api/evaluators/{evaluator_id}/"
headers = {
    "Authorization": "Bearer YOUR_API_KEY"
}

response = requests.get(url, headers=headers)
print(response.json())

​
Response

Status: 200 OK

​
LLM Evaluator Response

{
  "id": "0f4325f9-55ef-4c20-8abe-376694419947",
  "name": "Response Quality Evaluator",
  "evaluator_slug": "response_quality_v1",
  "type": "llm",
  "score_value_type": "numerical",
  "eval_class": "",
  "description": "Evaluates response quality on a 1-5 scale",
  "configurations": {
    "evaluator_definition": "Rate the response quality based on accuracy, relevance, and completeness.\n<llm_input>{{llm_input}}</llm_input>\n<llm_output>{{llm_output}}</llm_output>",
    "scoring_rubric": "1=Poor, 2=Fair, 3=Good, 4=Very Good, 5=Excellent",
    "llm_engine": "gpt-4o-mini",
    "model_options": {
      "temperature": 0.1,
      "max_tokens": 200
    },
    "min_score": 1.0,
    "max_score": 5.0,
    "passing_score": 3.0
  },
  "created_by": {
    "first_name": "Keywords AI",
    "last_name": "Team",
    "email": "admin@keywordsai.co"
  },
  "updated_by": {
    "first_name": "Keywords AI",
    "last_name": "Team",
    "email": "admin@keywordsai.co"
  },
  "created_at": "2025-09-11T09:43:55.858321Z",
  "updated_at": "2025-09-11T09:43:55.858331Z",
  "custom_required_fields": [],
  "categorical_choices": null,
  "starred": false,
  "organization": 2,
  "tags": []
}

​
Human Categorical Evaluator Response

{
  "id": "cat-eval-123",
  "name": "Content Quality Assessment",
  "evaluator_slug": "content_quality_categorical",
  "type": "human",
  "score_value_type": "categorical",
  "eval_class": "",
  "description": "Human assessment of content quality with predefined categories",
  "configurations": {},
  "categorical_choices": [
    { "name": "Excellent", "value": 5 },
    { "name": "Good", "value": 4 },
    { "name": "Average", "value": 3 },
    { "name": "Poor", "value": 2 },
    { "name": "Very Poor", "value": 1 }
  ],
  "created_by": {
    "first_name": "Keywords AI",
    "last_name": "Team",
    "email": "admin@keywordsai.co"
  },
  "updated_by": {
    "first_name": "Keywords AI",
    "last_name": "Team",
    "email": "admin@keywordsai.co"
  },
  "created_at": "2025-09-11T09:44:00.000000Z",
  "updated_at": "2025-09-11T09:44:00.000000Z",
  "custom_required_fields": [],
  "starred": false,
  "organization": 2,
  "tags": []
}

​
Code Evaluator Response

{
  "id": "bool-eval-456",
  "name": "Response Length Checker",
  "evaluator_slug": "length_checker_boolean",
  "type": "code",
  "score_value_type": "boolean",
  "eval_class": "",
  "description": "Checks if response meets minimum length requirement",
  "configurations": {
    "eval_code_snippet": "def evaluate(llm_input, llm_output, **kwargs):\n    '''\n    Check if response meets minimum length requirement\n    Returns True if length >= 50 characters, False otherwise\n    '''\n    if not llm_output:\n        return False\n    \n    return len(llm_output.strip()) >= 50"
  },
  "created_by": {
    "first_name": "Keywords AI",
    "last_name": "Team",
    "email": "admin@keywordsai.co"
  },
  "updated_by": {
    "first_name": "Keywords AI",
    "last_name": "Team",
    "email": "admin@keywordsai.co"
  },
  "created_at": "2025-09-11T09:45:00.000000Z",
  "updated_at": "2025-09-11T09:45:00.000000Z",
  "custom_required_fields": [],
  "categorical_choices": [],
  "starred": false,
  "organization": 2,
  "tags": ["automation", "validation"]
}

​
Response Fields

FieldTypeDescription
idstringUnique evaluator identifier
namestringDisplay name of the evaluator
evaluator_slugstringURL-friendly identifier
typestringEvaluator type: llm, human, or code
score_value_typestringScore format: numerical, boolean, categorical, or comment
eval_classstringPre-built template class (if used)
descriptionstringDescription of the evaluator
configurationsobjectType-specific configuration settings
categorical_choicesarrayChoices for categorical evaluators (null for non-categorical)
created_byobjectUser who created the evaluator
updated_byobjectUser who last updated the evaluator
created_atstringISO timestamp of creation
updated_atstringISO timestamp of last update
custom_required_fieldsarrayAdditional required fields
starredbooleanWhether the evaluator is starred
organizationintegerOrganization ID
tagsarrayTags associated with the evaluator

​
Configuration Fields by Type

​
LLM Evaluators (type: "llm")

FieldTypeDescription
evaluator_definitionstringThe evaluation prompt/instruction with template variables
scoring_rubricstringDescription of the scoring criteria
llm_enginestringLLM model to use (e.g., “gpt-4o-mini”, “gpt-4o”)
model_optionsobjectLLM parameters like temperature, max_tokens
min_scorenumberMinimum possible score
max_scorenumberMaximum possible score
passing_scorenumberScore threshold for passing

​
Code Evaluators (type: "code")

FieldTypeDescription
eval_code_snippetstringPython code with evaluate() function

​
Human Evaluators (type: "human")

  • No specific configuration fields
  • Use categorical_choices field when score_value_type is "categorical"

​
Error Responses

​
404 Not Found

{
  "detail": "Not found."
}

​
401 Unauthorized

{
  "detail": "Your API key is invalid or expired, please check your API key at https://platform.keywordsai.co/platform/api/api-keys"
}

​
403 Forbidden

{
  "detail": "You do not have permission to access this evaluator."
}