​
Get Evaluator

GET /api/evaluators/{evaluator_id}/ Retrieves detailed information about a specific evaluator, including full configuration.

​
Authentication

Requires API key authentication. Include your API key in the request headers: 
Authorization: Api-Key YOUR_API_KEY

​
Path Parameters

ParameterTypeDescription
evaluator_idstringThe unique ID of the evaluator

​
Examples

import requests

evaluator_id = "0f4325f9-55ef-4c20-8abe-376694419947"
url = f"https://api.keywordsai.co/api/evaluators/{evaluator_id}/"
headers = {
    "Authorization": "Api-Key YOUR_API_KEY"
}

response = requests.get(url, headers=headers)
print(response.json())

​
Response

Status: 200 OK

​
LLM Evaluator Response

{
  "id": "0f4325f9-55ef-4c20-8abe-376694419947",
  "name": "Response Quality Evaluator",
  "evaluator_slug": "response_quality_v1",
  "type": "llm",
  "score_value_type": "numerical",
  "eval_class": "",
  "description": "Evaluates response quality on a 1-5 scale",
  "configurations": {
    "evaluator_definition": "Rate the response quality based on accuracy, relevance, and completeness.\n<llm_input>{{llm_input}}</llm_input>\n<llm_output>{{llm_output}}</llm_output>",
    "scoring_rubric": "1=Poor, 2=Fair, 3=Good, 4=Very Good, 5=Excellent",
    "llm_engine": "gpt-4o-mini",
    "model_options": {
      "temperature": 0.1,
      "max_tokens": 200
    },
    "min_score": 1.0,
    "max_score": 5.0,
    "passing_score": 3.0
  },
  "created_by": {
    "first_name": "Keywords AI",
    "last_name": "Team",
    "email": "admin@keywordsai.co"
  },
  "updated_by": {
    "first_name": "Keywords AI",
    "last_name": "Team",
    "email": "admin@keywordsai.co"
  },
  "created_at": "2025-09-11T09:43:55.858321Z",
  "updated_at": "2025-09-11T09:43:55.858331Z",
  "custom_required_fields": [],
  "categorical_choices": null,
  "starred": false,
  "tags": []
}

​
Human Categorical Evaluator Response

{
  "id": "cat-eval-123",
  "name": "Content Quality Assessment",
  "evaluator_slug": "content_quality_categorical",
  "type": "human",
  "score_value_type": "categorical",
  "eval_class": "",
  "description": "Human assessment of content quality with predefined categories",
  "configurations": {},
  "created_by": {
    "first_name": "Keywords AI",
    "last_name": "Team",
    "email": "admin@keywordsai.co"
  },
  "updated_by": {
    "first_name": "Keywords AI",
    "last_name": "Team",
    "email": "admin@keywordsai.co"
  },
  "created_at": "2025-09-11T09:44:00.000000Z",
  "updated_at": "2025-09-11T09:44:00.000000Z",
  "custom_required_fields": [],
  "categorical_choices": [
    { "name": "Excellent", "value": 5 },
    { "name": "Good", "value": 4 },
    { "name": "Average", "value": 3 },
    { "name": "Poor", "value": 2 },
    { "name": "Very Poor", "value": 1 }
  ],
  "starred": false,
  "tags": []
}

​
Code Evaluator Response

{
  "id": "code-eval-456",
  "name": "Custom Length Check",
  "evaluator_slug": "custom_length_check",
  "type": "code",
  "score_value_type": "numerical",
  "eval_class": "",
  "description": "Evaluates response length appropriateness",
  "configurations": {
    "eval_code_snippet": "def evaluate(llm_input, llm_output, **kwargs):\n    length = len(llm_output.split())\n    if length < 10:\n        return 1\n    elif length < 50:\n        return 3\n    else:\n        return 5"
  },
  "created_by": {
    "first_name": "Keywords AI",
    "last_name": "Team",
    "email": "admin@keywordsai.co"
  },
  "updated_by": {
    "first_name": "Keywords AI",
    "last_name": "Team",
    "email": "admin@keywordsai.co"
  },
  "created_at": "2025-09-11T09:45:00.000000Z",
  "updated_at": "2025-09-11T09:45:00.000000Z",
  "custom_required_fields": [],
  "categorical_choices": [],
  "starred": false,
  "tags": []
}

​
Response Fields

FieldTypeDescription
idstringUnique evaluator identifier
namestringDisplay name of the evaluator
evaluator_slugstringUnique slug identifier
typestringEvaluator type: llm, human, or code
score_value_typestringScore type: numerical, boolean, categorical, or comment
eval_classstringPre-built template class (if used)
descriptionstringEvaluator description
configurationsobjectType-specific configuration settings
categorical_choicesarrayAvailable choices for categorical evaluators
created_byobjectUser who created the evaluator
updated_byobjectUser who last updated the evaluator
created_atstringISO timestamp of creation
updated_atstringISO timestamp of last update
custom_required_fieldsarrayAdditional required fields
starredbooleanWhether the evaluator is starred
tagsarrayAssociated tags

​
Error Responses

​
401 Unauthorized

{
  "detail": "Your API key is invalid or expired, please check your API key at https://platform.keywordsai.co/platform/api/api-keys"
}

​
404 Not Found

{
  "detail": "Not found."
}