Skip to main content
GET
/
api
/
evaluators
/
List Evaluators
curl --request GET \
  --url https://api.keywordsai.co/api/evaluators/ \
  --header 'Authorization: Bearer <token>'
Returns a paginated list of evaluators for your organization.

​
Authentication

All endpoints require API key authentication: 
Authorization: Bearer YOUR_API_KEY

​
Query Parameters

ParameterTypeDescription
pageintegerPage number for pagination (default: 1)
page_sizeintegerNumber of items per page (default: 20)
searchstringSearch evaluators by name or slug
typestringFilter by evaluator type: llm, human, or code
score_value_typestringFilter by score type: numerical, boolean, categorical, or comment
starredbooleanFilter by starred status
tagsstringFilter by tags (comma-separated)

​
Examples

​
Basic List Request

import requests

url = "https://api.keywordsai.co/api/evaluators/"
headers = {
    "Authorization": "Bearer YOUR_API_KEY"
}

response = requests.get(url, headers=headers)
print(response.json())

​
With Filters and Pagination

import requests

url = "https://api.keywordsai.co/api/evaluators/"
headers = {
    "Authorization": "Bearer YOUR_API_KEY"
}

# Filter by type and score value type
params = {
    "type": "llm",
    "score_value_type": "numerical",
    "page": 1,
    "page_size": 10
}

response = requests.get(url, headers=headers, params=params)
print(response.json())

​
Search Evaluators

import requests

url = "https://api.keywordsai.co/api/evaluators/"
headers = {
    "Authorization": "Bearer YOUR_API_KEY"
}

# Search by name or slug
params = {
    "search": "quality",
    "starred": True
}

response = requests.get(url, headers=headers, params=params)
print(response.json())

​
Response

Status: 200 OK 
{
  "results": [
    {
      "id": "0f4325f9-55ef-4c20-8abe-376694419947",
      "name": "Response Quality Evaluator",
      "evaluator_slug": "response_quality_v1",
      "type": "llm",
      "score_value_type": "numerical",
      "eval_class": "",
      "description": "Evaluates response quality on a 1-5 scale",
      "created_by": {
        "first_name": "Keywords AI",
        "last_name": "Team",
        "email": "admin@keywordsai.co"
      },
      "updated_by": {
        "first_name": "Keywords AI",
        "last_name": "Team",
        "email": "admin@keywordsai.co"
      },
      "created_at": "2025-09-11T09:43:55.858321Z",
      "updated_at": "2025-09-11T09:43:55.858331Z",
      "custom_required_fields": [],
      "categorical_choices": null,
      "starred": false,
      "organization": 2,
      "tags": []
    },
    {
      "id": "cat-eval-123",
      "name": "Content Quality Assessment",
      "evaluator_slug": "content_quality_categorical",
      "type": "human",
      "score_value_type": "categorical",
      "eval_class": "",
      "description": "Human assessment of content quality with predefined categories",
      "created_by": {
        "first_name": "Keywords AI",
        "last_name": "Team",
        "email": "admin@keywordsai.co"
      },
      "updated_by": {
        "first_name": "Keywords AI",
        "last_name": "Team",
        "email": "admin@keywordsai.co"
      },
      "created_at": "2025-09-11T09:44:00.000000Z",
      "updated_at": "2025-09-11T09:44:00.000000Z",
      "custom_required_fields": [],
      "categorical_choices": [
        { "name": "Excellent", "value": 5 },
        { "name": "Good", "value": 4 },
        { "name": "Average", "value": 3 },
        { "name": "Poor", "value": 2 },
        { "name": "Very Poor", "value": 1 }
      ],
      "starred": false,
      "organization": 2,
      "tags": []
    },
    {
      "id": "bool-eval-456",
      "name": "Response Length Checker",
      "evaluator_slug": "length_checker_boolean",
      "type": "code",
      "score_value_type": "boolean",
      "eval_class": "",
      "description": "Checks if response meets minimum length requirement",
      "created_by": {
        "first_name": "Keywords AI",
        "last_name": "Team",
        "email": "admin@keywordsai.co"
      },
      "updated_by": {
        "first_name": "Keywords AI",
        "last_name": "Team",
        "email": "admin@keywordsai.co"
      },
      "created_at": "2025-09-11T09:45:00.000000Z",
      "updated_at": "2025-09-11T09:45:00.000000Z",
      "custom_required_fields": [],
      "categorical_choices": [],
      "starred": false,
      "organization": 2,
      "tags": ["automation", "validation"]
    }
  ],
  "count": 3,
  "previous": null,
  "next": null,
  "current_filters": {
    "type": null,
    "score_value_type": null,
    "search": null,
    "starred": null,
    "tags": null
  }
}

​
Response Fields

​
Evaluator Object

FieldTypeDescription
idstringUnique evaluator identifier
namestringDisplay name of the evaluator
evaluator_slugstringURL-friendly identifier
typestringEvaluator type: llm, human, or code
score_value_typestringScore format: numerical, boolean, categorical, or comment
eval_classstringPre-built template class (if used)
descriptionstringDescription of the evaluator
created_byobjectUser who created the evaluator
updated_byobjectUser who last updated the evaluator
created_atstringISO timestamp of creation
updated_atstringISO timestamp of last update
custom_required_fieldsarrayAdditional required fields
categorical_choicesarrayChoices for categorical evaluators
starredbooleanWhether the evaluator is starred
organizationintegerOrganization ID
tagsarrayTags associated with the evaluator

​
Pagination Object

FieldTypeDescription
countintegerTotal number of evaluators
previousstringURL for previous page (null if first page)
nextstringURL for next page (null if last page)
current_filtersobjectCurrently applied filters

​
Error Responses

​
401 Unauthorized

{
  "detail": "Your API key is invalid or expired, please check your API key at https://platform.keywordsai.co/platform/api/api-keys"
}

​
400 Bad Request

{
  "detail": "Invalid filter parameter: type must be one of 'llm', 'human', 'code'"
}