Skip to main content
GET
/
api
/
evaluators
/
List Evaluators
curl --request GET \
  --url https://api.keywordsai.co/api/evaluators/ \
  --header 'Authorization: Bearer <token>'
Returns a paginated list of evaluators for your organization.

Authentication

All endpoints require API key authentication:
Authorization: Bearer YOUR_API_KEY

Query Parameters

ParameterTypeDescription
pageintegerPage number for pagination (default: 1)
page_sizeintegerNumber of items per page (default: 20)
searchstringSearch evaluators by name or slug
typestringFilter by evaluator type: llm, human, or code
score_value_typestringFilter by score type: numerical, boolean, categorical, or comment
starredbooleanFilter by starred status
tagsstringFilter by tags (comma-separated)

Examples

Basic List Request

import requests

url = "https://api.keywordsai.co/api/evaluators/"
headers = {
    "Authorization": "Bearer YOUR_API_KEY"
}

response = requests.get(url, headers=headers)
print(response.json())

With Filters and Pagination

import requests

url = "https://api.keywordsai.co/api/evaluators/"
headers = {
    "Authorization": "Bearer YOUR_API_KEY"
}

# Filter by type and score value type
params = {
    "type": "llm",
    "score_value_type": "numerical",
    "page": 1,
    "page_size": 10
}

response = requests.get(url, headers=headers, params=params)
print(response.json())

Search Evaluators

import requests

url = "https://api.keywordsai.co/api/evaluators/"
headers = {
    "Authorization": "Bearer YOUR_API_KEY"
}

# Search by name or slug
params = {
    "search": "quality",
    "starred": True
}

response = requests.get(url, headers=headers, params=params)
print(response.json())

Response

Status: 200 OK
{
  "results": [
    {
      "id": "0f4325f9-55ef-4c20-8abe-376694419947",
      "name": "Response Quality Evaluator",
      "evaluator_slug": "response_quality_v1",
      "type": "llm",
      "score_value_type": "numerical",
      "eval_class": "",
      "description": "Evaluates response quality on a 1-5 scale",
      "created_by": {
        "first_name": "Keywords AI",
        "last_name": "Team",
        "email": "admin@keywordsai.co"
      },
      "updated_by": {
        "first_name": "Keywords AI",
        "last_name": "Team",
        "email": "admin@keywordsai.co"
      },
      "created_at": "2025-09-11T09:43:55.858321Z",
      "updated_at": "2025-09-11T09:43:55.858331Z",
      "custom_required_fields": [],
      "categorical_choices": null,
      "starred": false,
      "organization": 2,
      "tags": []
    },
    {
      "id": "cat-eval-123",
      "name": "Content Quality Assessment",
      "evaluator_slug": "content_quality_categorical",
      "type": "human",
      "score_value_type": "categorical",
      "eval_class": "",
      "description": "Human assessment of content quality with predefined categories",
      "created_by": {
        "first_name": "Keywords AI",
        "last_name": "Team",
        "email": "admin@keywordsai.co"
      },
      "updated_by": {
        "first_name": "Keywords AI",
        "last_name": "Team",
        "email": "admin@keywordsai.co"
      },
      "created_at": "2025-09-11T09:44:00.000000Z",
      "updated_at": "2025-09-11T09:44:00.000000Z",
      "custom_required_fields": [],
      "categorical_choices": [
        { "name": "Excellent", "value": 5 },
        { "name": "Good", "value": 4 },
        { "name": "Average", "value": 3 },
        { "name": "Poor", "value": 2 },
        { "name": "Very Poor", "value": 1 }
      ],
      "starred": false,
      "organization": 2,
      "tags": []
    },
    {
      "id": "bool-eval-456",
      "name": "Response Length Checker",
      "evaluator_slug": "length_checker_boolean",
      "type": "code",
      "score_value_type": "boolean",
      "eval_class": "",
      "description": "Checks if response meets minimum length requirement",
      "created_by": {
        "first_name": "Keywords AI",
        "last_name": "Team",
        "email": "admin@keywordsai.co"
      },
      "updated_by": {
        "first_name": "Keywords AI",
        "last_name": "Team",
        "email": "admin@keywordsai.co"
      },
      "created_at": "2025-09-11T09:45:00.000000Z",
      "updated_at": "2025-09-11T09:45:00.000000Z",
      "custom_required_fields": [],
      "categorical_choices": [],
      "starred": false,
      "organization": 2,
      "tags": ["automation", "validation"]
    }
  ],
  "count": 3,
  "previous": null,
  "next": null,
  "current_filters": {
    "type": null,
    "score_value_type": null,
    "search": null,
    "starred": null,
    "tags": null
  }
}

Response Fields

Evaluator Object

FieldTypeDescription
idstringUnique evaluator identifier
namestringDisplay name of the evaluator
evaluator_slugstringURL-friendly identifier
typestringEvaluator type: llm, human, or code
score_value_typestringScore format: numerical, boolean, categorical, or comment
eval_classstringPre-built template class (if used)
descriptionstringDescription of the evaluator
created_byobjectUser who created the evaluator
updated_byobjectUser who last updated the evaluator
created_atstringISO timestamp of creation
updated_atstringISO timestamp of last update
custom_required_fieldsarrayAdditional required fields
categorical_choicesarrayChoices for categorical evaluators
starredbooleanWhether the evaluator is starred
organizationintegerOrganization ID
tagsarrayTags associated with the evaluator

Pagination Object

FieldTypeDescription
countintegerTotal number of evaluators
previousstringURL for previous page (null if first page)
nextstringURL for next page (null if last page)
current_filtersobjectCurrently applied filters

Error Responses

401 Unauthorized

{
  "detail": "Your API key is invalid or expired, please check your API key at https://platform.keywordsai.co/platform/api/api-keys"
}

400 Bad Request

{
  "detail": "Invalid filter parameter: type must be one of 'llm', 'human', 'code'"
}
I