Skip to main content
PATCH
/
api
/
evaluators
/
{evaluator_id}
/
Update Evaluator
curl --request PATCH \
  --url https://api.keywordsai.co/api/evaluators/{evaluator_id}/ \
  --header 'Authorization: Bearer <token>'
Updates specific fields of an evaluator. Supports partial updates of configuration fields.

​
Authentication

All endpoints require API key authentication: 
Authorization: Bearer YOUR_API_KEY

​
Path Parameters

ParameterTypeDescription
evaluator_idstringThe unique ID of the evaluator to update

​
Request Body

You can update any of the following fields. Only provide the fields you want to update:
FieldTypeDescription
namestringDisplay name for the evaluator
descriptionstringDescription of what the evaluator does
configurationsobjectType-specific configuration settings
categorical_choicesarrayChoices for categorical evaluators
custom_required_fieldsarrayAdditional required fields
starredbooleanWhether the evaluator is starred
tagsarrayTags for organization
Note: Configuration fields are merged with existing values. Non-null values take precedence over existing null values.

​
Examples

​
Update LLM Evaluator Configuration

import requests

evaluator_id = "0f4325f9-55ef-4c20-8abe-376694419947"
url = f"https://api.keywordsai.co/api/evaluators/{evaluator_id}/"
headers = {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json"
}

# Update scoring rubric and passing score
data = {
    "configurations": {
        "scoring_rubric": "Updated: 1=Very Poor, 2=Poor, 3=Fair, 4=Good, 5=Excellent",
        "passing_score": 4.0
    }
}

response = requests.patch(url, headers=headers, json=data)
print(response.json())

​
Update Name and Description

data = {
    "name": "Enhanced Response Quality Evaluator",
    "description": "Advanced evaluator for response quality assessment with updated criteria"
}

response = requests.patch(url, headers=headers, json=data)
print(response.json())

​
Update Categorical Choices

# For categorical evaluators
categorical_evaluator_id = "cat-eval-123"
url = f"https://api.keywordsai.co/api/evaluators/{categorical_evaluator_id}/"

data = {
    "categorical_choices": [
        { "name": "Outstanding", "value": 5 },
        { "name": "Very Good", "value": 4 },
        { "name": "Good", "value": 3 },
        { "name": "Fair", "value": 2 },
        { "name": "Poor", "value": 1 }
    ]
}

response = requests.patch(url, headers=headers, json=data)
print(response.json())

​
Update Code Evaluator

# For code evaluators
code_evaluator_id = "bool-eval-456"
url = f"https://api.keywordsai.co/api/evaluators/{code_evaluator_id}/"

data = {
    "name": "Enhanced Length Checker",
    "configurations": {
        "eval_code_snippet": "def evaluate(llm_input, llm_output, **kwargs):\n    '''\n    Enhanced length checker with word count\n    Returns True if response has >= 10 words, False otherwise\n    '''\n    if not llm_output:\n        return False\n    \n    word_count = len(llm_output.strip().split())\n    return word_count >= 10"
    }
}

response = requests.patch(url, headers=headers, json=data)
print(response.json())

​
Update Tags and Starred Status

data = {
    "starred": True,
    "tags": ["quality", "assessment", "production"]
}

response = requests.patch(url, headers=headers, json=data)
print(response.json())

​
Update LLM Engine and Model Options

data = {
    "configurations": {
        "llm_engine": "gpt-4o",
        "model_options": {
            "temperature": 0.2,
            "max_tokens": 300
        }
    }
}

response = requests.patch(url, headers=headers, json=data)
print(response.json())

​
Response

Status: 200 OK Returns the updated evaluator object with all current field values: 
{
  "id": "0f4325f9-55ef-4c20-8abe-376694419947",
  "name": "Enhanced Response Quality Evaluator",
  "evaluator_slug": "response_quality_v1",
  "type": "llm",
  "score_value_type": "numerical",
  "eval_class": "",
  "description": "Advanced evaluator for response quality assessment with updated criteria",
  "configurations": {
    "evaluator_definition": "Rate the response quality based on accuracy, relevance, and completeness.\n<llm_input>{{llm_input}}</llm_input>\n<llm_output>{{llm_output}}</llm_output>",
    "scoring_rubric": "Updated: 1=Very Poor, 2=Poor, 3=Fair, 4=Good, 5=Excellent",
    "llm_engine": "gpt-4o",
    "model_options": {
      "temperature": 0.2,
      "max_tokens": 300
    },
    "min_score": 1.0,
    "max_score": 5.0,
    "passing_score": 4.0
  },
  "created_by": {
    "first_name": "Keywords AI",
    "last_name": "Team",
    "email": "admin@keywordsai.co"
  },
  "updated_by": {
    "first_name": "Keywords AI",
    "last_name": "Team",
    "email": "admin@keywordsai.co"
  },
  "created_at": "2025-09-11T09:43:55.858321Z",
  "updated_at": "2025-09-11T10:15:22.123456Z",
  "custom_required_fields": [],
  "categorical_choices": null,
  "starred": true,
  "organization": 2,
  "tags": ["quality", "assessment", "production"]
}

​
Configuration Update Rules

​
Partial Configuration Updates

When updating configurations, the system performs a merge operation:
  • Existing configuration fields are preserved unless explicitly overridden
  • New fields are added to the configuration
  • Setting a field to null removes it from the configuration
  • Nested objects (like model_options) are completely replaced, not merged

​
Example Configuration Merge

Original Configuration: 
{
  "evaluator_definition": "Original prompt",
  "scoring_rubric": "Original rubric",
  "llm_engine": "gpt-4o-mini",
  "min_score": 1.0,
  "max_score": 5.0
}
Update Request: 
{
  "configurations": {
    "scoring_rubric": "Updated rubric",
    "passing_score": 3.0
  }
}
Resulting Configuration: 
{
  "evaluator_definition": "Original prompt",
  "scoring_rubric": "Updated rubric",
  "llm_engine": "gpt-4o-mini",
  "min_score": 1.0,
  "max_score": 5.0,
  "passing_score": 3.0
}

​
Error Responses

​
400 Bad Request

{
  "configurations": [
    "Configuration validation failed: llm_engine 'invalid-model' is not supported"
  ]
}

​
401 Unauthorized

{
  "detail": "Your API key is invalid or expired, please check your API key at https://platform.keywordsai.co/platform/api/api-keys"
}

​
403 Forbidden

{
  "detail": "You do not have permission to update this evaluator."
}

​
404 Not Found

{
  "detail": "Not found."
}

​
422 Unprocessable Entity

{
  "categorical_choices": [
    "This field is required when score_value_type is 'categorical'."
  ]
}
⌘I