Skip to main content
POST
/
api
/
request-logs
/
create
/
import requests

url = "https://api.keywordsai.co/api/request-logs/create/"
headers = {
    "Authorization": "Bearer YOUR_KEYWORDS_AI_API_KEY",
    "Content-Type": "application/json"
}
payload = {
    "model": "gpt-4",
    "prompt_messages": [
        {
          "role": "user",
          "content": "Hi"
        },
        {
          "role": "assistant",
          "content": None,
          "tool_calls": [
            {
              "id": "xxxx",
              "type": "function",
              "function": {
                "name": "get_current_weather", # Function name
                "arguments": "{\n\"location\": \"Boston, MA\"\n}" # Function arguments
              }
            }
          ]
        }, #optional
    ],
    "completion_message": {
        "role": "assistant",
        "content": "Hi, how can I assist you today?"
    },
    "tool_choice": {
        "type": "function",
        "function": {
            "name": "get_current_weather"
        }
    },
    "tools":[
      {
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "description": "Get the current weather in a given location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state, e.g. San Francisco, CA",
            },
            "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
          },
          "required": ["location"],
        },
      },
      }
    ],
    "customer_params": {
        "customer_identifier": "customer_123",
        "name": "Hendrix Liu", # optional
        "email": "hendrix@keywordsai.co" # optional
    },
    "prompt_tokens": 8,
    "completion_tokens": 16,
    "cost": 0.00042,
    "latency": 0.0,
    "timestamp": "2024-04-15T08:30:37.721313Z",
    "time_to_first_token": 0.0,
    "metadata": {},
    "stream": False,
    "status_code": 200,
    "warnings": "",
    "error_message": "",
    "type":"text", # "json_schema", "json_object"
}
response = requests.request("POST", url, headers=headers, json=payload)
This guide shows you how to log LLM requests to Keywords AI using the structured 3-layer approach for comprehensive LLM logging and monitoring.

​
Parameters

These are the essential parameters needed for basic LLM request logging.

​
Core required fields

​
model
string
required
The LLM model used for the inference. See the list of supported models here.
​
prompt_messages
array
required
An array of prompt messages sent to the model.
​
role
string
required
The role of the message. Could be system, developer, user, assistant, tool.
​
content
string
required
The content of the message.
​
tool_call_id
string
The tool call id for tool messages.
"prompt_messages": [
    {
        "role": "system",
        "content": "You are a helpful assistant."
    },
    {
        "role": "user", 
        "content": "What is machine learning?"
    },
    # optional tool call
    {
        "role": "tool",
        "tool_call_id": "your tool call id",
        "content": "...." # tool call content
    }
]
​
completion_message
object
The model’s response message in JSON format.
"completion_message": {
    "role": "assistant",
    "content": "Machine learning is a subset of artificial intelligence..."
}

​
Telemetry

Performance metrics and cost tracking for monitoring LLM efficiency.
​
prompt_tokens
integer
Number of tokens in the prompt.
​
completion_tokens
integer
Number of tokens in the completion.
​
cost
float
default:0
Cost of the inference in US dollars.
​
generation_time
float
default:0
Total generation time. Generation time = TTFT (Time To First Token) + TPOT (Time Per Output Token) * #tokens. Do not confuse this with ttft.
The unit of generation time is seconds.
​
ttft
float
default:0
Time to first token. The time it takes for the model to generate the first token after receiving a request.
The unit of ttft is seconds.

​
Metadata

Custom tracking and identification parameters for advanced analytics and filtering.
​
metadata
dict
You can add any key-value pair to this metadata field for your reference.
{
  "metadata": {
    "language": "en",
    "environment": "production",
    "version": "v1.0.0",
    "feature": "chat_support"
  }
}
​
customer_params
object
Parameters related to the customer.
​
customer_identifier
string
required
An identifier for the customer that invoked this LLM inference, helps with visualizing user activities. See the details of customer identifier here.
​
name
string
Name of the customer.
​
email
string
Email of the customer.
{
  "customer_params": {
    "customer_identifier": "customer_123",
    "name": "John Doe",
    "email": "john.doe@example.com"
  }
}
​
group_identifier
string
Group identifier. Use group identifier to group logs together.
​
thread_identifier
string
A unique identifier for the thread.
​
custom_identifier
string
Same functionality as metadata, but it’s faster to query since it’s indexed.
{
  "custom_identifier": "my_value"
}

​
Advanced Parameters

​
Tool Calls and Function Calling

​
tools
array
A list of tools the model may call. Currently, only functions are supported as a tool.
​
type
string
required
The type of the tool. Currently, only function is supported.
​
function
object
required
​
name
string
required
The name of the function.
​
description
string
A description of what the function does.
​
parameters
object
The parameters the function accepts.
"tools": [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA",
                    },
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                },
                "required": ["location"],
            },
        },
    }
]
​
tool_choice
object
Controls which (if any) tool is called by the model. none means the model will not call any tool and instead generates a message.
​
type
string
required
The type of the tool. Currently, only function is supported.
​
function
object
required
​
name
string
required
The name of the function to call.
"tool_choice": {
    "type": "function",
    "function": {
        "name": "get_current_weather"
    }
}

​
Response Configuration

​
response_format
object
Setting to { "type": "json_schema", "json_schema": {...} } enables Structured Outputs which ensures the model will match your supplied JSON schema.
​
Text
Object
Default response format. Used to generate text responses.
​
type
string
required
The type of response format being defined. Always text.
​
JSON Schema
Object
JSON Schema response format. Used to generate structured JSON responses.
​
type
string
required
The type of response format being defined. Always json_schema.
​
json_schema
object
required
Structured Outputs configuration options, including a JSON Schema.
​
JSON Object
Object
JSON object response format. An older method of generating JSON responses.
​
type
string
required
The type of response format being defined. Always json_object.

​
Model Configuration

​
temperature
number
default:1
Controls randomness in the output in the range of 0-2, higher temperature will result in more random responses.
​
top_p
number
default:1
An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.We generally recommend altering this or temperature but not both.
​
frequency_penalty
number
Penalizes new tokens based on their frequency in the text so far. Decreases the model’s likelihood of repeating the same line verbatim.
​
presence_penalty
number
Penalizes new tokens based on whether they appear in the text so far. Increases the model’s likelihood of talking about new topics.
​
stop
array[string]
Stop sequences where the API will stop generating further tokens.

​
Error Handling and Status

​
status_code
integer
default:200
The status code of the LLM inference. Default is 200 (ok). See supported status codes here.
We support all status codes that are valid HTTP status codes. 200,201,204,301,304,400, 401,402,403,404,405,415,422,429,500,502,503,504 etc.
​
error_message
text
Error message if the LLM inference failed. Default is an empty string.
​
warnings
string
Any warnings that occurred during the LLM inference. You could pass a warning message here. Default is an empty string.

​
Additional Configuration

​
stream
boolean
Whether the LLM inference was streamed. Default is false.
​
is_custom_prompt
boolean
default:false
Whether the prompt is a custom prompt. Default is False.
​
prompt_id
string
ID of the prompt. If you want to log a custom prompt ID, you need to pass is_custom_prompt as True. Otherwise, use the Prompt ID in Prompts.
​
prompt_name
string
Name of the prompt.
​
full_request
object
The full request object. Default is an empty dictionary. This is optional and it is helpful for logging configurations such as temperature, presence_penalty etc.
completion_messages, tool_calls will be automatically extracted from full_request
{
    "full_request": {
        "temperature": 0.5,
        "top_p": 0.5,
        //... other parameters
    }
}
​
keywordsai_api_controls
object
Use this parameter to control the behavior of the Keywords AI API. Default is an empty dictionary.
​
block
boolean
default:true
If false, the server will immediately return a status of whether the logging task is initialized successfully with no log data.
{
  "keywordsai_api_controls": {
    "block": true // true or false.
  }
//...other params...
}

​
Pricing Configuration

​
prompt_unit_price
number
Pass this parameter in if you want to log your self-host / fine-tuned model.
{
"prompt_unit_price": 0.0042 // $0.0042 per 1M tokens
}
​
completion_unit_price
number
Pass this parameter in if you want to log your self-host / fine-tuned model.
{
"completion_unit_price": 0.0042 // $0.0042 per 1M tokens
}

​
Usage Details

​
usage
object
Usage details for the LLM inference. Currently, only support Prompt Caching.
​
prompt_tokens_details
object
​
cached_tokens
integer
Number of tokens in the prompt.
​
cache_creation_prompt_tokens
integer
This parameter is only applicable for Anthropic models. It represents the number of tokens used to create the cache.
{
  "usage": {
      "prompt_tokens_details": {
          "cached_tokens": 10,
      },
      "cache_creation_prompt_tokens": 20, // Anthropic only
  }
}
​
positive_feedback
boolean
Whether the user liked the output. True means the user liked the output.
import requests

url = "https://api.keywordsai.co/api/request-logs/create/"
headers = {
    "Authorization": "Bearer YOUR_KEYWORDS_AI_API_KEY",
    "Content-Type": "application/json"
}
payload = {
    "model": "gpt-4",
    "prompt_messages": [
        {
          "role": "user",
          "content": "Hi"
        },
        {
          "role": "assistant",
          "content": None,
          "tool_calls": [
            {
              "id": "xxxx",
              "type": "function",
              "function": {
                "name": "get_current_weather", # Function name
                "arguments": "{\n\"location\": \"Boston, MA\"\n}" # Function arguments
              }
            }
          ]
        }, #optional
    ],
    "completion_message": {
        "role": "assistant",
        "content": "Hi, how can I assist you today?"
    },
    "tool_choice": {
        "type": "function",
        "function": {
            "name": "get_current_weather"
        }
    },
    "tools":[
      {
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "description": "Get the current weather in a given location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state, e.g. San Francisco, CA",
            },
            "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
          },
          "required": ["location"],
        },
      },
      }
    ],
    "customer_params": {
        "customer_identifier": "customer_123",
        "name": "Hendrix Liu", # optional
        "email": "hendrix@keywordsai.co" # optional
    },
    "prompt_tokens": 8,
    "completion_tokens": 16,
    "cost": 0.00042,
    "latency": 0.0,
    "timestamp": "2024-04-15T08:30:37.721313Z",
    "time_to_first_token": 0.0,
    "metadata": {},
    "stream": False,
    "status_code": 200,
    "warnings": "",
    "error_message": "",
    "type":"text", # "json_schema", "json_object"
}
response = requests.request("POST", url, headers=headers, json=payload)
⌘I