POST
/
api
/
request-logs
/
create
/
import requests

url = "https://api.keywordsai.co/api/request-logs/create/"
headers = {
    "Authorization": "Bearer YOUR_KEYWORDS_AI_API_KEY",
    "Content-Type": "application/json"
}
payload = {
    "model": "gpt-4",
    "prompt_messages": [
        {
          "role": "user",
          "content": "Hi"
        },
        {
          "role": "assistant",
          "content": None,
          "tool_calls": [
            {
              "id": "xxxx",
              "type": "function",
              "function": {
                "name": "get_current_weather", # Function name
                "arguments": "{\n\"location\": \"Boston, MA\"\n}" # Function arguments
              }
            }
          ]
        }, #optional
    ],
    "completion_message": {
        "role": "assistant",
        "content": "Hi, how can I assist you today?"
    },
    "tool_choice": {
        "type": "function",
        "function": {
            "name": "get_current_weather"
        }
    },
    "tools":[
      {
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "description": "Get the current weather in a given location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state, e.g. San Francisco, CA",
            },
            "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
          },
          "required": ["location"],
        },
      },
      }
    ],
    "customer_params": {
        "customer_identifier": "customer_123",
        "name": "Hendrix Liu", # optional
        "email": "hendrix@keywordsai.co" # optional
    },
    "prompt_tokens": 8,
    "completion_tokens": 16,
    "cost": 0.00042,
    "latency": 0.0,
    "timestamp": "2024-04-15T08:30:37.721313Z",
    "time_to_first_token": 0.0,
    "metadata": {},
    "stream": False,
    "status_code": 200,
    "warnings": "",
    "error_message": "",
    "type":"text", # "json_schema", "json_object"
}
response = requests.request("POST", url, headers=headers, json=payload)
This guide shows you how to log LLM requests to Keywords AI using the structured 3-layer approach for comprehensive LLM logging and monitoring.

​
Parameters

These are the essential parameters needed for basic LLM request logging.

​
Core required fields

​
model
string
required
The LLM model used for the inference. See the list of supported models here.
​
prompt_messages
array
required
An array of prompt messages sent to the model.
​
completion_message
object
The model’s response message in JSON format.

​
Telemetry

Performance metrics and cost tracking for monitoring LLM efficiency.
​
prompt_tokens
integer
Number of tokens in the prompt.
​
completion_tokens
integer
Number of tokens in the completion.
​
cost
float
default:0
Cost of the inference in US dollars.
​
generation_time
float
default:0
Total generation time. Generation time = TTFT (Time To First Token) + TPOT (Time Per Output Token) * #tokens. Do not confuse this with ttft.
The unit of generation time is seconds.
​
ttft
float
default:0
Time to first token. The time it takes for the model to generate the first token after receiving a request.
The unit of ttft is seconds.

​
Metadata

Custom tracking and identification parameters for advanced analytics and filtering.
​
metadata
dict
You can add any key-value pair to this metadata field for your reference.
​
customer_params
object
Parameters related to the customer.
​
group_identifier
string
Group identifier. Use group identifier to group logs together.
​
thread_identifier
string
A unique identifier for the thread.
​
custom_identifier
string
Same functionality as metadata, but it’s faster to query since it’s indexed.

​
Advanced Parameters

​
Tool Calls and Function Calling

​
tools
array
A list of tools the model may call. Currently, only functions are supported as a tool.
​
tool_choice
object
Controls which (if any) tool is called by the model. none means the model will not call any tool and instead generates a message.

​
Response Configuration

​
response_format
object
Setting to { "type": "json_schema", "json_schema": {...} } enables Structured Outputs which ensures the model will match your supplied JSON schema.

​
Model Configuration

​
temperature
number
default:1
Controls randomness in the output in the range of 0-2, higher temperature will result in more random responses.
​
top_p
number
default:1
An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.We generally recommend altering this or temperature but not both.
​
frequency_penalty
number
Penalizes new tokens based on their frequency in the text so far. Decreases the model’s likelihood of repeating the same line verbatim.
​
presence_penalty
number
Penalizes new tokens based on whether they appear in the text so far. Increases the model’s likelihood of talking about new topics.
​
stop
array[string]
Stop sequences where the API will stop generating further tokens.

​
Error Handling and Status

​
status_code
integer
default:200
The status code of the LLM inference. Default is 200 (ok). See supported status codes here.
​
error_message
text
Error message if the LLM inference failed. Default is an empty string.
​
warnings
string
Any warnings that occurred during the LLM inference. You could pass a warning message here. Default is an empty string.

​
Additional Configuration

​
stream
boolean
Whether the LLM inference was streamed. Default is false.
​
is_custom_prompt
boolean
default:false
Whether the prompt is a custom prompt. Default is False.
​
prompt_id
string
ID of the prompt. If you want to log a custom prompt ID, you need to pass is_custom_prompt as True. Otherwise, use the Prompt ID in Prompts.
​
prompt_name
string
Name of the prompt.
​
full_request
object
The full request object. Default is an empty dictionary. This is optional and it is helpful for logging configurations such as temperature, presence_penalty etc.
completion_messages, tool_calls will be automatically extracted from full_request
{
    "full_request": {
        "temperature": 0.5,
        "top_p": 0.5,
        //... other parameters
    }
}
​
keywordsai_api_controls
object
Use this parameter to control the behavior of the Keywords AI API. Default is an empty dictionary.

​
Pricing Configuration

​
prompt_unit_price
number
Pass this parameter in if you want to log your self-host / fine-tuned model.
​
completion_unit_price
number
Pass this parameter in if you want to log your self-host / fine-tuned model.

​
Usage Details

​
usage
object
Usage details for the LLM inference. Currently, only support Prompt Caching.
​
positive_feedback
boolean
Whether the user liked the output. True means the user liked the output.
import requests

url = "https://api.keywordsai.co/api/request-logs/create/"
headers = {
    "Authorization": "Bearer YOUR_KEYWORDS_AI_API_KEY",
    "Content-Type": "application/json"
}
payload = {
    "model": "gpt-4",
    "prompt_messages": [
        {
          "role": "user",
          "content": "Hi"
        },
        {
          "role": "assistant",
          "content": None,
          "tool_calls": [
            {
              "id": "xxxx",
              "type": "function",
              "function": {
                "name": "get_current_weather", # Function name
                "arguments": "{\n\"location\": \"Boston, MA\"\n}" # Function arguments
              }
            }
          ]
        }, #optional
    ],
    "completion_message": {
        "role": "assistant",
        "content": "Hi, how can I assist you today?"
    },
    "tool_choice": {
        "type": "function",
        "function": {
            "name": "get_current_weather"
        }
    },
    "tools":[
      {
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "description": "Get the current weather in a given location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state, e.g. San Francisco, CA",
            },
            "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
          },
          "required": ["location"],
        },
      },
      }
    ],
    "customer_params": {
        "customer_identifier": "customer_123",
        "name": "Hendrix Liu", # optional
        "email": "hendrix@keywordsai.co" # optional
    },
    "prompt_tokens": 8,
    "completion_tokens": 16,
    "cost": 0.00042,
    "latency": 0.0,
    "timestamp": "2024-04-15T08:30:37.721313Z",
    "time_to_first_token": 0.0,
    "metadata": {},
    "stream": False,
    "status_code": 200,
    "warnings": "",
    "error_message": "",
    "type":"text", # "json_schema", "json_object"
}
response = requests.request("POST", url, headers=headers, json=payload)