Prompt version

The list of messages for the prompt version. If you want to add a variable, you can use the following format: {{variable_name}}.

{
"messages": [
    {"role": "system", "content": "You are a helpful {{role}}."}, // role could be assistant, teacher, etc.
    {"role": "user", "content": "Hello, how are you?"}
  ],
}

Speciy the model you want to use in this version.

Description of the prompt version

Whether the prompt version should be streamed or not. Default is false.

The temperature of the model.

The maximum number of tokens to generate.

The nucleus sampling probability.

Specify how much to penalize new tokens based on their existing frequency in the text so far. Decreases the model’s likelihood of repeating the same line verbatim

Specify how much to penalize new tokens based on whether they appear in the text so far. Increases the model’s likelihood of talking about new topics

The list of variables for the prompt version. You can use these variables in the messages.

{
  "variables": {
    "role": ["assistant", "teacher", "student"]
  }
}

The list of fallback models for the prompt version. Check out fallback models for more information.

{
  "fallback_models": ["gpt-4o", "gpt-4o-mini"]
}

The list of models to load balance the prompt version. Check out load balancing for more information.

{
"load_balance_models": [
    {"model": "gpt-4o", "weight": 0.7},
    {"model": "gpt-4o-mini", "weight": 0.3}
  ],
}

The list of tools to use for the prompt version. Check out tools for more information.

{
"tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather in a given location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state, e.g. San Francisco, CA"
            }
          },
          "required": ["location"]
        }
      }
    }
  ],
}

Whether to deploy this version as the live version Default is false.