Proxy
Retries
When an LLM call fails, our system detects the error and retries the request to prevent failovers.
There are 2 ways to enable Retries and set the number of retries and the time to wait before retrying:
Add it from the UI
Go to the Retries page and enable retries and set the number of retries and the initial retry time.
Add it in code
You can also add the retry parameters in the request body to enable retries and set the number of retries and the time to wait before retrying.
Supported parameters
retry_params
object
Enable or disable retries and set the number of retries and the time to wait before retrying.