When an LLM call fails, our system detects the error and retries the request to prevent failovers.
There are 2 ways to enable Retries and set the number of retries and the time to wait before retrying:
Go to the Retries page and enable retries and set the number of retries and the initial retry time.
You can also add the retry parameters in the request body to enable retries and set the number of retries and the time to wait before retrying.