Vertex AI

There are 2 ways to add your Vertex AI credentials to your requests:

​

Add it from the UI

1

Go to the Credentials page

Go to Credentials page

2

Add your Vertex AI credentials.

3

(Optional) Choose models you want to use credits

You can choose the models you want to use your credits with. Just simply type the model ID from the Models page and copy the model id and paste it in the Available models. Press Enter to add the model.

Leave it empty to apply your credentials for all Vertex AI models.

​

Add it in code

Add customer_credentials parameter in your request body to use your own Vertex AI credits.

{
  // Rest of the request body
  "customer_credentials": {
    "google_vertex_ai": {
      "vertex_ai_project": "your-project",
      "vertex_ai_location": "your-location",
      "vertex_ai_credentials": credentials // a JSON object, you could also pass an entire JSON object here
    }
  }
}

​

Override credentials for a particular model. (Optional)

One-off credential overrides. Instead of using what is uploaded for each provider, this targets credentials for individual models.

{
  // Rest of the request body
  "customer_credentials": {
    "google_vertex_ai": {
      "vertex_ai_project": "your-project",
      "vertex_ai_location": "your-location",
      "vertex_ai_credentials": credentials // a JSON object, you could also pass an entire JSON object here
    }
  },
  "credential_override": {
    "vertex_ai_beta/gemini-1.0-pro-001":{ // override for a specific model.
      "vertex_ai_project": "your-project",
      "vertex_ai_location": "your-location",
      "vertex_ai_credentials": another_credentials // a JSON object, you could also pass an entire JSON object here
    }
  }
}

​

Requirements

1. Ensure your deployment name matches those listed on our Models page.
2. Confirm that your models are available in the region specified by your credentials.
   Credentials validation is not supported. Failure to comply with these requirements will result in errors!

​

How to get your Vertex AI credentials

1. Run gcloud init to initialize the gcloud CLI.
2. Run gcloud info to get the path of your credential. You will see a line like
   User Config Directory: [/home/USERNAME/.config/gcloud]
3. You will find a file named application_default_credentials.json in the path you got from the previous step. This file contains your credentials.
4. Load this json file into the vertex_ai_credentials field in the request body.

For example:

with open("./credentials/google/application_default_credentials.json", "r") as f:
    credentials = json.load(f)