Vertex AI

There are 2 ways to add your Vertex AI credentials to your requests:

​

Add it from the UI

1. Go to Credentials page

2. Add your Vertex AI credentials.

​

Add it in code

Add customer_credentials parameter in your request body to use your own Vertex AI credits.

{
  // Rest of the request body
  "customer_credentials": {
    "google_vertex_ai": {
      "vertex_ai_project": "your-project",
      "vertex_ai_location": "your-location",
      "vertex_ai_credentials": credentials // a JSON object, you could also pass an entire JSON object here
    }
  }
}

​

Override credentials for a particular model. (Optional)

One-off credential overrides. Instead of using what is uploaded for each provider, this targets credentials for individual models.

{
  // Rest of the request body
  "customer_credentials": {
    "google_vertex_ai": {
      "vertex_ai_project": "your-project",
      "vertex_ai_location": "your-location",
      "vertex_ai_credentials": credentials // a JSON object, you could also pass an entire JSON object here
    }
  },
  "credential_override": {
    "vertex_ai_beta/gemini-1.0-pro-001":{ // override for a specific model.
      "vertex_ai_project": "your-project",
      "vertex_ai_location": "your-location",
      "vertex_ai_credentials": another_credentials // a JSON object, you could also pass an entire JSON object here
    }
  }
}

​

Requirements

1. Ensure your deployment name matches those listed on our Models page.
2. Confirm that your models are available in the region specified by your credentials.
   Credentials validation is not supported. Failure to comply with these requirements will result in errors!

​

How to get your Vertex AI credentials

1. Run gcloud init to initialize the gcloud CLI.
2. Run gcloud info to get the path of your credential. You will see a line like
   User Config Directory: [/home/USERNAME/.config/gcloud]
3. You will find a file named application_default_credentials.json in the path you got from the previous step. This file contains your credentials.
4. Load this json file into the vertex_ai_credentials field in the request body.

For example:

with open("./credentials/google/application_default_credentials.json", "r") as f:
    credentials = json.load(f)