Vertex AI

info

You need to enable the Vertex AI API and set up the Google Application Default Credentials.

Chat model

We recommend configuring Claude 3.5 Sonnet as your chat model.

YAML

Package or config.yaml

models:
  - name: Claude 3.5 Sonnet
    provider: vertexai
    model: claude-3-5-sonnet-20240620
    projectId: <PROJECT_ID>
    region: us-east5

JSON

config.json

{
  "models": [
    {
      "title": "Claude 3.5 Sonnet",
      "provider": "vertexai",
      "model": "claude-3-5-sonnet-20240620",
      "projectId": "[PROJECT_ID]",
      "region": "us-east5"
    }
  ]
}

Autocomplete model

We recommend configuring Codestral or code-gecko as your autocomplete model.

YAML

config.yaml

models:
  - name: Codestral (Vertex AI)
    provider: vertexai
    model: codestral
    projectId: <PROJECT_ID>
    region: us-central1
    roles:
      - autocomplete

JSON

config.json

{
  "tabAutocompleteModel": {
      "title": "Codestral (Vertex AI)",
      "provider": "vertexai",
      "model": "codestral",
      "projectId": "[PROJECT_ID]",
      "region": "us-central1"
  }
}

Embeddings model

We recommend configuring text-embedding-004 as your embeddings model.

YAML

config.yaml

models:
  - name: Text Embedding-004
    provider: vertexai
    model: text-embedding-004
    projectId: <PROJECT_ID>
    region: us-central1
    roles:
      - embed

JSON

config.json

{
  "embeddingsProvider": {
    "provider": "vertexai",
    "model": "text-embedding-004",
    "projectId": "[PROJECT_ID]",
    "region": "us-central1"
  }
}

Reranking model

Vertex AI currently does not offer any reranking models.

Click here to see a list of reranking model providers.