NVIDIA

info

View the docs to learn how to get an API key.

Chat model

We recommend configuring Nemotron-4-340B-Instruct as your chat model.

YAML

config.yaml

models:
  - name: Nemotron-4-340B-Instruct
    provider: nvidia
    model: nvidia-nemotron-4-340b-instruct
    apiKey: <YOUR_NVIDIA_API_KEY>

JSON

config.json

{
  "models": [
    {
      "title": "Nemotron-4-340B-Instruct",
      "provider": "nvidia",
      "model": "nvidia-nemotron-4-340b-instruct",
      "apiKey": "<YOUR_NVIDIA_API_KEY>"
    }
  ]
}

Autocomplete model

NVIDIA currently does not offer any autocomplete models.

Click here to see a list of autocomplete model providers.

Embeddings model

We recommend configuring NVIDIA Retrieval QA Mistral 7B as your embeddings model.

YAML

config.yaml

models:
  - name: Nvidia Embedder
    provider: nvidia
    model: nvidia/nv-embedqa-mistral-7b-v2
    apiKey: <YOUR_NVIDIA_API_KEY>
    roles:
      - embed

JSON

config.json

{
  "embeddingsProvider": {
    "provider": "nvidia",
    "model": "nvidia/nv-embedqa-mistral-7b-v2",
    "apiKey": "<YOUR_NVIDIA_API_KEY>"
  }
}

Reranking model

NVIDIA currently does not offer any reranking models.

Click here to see a list of reranking model providers.