Skip to main content

NVIDIA

info

View the docs to learn how to get an API key.

Chat model

We recommend configuring Nemotron-4-340B-Instruct as your chat model.

config.yaml
models:
- name: Nemotron-4-340B-Instruct
provider: nvidia
model: nvidia-nemotron-4-340b-instruct
apiKey: <YOUR_NVIDIA_API_KEY>

Autocomplete model

NVIDIA currently does not offer any autocomplete models.

Click here to see a list of autocomplete model providers.

Embeddings model

We recommend configuring NVIDIA Retrieval QA Mistral 7B as your embeddings model.

config.yaml
models:
- name: Nvidia Embedder
provider: nvidia
model: nvidia/nv-embedqa-mistral-7b-v2
apiKey: <YOUR_NVIDIA_API_KEY>
roles:
- embed

Reranking model

NVIDIA currently does not offer any reranking models.

Click here to see a list of reranking model providers.