IBM watsonx
watsonx, developed by IBM, offers a variety of pre-trained AI foundation models that can be used for natural language processing (NLP), computer vision, and speech recognition tasks.
Setup
Accessing watsonx models can be done either through watsonx SaaS on IBM Cloud or using a dedicated watsonx.ai Software instance.
watsonx.ai SaaS - IBM Cloud
To get started with watsonx SaaS, visit the registration page. If you do not have an existing IBM Cloud account, you can sign up for a free trial.
To authenticate to watsonx.ai SaaS with Continue, you will need to create a project and set up an API key. Then, in continue:
- Set apiBase to your watsonx SaaS endpoint, e.g.
https://us-south.ml.cloud.ibm.com
for US South region. - Set projectId to your watsonx project ID.
- Set apiKey to your watsonx API Key.
watsonx.ai Software
To authenticate to your watsonx.ai Software instance with Continue, you can use either username/password
or ZenApiKey
method:
- Option 1 (Recommended): using
ZenApiKey
authentication:- Set apiBase to your watsonx software endpoint, e.g.
https://cpd-watsonx.apps.example.com
. - Set projectId to your watsonx project ID.
- Set apiKey to your watsonx Zen API Key. To generate it:
- Log in to the CPD web client.
- From the toolbar, click your avatar.
- Click Profile and settings.
- Click API key > Generate new key.
- Click Generate.
- Click Copy and save your key somewhere safe. You cannot recover this key if you lose it.
- Generate your ZenApiKey by running the following command in your preferred terminal:
echo "<username>:<apikey>" | base64
, replacing<username>
with your CPD username and<apikey>
with the API Key you just created.
- Set apiBase to your watsonx software endpoint, e.g.
- Option 2: using
username/password
authentication:- Set apiBase to your watsonx software endpoint, e.g.
https://cpd-watsonx.apps.example.com
. - Set projectId to your watsonx project ID.
- Set API Key to your watsonx Username and Password using
username:password
as format.
- Set apiBase to your watsonx software endpoint, e.g.
Configuration
Add the following configuration:
- YAML
- JSON
models:
- name: watsonx - Model Name
provider: watsonx
model: model ID
apiBase: https://us-south.ml.cloud.ibm.com
projectId: PROJECT_ID
apiKey: API_KEY/ZENAPI_KEY/USERNAME:PASSWORD
apiVersion: 2024-03-14
{
"models": [
{
"model": "model ID",
"title": "watsonx - Model Name",
"provider": "watsonx",
"apiBase": "https://us-south.ml.cloud.ibm.com",
"projectId": "PROJECT_ID",
"apiKey": "API_KEY/ZENAPI_KEY/USERNAME:PASSWORD",
"apiVersion": "2024-03-14"
}
]
}
apiVersion
is optional and defaults to the latest version.
If you are using a custom deployment endpoint, set deploymentID
to the model's deployment ID. You can find it in the watsonx.ai Prompt Lab UI by selecting the corresponding model and opening the </>
tab on the right, which will display the endpoint's URL containing the deployment ID.
- YAML
- JSON
models:
- name: watsonx - Model Name
provider: watsonx
model: model ID
apiBase: watsonx endpoint e.g. https://us-south.ml.cloud.ibm.com
apiKey: API_KEY/ZENAPI_KEY/USERNAME:PASSWORD
apiVersion: 2024-03-14
deploymentId: DEPLOYMENT_ID
{
"models": [
{
"model": "model ID",
"title": "watsonx - Model Name",
"provider": "watsonx",
"apiBase": "watsonx endpoint e.g. https://us-south.ml.cloud.ibm.com",
"apiKey": "API_KEY/ZENAPI_KEY/USERNAME:PASSWORD",
"apiVersion": "2024-03-14",
"deploymentId": "DEPLOYMENT_ID"
}
]
}
Configuration Options
Make sure to specify a template name, such as granite
or llama3
, and to set the contextLength
to the model's context window size.
You can also configure generation parameters, such as temperature, topP, topK, frequency penalty, and stop sequences:
- YAML
- JSON
models:
- name: Granite Code 20b
provider: watsonx
model: ibm/granite-20b-code-instruct
apiBase: watsonx endpoint e.g. https://us-south.ml.cloud.ibm.com
projectId: PROJECT_ID
apiKey: API_KEY/ZENAPI_KEY/USERNAME:PASSWORD
apiVersion: 2024-03-14
template: granite
contextLength: 8000
defaultCompletionOptions:
temperature: 0.1
topP: 0.3
topK: 20
maxTokens: 2000
frequencyPenalty: 1.1
stop:
- Question:
- "\n\n\n"
{
"models": [
{
"model": "ibm/granite-20b-code-instruct",
"title": "Granite Code 20b",
"provider": "watsonx",
"apiBase": "watsonx endpoint e.g. https://us-south.ml.cloud.ibm.com",
"projectId": "PROJECT_ID",
"apiKey": "API_KEY/ZENAPI_KEY/USERNAME:PASSWORD",
"apiVersion": "2024-03-14",
"template": "granite",
"contextLength": 8000,
"completionOptions": {
"temperature": 0.1,
"topP": 0.3,
"topK": 20,
"maxTokens": 2000,
"frequencyPenalty": 1.1,
"stop": ["Question:", "\n\n\n"]
}
}
]
}
Tab Auto Complete Model
Granite models are recommended for tab auto complete. The configuration is similar to that of the chat models:
- YAML
- JSON
models:
- name: Granite Code 8b
provider: watsonx
model: ibm/granite-8b-code-instruct
apiBase: watsonx endpoint e.g. https://us-south.ml.cloud.ibm.com
projectId: PROJECT_ID
apiKey: API_KEY/ZENAPI_KEY/USERNAME:PASSWORD
apiVersion: 2024-03-14
roles:
- autocomplete
{
"tabAutocompleteModel": {
"model": "ibm/granite-8b-code-instruct",
"title": "Granite Code 8b",
"provider": "watsonx",
"apiBase": "watsonx endpoint e.g. https://us-south.ml.cloud.ibm.com",
"projectId": "PROJECT_ID",
"apiKey": "API_KEY/ZENAPI_KEY/USERNAME:PASSWORD",
"apiVersion": "2024-03-14",
"contextLength": 4000
}
}
Embeddings Model
To view the list of available embeddings models, visit this page.
- YAML
- JSON
models:
- name: Watsonx Embedder
provider: watsonx
model: ibm/slate-30m-english-rtrvr-v2
apiBase: https://us-south.ml.cloud.ibm.com
projectId: PROJECT_ID
apiKey: API_KEY/ZENAPI_KEY/USERNAME:PASSWORD
apiVersion: 2024-03-14
roles:
- embed
{
"embeddingsProvider": {
"provider": "watsonx",
"model": "ibm/slate-30m-english-rtrvr-v2",
"apiBase": "watsonx endpoint e.g. https://us-south.ml.cloud.ibm.com",
"projectId": "PROJECT_ID",
"apiKey": "API_KEY/ZENAPI_KEY/USERNAME:PASSWORD",
"apiVersion": "2024-03-14"
}
}