HuggingFace Inference Endpoints
Hugging Face Inference Endpoints are an easy way to setup instances of open-source language models on any cloud. Sign up for an account and add billing here, access the Inference Endpoints here, click on “New endpoint”, and fill out the form (e.g. select a model like WizardCoder-Python-34B-V1.0), and then deploy your model by clicking “Create Endpoint”. Change ~/.continue/config.json to look like this:
config.json
{
"models": [
{
"title": "Hugging Face Inference API",
"provider": "huggingface-inference-api",
"model": "MODEL_NAME",
"apiKey": "YOUR_HF_TOKEN",
"apiBase": "INFERENCE_API_ENDPOINT_URL"
}
]
}