LM Studio is an application for Mac, Windows, and Linux that makes it easy to locally run open-source models and comes with a great UI. To get started with LM Studio, download from the website, use the UI to download a model, and then start the local inference server. Continue can then be configured to use the LMStudio LLM class:

​
Chat model

We recommend configuring Llama3.1 8B as your chat model.
config.yaml
models:
  - name: Llama3.1 8B Instruct
    provider: lmstudio
    model: meta-llama-3.1-8b-instruct

​
Embeddings model

LMStudio supports embeddings endpoints, and comes with the nomic-ai/nomic-embed-text-v1.5-GGUF model (as of Nov 2024, check your models)
config.yaml
models:
  - name: Nomic Embed Text
    provider: lmstudio
    model: nomic-ai/nomic-embed-text-v1.5-GGUF
    roles:
      - embed

​
Setting up a remote instance

To configure a remote instance of LM Studio, add the "apiBase" property to your model in config.json:
config.yaml
models:
  - name: Llama3.1 8B Instruct
    provider: lmstudio
    model: meta-llama-3.1-8b-instruct
    apiBase: http://<MY ENDPOINT>/v1
This apiBase will now be used instead of the default http://localhost:1234/v1. View the source