An open coding assistant leaves the model choice to the team

The setup

Continue is an open-source coding-assistant framework rather than a single hosted product, and its model documentation makes that structural choice explicit: a team edits a configuration file to plug in whichever language model handles chat, autocomplete, editing and code-apply roles, rather than accepting one vendor's default. The setup step is choosing and wiring providers per role, from a small set of familiar frontier APIs to a fully local model run on a developer's own machine.

What the documents show

The documentation lists direct support for Anthropic, OpenAI, Gemini, Amazon Bedrock, Azure, xAI, Mistral, and locally run Ollama models, each requiring only an API key or, for Ollama, a locally pulled model file. The Ollama guide frames the local path explicitly as privacy-focused, offline coding assistance, with modest minimum requirements: 8 GB of RAM, more recommended, and 10 GB of free storage. A separate self-hosting guide covers wiring up a custom endpoint through HuggingFace TGI, vLLM or any OpenAI-compatible server, with API-key, custom-header or client-certificate authentication for a private deployment. Together these describe a documented difference in where a prompt and its code travel: to a named third-party API under a hosted setup, or no further than the machine or private endpoint the team runs under a local one.

The friction

The Ollama guide is candid about a specific limitation rather than presenting local models as a drop-in replacement: locally run models have limited tool-calling and reasoning capability, which the documentation says makes Agent mode difficult to use with them. It documents a known failure mode where a local model advertises tool support that does not actually work, producing an Agent-mode error even after a team configures the capability flag the docs recommend. The self-hosting guide's authentication options assume a team capable of managing its own certificates and headers, a setup cost a hosted API key does not carry.

What changed in the work

For a team that cannot or will not send code to a third-party model API, Continue's documented local and self-hosted paths are what make an AI coding assistant usable at all, a genuine before-and-after distinct from cosmetic customization. What the documentation does not support is treating the tool as equally capable in every mode regardless of which model sits behind it: agent-mode reliability is tied to the model chosen, not just to Continue itself, an editorial reading of what the tool-support caveats mean for a team picking a model.

• Which Continue roles, chat, autocomplete, edit or agent, does the team actually need a local model to cover?
• Does the chosen local model's advertised tool support hold up in Agent mode, or only in chat?
• Who manages certificates and endpoint uptime if the team moves from a hosted API key to a self-hosted server?

Continue's own documentation supports the claim that model choice, including a fully local one, is real and configurable rather than aspirational, but the same pages are where the caveat lives: that choice is least reliable in exactly the agent mode a team most wants a coding assistant for.