Local OpenAI-compatible LLM

Configure an operator-managed Chat Completions API base URL including /v1, an exact served model ID and optional Bearer credential. The endpoint is reached from the backend plugin process: localhost refers to that process's host or container. For a separate model server use its approved internal address.

HTTPS verifies certificates. Supply custom CA PEM for a private CA. Allow plain HTTP explicitly only for a trusted local network. Redirects and proxy environment variables are ignored; metadata service destinations are rejected.

Connection checks list models without generating text. A matching model proves metadata access only. A server without the compatible models endpoint returns an unsupported check; it does not trigger a sample generation or model download.

Start with text mode. Enable JSON or native schema mode only after confirming the specific server/model supports the corresponding OpenAI response_format. There is no automatic downgrade. Completed candidates still need host validation and human review before saving. This provider cannot execute actions or rules.

Timeout, cancellation, authentication, context limit, refusal and incomplete generation remain failures. There are no retries or cloud fallback. Raw error bodies and reasoning content are excluded from results. Usage can be unavailable.

Real local inference compatibility remains unverified in this implementation. The mocked tests cover the API contract. Ollama native API, tools, embedding, retrieval, streaming and model lifecycle management are outside this provider.