vLLM

vLLM can serve open-source (and some custom) models via an OpenAI-compatible HTTP API. OpenClaw connects to vLLM using the

OpenClaw can also auto-discover available models from vLLM when you opt in with

OpenClaw treats

Getting started

text
Copy
```
http://127.0.0.1:8000/v1
```

text
Copy
```bash}
export VLLM_API_KEY="vllm-local"
```

text
Copy
```json5}
{
  agents: {
    defaults: {
      model: { primary: "vllm/your-model-id" },
    },
  },
}
```

Model discovery (implicit provider)

When

text
Copy
GET http://127.0.0.1:8000/v1/models

and converts the returned IDs into model entries.

Explicit configuration (manual models)

Use explicit config when:

• vLLM runs on a different host or port
• You want to pin
  text

  Copy

  contextWindow
  or
  text

  Copy

  maxTokens
  values
• Your server requires a real API key (or you want to control headers)
• You connect to a trusted loopback, LAN, or Tailscale vLLM endpoint

json5
Copy
{
  models: {
    providers: {
      vllm: {
        baseUrl: "http://127.0.0.1:8000/v1",
        apiKey: "${VLLM_API_KEY}",
        api: "openai-completions",
        request: { allowPrivateNetwork: true },
        timeoutSeconds: 300, // Optional: extend connect/header/body/request timeout for slow local models
        models: [
          {
            id: "your-model-id",
            name: "Local vLLM Model",
            reasoning: false,
            input: ["text"],
            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
            contextWindow: 128000,
            maxTokens: 8192,
          },
        ],
      },
    },
  },
}

Advanced configuration

Troubleshooting

Related