{"id":630,"date":"2026-03-21T22:52:57","date_gmt":"2026-03-21T14:52:57","guid":{"rendered":"https:\/\/pa.yingzhi8.cn\/index.php\/2026\/03\/21\/providers-vllm\/"},"modified":"2026-03-21T23:08:57","modified_gmt":"2026-03-21T15:08:57","slug":"providers-vllm","status":"publish","type":"post","link":"https:\/\/pa.yingzhi8.cn\/index.php\/2026\/03\/21\/providers-vllm\/","title":{"rendered":"vLLM"},"content":{"rendered":"

vLLM<\/h1>\n

vLLM can serve open-source (and some custom) models via an OpenAI-compatible<\/strong> HTTP API. OpenClaw can connect to vLLM using the openai-completions<\/code> API.<\/p>\n

OpenClaw can also auto-discover<\/strong> available models from vLLM when you opt in with VLLM_API_KEY<\/code> (any value works if your server doesn\u2019t enforce auth) and you do not define an explicit models.providers.vllm<\/code> entry.<\/p>\n

Quick start<\/h2>\n
    \n
  1. Start vLLM with an OpenAI-compatible server.<\/li>\n<\/ol>\n

    Your base URL should expose \/v1<\/code> endpoints (e.g. \/v1\/models<\/code>, \/v1\/chat\/completions<\/code>). vLLM commonly runs on:<\/p>\n

      \n
    • http:\/\/127.0.0.1:8000\/v1<\/code><\/li>\n<\/ul>\n
        \n
      1. Opt in (any value works if no auth is configured):<\/li>\n<\/ol>\n

        “`bash theme={“theme”:{“light”:”min-light”,”dark”:”min-dark”}}
        \nexport VLLM_API_KEY=”vllm-local”<\/p>\n

        \n3. Select a model (replace with one of your vLLM model IDs):\n\n```json5  theme={"theme":{"light":"min-light","dark":"min-dark"}}\n{\n  agents: {\n    defaults: {\n      model: { primary: "vllm\/your-model-id" },\n    },\n  },\n}\n<\/code><\/pre>\n
        Model discovery (implicit provider)<\/h2>\n
        When VLLM_API_KEY<\/code> is set (or an auth profile exists) and you do not<\/strong> define models.providers.vllm<\/code>, OpenClaw will query:<\/p>\n
          \n
        • GET http:\/\/127.0.0.1:8000\/v1\/models<\/code><\/li>\n<\/ul>\n
          \u2026and convert the returned IDs into model entries.<\/p>\n
          If you set models.providers.vllm<\/code> explicitly, auto-discovery is skipped and you must define models manually.<\/p>\n
          Explicit configuration (manual models)<\/h2>\n
          Use explicit config when:<\/p>\n
            \n
          • vLLM runs on a different host\/port.<\/li>\n
          • You want to pin contextWindow<\/code>\/maxTokens<\/code> values.<\/li>\n
          • Your server requires a real API key (or you want to control headers).<\/li>\n<\/ul>\n
            “`json5  theme={“theme”:{“light”:”min-light”,”dark”:”min-dark”}}
            \n{
            \n  models: {
            \n    providers: {
            \n      vllm: {
            \n        baseUrl: “http:\/\/127.0.0.1:8000\/v1”,
            \n        apiKey: “${VLLM_API_KEY}”,
            \n        api: “openai-completions”,
            \n        models: [
            \n          {
            \n            id: “your-model-id”,
            \n            name: “Local vLLM Model”,
            \n            reasoning: false,
            \n            input: [“text”],
            \n            cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
            \n            contextWindow: 128000,
            \n            maxTokens: 8192,
            \n          },
            \n        ],
            \n      },
            \n    },
            \n  },
            \n}<\/p>\n
            \n## Troubleshooting\n\n* Check the server is reachable:\n\n```bash  theme={"theme":{"light":"min-light","dark":"min-dark"}}\ncurl http:\/\/127.0.0.1:8000\/v1\/models\n<\/code><\/pre>\n
              \n
            • If requests fail with auth errors, set a real VLLM_API_KEY<\/code> that matches your server configuration, or configure the provider explicitly under models.providers.vllm<\/code>.<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"
              vLLM vLLM can serve open-source (and some custom) model […]<\/p>\n","protected":false},"author":0,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[],"class_list":["post-630","post","type-post","status-publish","format-standard","hentry","category-docs"],"_links":{"self":[{"href":"https:\/\/pa.yingzhi8.cn\/index.php\/wp-json\/wp\/v2\/posts\/630","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pa.yingzhi8.cn\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/pa.yingzhi8.cn\/index.php\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/pa.yingzhi8.cn\/index.php\/wp-json\/wp\/v2\/comments?post=630"}],"version-history":[{"count":2,"href":"https:\/\/pa.yingzhi8.cn\/index.php\/wp-json\/wp\/v2\/posts\/630\/revisions"}],"predecessor-version":[{"id":734,"href":"https:\/\/pa.yingzhi8.cn\/index.php\/wp-json\/wp\/v2\/posts\/630\/revisions\/734"}],"wp:attachment":[{"href":"https:\/\/pa.yingzhi8.cn\/index.php\/wp-json\/wp\/v2\/media?parent=630"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/pa.yingzhi8.cn\/index.php\/wp-json\/wp\/v2\/categories?post=630"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/pa.yingzhi8.cn\/index.php\/wp-json\/wp\/v2\/tags?post=630"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}