Web fetch

The

tool does a plain HTTP GET and extracts readable content (HTML to markdown or text). It does not execute JavaScript.

For JS-heavy sites or login-protected pages, use the Web Browser instead.

Quick start

is enabled by default -- no configuration needed. The agent can call it immediately:

javascript
Copy
await web_fetch({ url: "https://example.com/article" });

Tool parameters

URL to fetch. `http(s)` only. Output format after main-content extraction. Truncate output to this many characters.

How it works

Config

json5
Copy
{
  tools: {
    web: {
      fetch: {
        enabled: true, // default: true
        provider: "firecrawl", // optional; omit for auto-detect
        maxChars: 50000, // max output chars
        maxCharsCap: 50000, // hard cap for maxChars param
        maxResponseBytes: 2000000, // max download size before truncation
        timeoutSeconds: 30,
        cacheTtlMinutes: 15,
        maxRedirects: 3,
        readability: true, // use Readability extraction
        userAgent: "Mozilla/5.0 ...", // override User-Agent
        ssrfPolicy: {
          allowRfc2544BenchmarkRange: true, // opt-in for trusted fake-IP proxies using 198.18.0.0/15
          allowIpv6UniqueLocalRange: true, // opt-in for trusted fake-IP proxies using fc00::/7
        },
      },
    },
  },
}

Firecrawl fallback

If Readability extraction fails,

can fall back to Firecrawl for bot-circumvention and better extraction:

json5
Copy
{
  tools: {
    web: {
      fetch: {
        provider: "firecrawl", // optional; omit for auto-detect from available credentials
      },
    },
  },
  plugins: {
    entries: {
      firecrawl: {
        enabled: true,
        config: {
          webFetch: {
            apiKey: "fc-...", // optional if FIRECRAWL_API_KEY is set
            baseUrl: "https://api.firecrawl.dev",
            onlyMainContent: true,
            maxAgeMs: 86400000, // cache duration (1 day)
            timeoutSeconds: 60,
          },
        },
      },
    },
  },
}

supports SecretRef objects. Legacy config is auto-migrated by .

Current runtime behavior:

• text

  Copy

  tools.web.fetch.provider
  selects the fetch fallback provider explicitly.
• If
  text

  Copy

  provider
  is omitted, OpenClaw auto-detects the first ready web-fetch provider from available credentials. Today the bundled provider is Firecrawl.
• If Readability is disabled,
  text

  Copy

  web_fetch
  skips straight to the selected provider fallback. If no provider is available, it fails closed.

Limits and safety

• text

  Copy

  maxChars
  is clamped to
  text

  Copy

  tools.web.fetch.maxCharsCap
• Response body is capped at
  text

  Copy

  maxResponseBytes
  before parsing; oversized responses are truncated with a warning
• Private/internal hostnames are blocked
• text

  Copy

  tools.web.fetch.ssrfPolicy.allowRfc2544BenchmarkRange
  and
  text

  Copy

  tools.web.fetch.ssrfPolicy.allowIpv6UniqueLocalRange
  are narrow opt-ins for trusted fake-IP proxy stacks; leave them unset unless your proxy owns those synthetic ranges and enforces its own destination policy
• Redirects are checked and limited by
  text

  Copy

  maxRedirects
• text

  Copy

  web_fetch
  is best-effort -- some sites need the Web Browser

Tool profiles

If you use tool profiles or allowlists, add

or :

json5
Copy
{
  tools: {
    allow: ["web_fetch"],
    // or: allow: ["group:web"]  (includes web_fetch, web_search, and x_search)
  },
}

Related

• Web Search -- search the web with multiple providers
• Web Browser -- full browser automation for JS-heavy sites
• Firecrawl -- Firecrawl search and scrape tools