> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.runtype.com/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.runtype.com/_mcp/server.

# Firecrawl (web scraping)

Use Firecrawl to scrape and extract content from web pages. Firecrawl is available as a fetch method on the **Fetch URL** Flow step and as a built-in tool for Agents.

## When to use Firecrawl

Firecrawl renders the page, including JavaScript, and returns it in the formats that you request. Use it instead of a **Standard HTTP** fetch when a site blocks, rate-limits, or challenges automated requests, or when you need the page as Markdown or structured JSON.

The Flow step and the Agent tool support different options. The following table compares them:

| Option                                    | Fetch URL step                                               | Agent tool                                           |
| ----------------------------------------- | ------------------------------------------------------------ | ---------------------------------------------------- |
| Output formats                            | `markdown`, `html`, `rawHtml`, `screenshot`, `links`, `json` | `markdown`, `html`, `rawHtml`, `screenshot`, `links` |
| JSON extraction with a schema or a prompt | Yes                                                          | No                                                   |
| Main content only                         | Yes                                                          | Yes                                                  |
| Wait for dynamic content                  | Only with a `wait` page action                               | Yes, with `waitFor`                                  |
| Cache, location, and language settings    | Yes                                                          | No                                                   |
| Page actions before the scrape            | Yes, in the flow definition only                             | No                                                   |

To extract structured JSON from a page, use the **Fetch URL** step.

## Use your own Firecrawl API key (optional)

Runtype provides a platform key for Firecrawl, so you can scrape pages without a Firecrawl account. Scrapes that use the platform key count toward your Runtype usage. Scrapes that use your own key are billed by Firecrawl to your Firecrawl account.

To use your own Firecrawl account, follow these steps:

1. Sign up for an account at [Firecrawl](https://firecrawl.dev).
2. Get your API key from the Firecrawl dashboard.
3. In Runtype, open **Settings > Integrations**.
4. Find **Firecrawl**.
5. In **Firecrawl API Key**, enter your API key.
6. Click **Save Key**.

To go back to the platform key, click **Remove Key** on the **Firecrawl** card.

## Use Firecrawl in a Flow

To use Firecrawl in a Flow, follow these steps:

1. Open a Flow.
2. Click **Add step**.
3. Select **Fetch URL**.
4. In the **Fetch** selector, choose **Firecrawl (Web Scraping)**.
5. In **from URL**, enter the page URL or a `{{variable}}` reference, such as `{{_record.metadata.website}}`. If the URL has no scheme, Runtype adds `https://`.
6. In **output as**, select one or more output formats. The default is **Markdown**.
7. Optional: To drop navigation, headers, and footers, turn on **Extract main content only**.
8. If you selected **JSON**, the **JSON Extraction** section opens. Enter a **JSON Schema**, or describe the data in **or describe what to extract**.
9. Optional: To change caching, location, and language, open **Advanced**:
   * **Cache results**: store the scrape in Firecrawl's cache.
   * **cache for**: the maximum age, in milliseconds, of a cached copy that Firecrawl can return. For example, `172800000` is 2 days. This field shows only when **Cache results** is on.
   * **country**: a country code for the request, such as `US`.
   * **languages**: a comma-separated list of language codes, such as `en, es`.

If you set both a JSON schema and a description, the step uses the schema. If the schema isn't valid JSON, the step ignores it and doesn't use the description either. Check that the schema parses before you run the Flow.

### Step output

The step writes an object to its output variable. The object has one key for each format that you requested (`markdown`, `html`, `rawHtml`, `screenshot`, `links`, or `json`) and a `metadata` key. The `metadata` object includes `sourceURL` and `statusCode`.

If the output variable is `page`, reference the result in a later step:

```plaintext
{{page.markdown}}
{{page.json}}
{{page.metadata.statusCode}}
```

The result of JSON extraction is under `json`.

### Define the step in code

This flow definition step scrapes a pricing page and extracts the plan names and prices as JSON:

```json
{
  "type": "fetch-url",
  "name": "Scrape pricing",
  "config": {
    "http": { "url": "https://PRICING_PAGE_URL" },
    "fetchMethod": "firecrawl",
    "firecrawl": {
      "formats": ["markdown", "json"],
      "onlyMainContent": true,
      "jsonPrompt": "Extract each plan name and its monthly price."
    },
    "outputVariable": "page"
  }
}
```

Replace `PRICING_PAGE_URL` with the page to scrape. The `firecrawl` object accepts the following fields:

* `formats`: the output formats. The default is `["markdown"]`.
* `onlyMainContent`: when `true`, returns only the page's main content.
* `jsonSchema`: a JSON Schema for JSON extraction, as an object or a string.
* `jsonPrompt`: a plain-language description of the data to extract.
* `storeInCache`: when `false`, Firecrawl doesn't cache the scrape.
* `maxAge`: the maximum age of a cached copy, in milliseconds.
* `location`: an object with `country` and `languages`, such as `{ "country": "US", "languages": ["en"] }`.
* `actions`: a list of `click`, `write`, `press`, `wait`, or `screenshot` actions that run on the page before the scrape. The Flow editor doesn't show this option.

A later prompt step can then use `{{page.json}}` to compare the plans. For the other Fetch URL settings, see [Using fetch-url and api-call steps](/user-guide/flows/using-fetch-url-and-api-call-steps).

### Handle failed scrapes

A failed scrape follows the step's error handling. With **Continue on error**, which is the default, the Flow keeps going and the output variable holds the step's default value, or `null` if no default is set. With **Stop on error**, the run ends with an error that starts with `Failed to fetch URL with Firecrawl`. When you scrape many URLs in a batch, check the output before you use it. For the error-handling modes, see [Error handling](/user-guide/flows/creating-and-editing-flows#error-handling).

## Use Firecrawl as a tool

Add the **Firecrawl** built-in tool to an Agent. The Agent can then call Firecrawl during an execution to scrape a page. To add it in the dashboard, see [Add tools to an Agent](/user-guide/tools/built-in-tools#add-tools-to-an-agent). In an Agent definition, add the tool ID `builtin:firecrawl` to `tools.toolIds`.

The model passes the following arguments when it calls the tool:

* `url` (required): the full URL of the page to scrape.
* `formats`: the output formats. Accepts `markdown` (default), `html`, `rawHtml`, `screenshot`, and `links`.
* `onlyMainContent`: when `true`, strips navigation, footers, and ads. The default is `true`.
* `waitFor`: the time to wait for dynamic content before the scrape, in milliseconds, from `0` to `30000`.

The tool returns `success`, `url`, `metadata`, and one field for each format that the model requested.

## Troubleshooting

You can see the following errors:

* **`Firecrawl API key not configured`**: no Firecrawl key is available. If you use the platform key, check your credit balance in **Settings > Billing**. When platform credit runs out, the Fetch URL step reports this error. To keep scraping, add your own key in **Settings > Integrations**.
* **`Platform key spend credit is exhausted`**: the Agent tool reports this error when platform credit runs out. Restore your credit balance, or add your own Firecrawl key.
* **`Firecrawl API error`** followed by a status code: Firecrawl rejected the request. The message includes Firecrawl's response, which usually names the problem, such as an invalid key or an unreachable URL.
* **The `json` key is missing or empty**: check that the JSON schema is valid JSON, or remove it and use a description.

For more checks, see [Firecrawl scraping failed](/user-guide/troubleshooting-faq/common-errors-and-solutions#firecrawl-scraping-failed).

## Next steps

Choose a guide for your next task:

* [Using fetch-url and api-call steps](/user-guide/flows/using-fetch-url-and-api-call-steps): configure other Fetch URL settings
* [Built-in tools](/user-guide/tools/built-in-tools): compare Firecrawl with the other built-in tools
* [Agent tools](/user-guide/agents/agent-tools): control how an Agent uses its tools
* [Massive (web rendering)](/user-guide/integrations/massive-web-render): fetch pages that resist ordinary requests
* [Exa (web search)](/user-guide/integrations/exa-web-search): find pages to scrape