Integrations

Firecrawl (web scraping)

Use Firecrawl to scrape and extract content from web pages. Firecrawl is available as a built-in tool for Agents and as a fetch method on the Fetch URL Flow step.

How Firecrawl works

Firecrawl converts web pages into clean, structured data. It renders JavaScript, handles anti-bot measures, and supports data extraction.

Set up Firecrawl

Runtype provides a platform key for Firecrawl, so you can use Firecrawl without adding your own API key. To connect your Firecrawl account, follow these steps:

  1. Sign up for an account at Firecrawl.
  2. Get your API key from the Firecrawl dashboard.
  3. In Runtype, open Settings > Integrations.
  4. Find Firecrawl.
  5. Enter your API key.
  6. Click Save Key.

Use Firecrawl in a Flow

To use Firecrawl in a Flow, follow these steps:

  1. Open a Flow in the Flow editor.
  2. Click Add Step.
  3. Select Fetch URL.
  4. In the Fetch selector, choose Firecrawl (Web Scraping).
  5. Set the page URL.
  6. Choose one or more output formats.

The Flow supports Markdown, HTML, Raw HTML, Screenshot, Links, and JSON output.

Use Firecrawl as a tool

When you configure an Agent, add the Firecrawl built-in tool. The Agent can call Firecrawl during an execution to scrape a page.

The built-in tool accepts the following parameters:

  • url (required): Enter the page URL.
  • formats: Choose one or more output formats: markdown (default), html, rawHtml, screenshot, or links.
  • onlyMainContent: Set to true to strip navigation, footers, and ads. The default is true.
  • waitFor: Set the wait time for dynamic content before scraping in milliseconds. Use a value from 0 to 30000.

Use cases

Use Firecrawl for the following tasks:

  • Competitive intelligence: Monitor competitor pricing.
  • Lead enrichment: Extract company data from websites.
  • Content aggregation: Collect articles or listings.
  • Research: Gather data from multiple sources.

Next steps

Choose a guide for your next task: