Using fetch-url and api-call steps | Runtype Documentation

Use Fetch URL to call an HTTP endpoint or retrieve web page content. Use Make API Call to call a JSON API and select fields from the response.

Before you begin

Open a flow in the Flow editor. If a request needs credentials, create a secret in Managing secrets before you configure the step.

Fetch URL step

Fetch URL sends a standard HTTP request or retrieves page content through Firecrawl.

Add a Fetch URL step

To add a Fetch URL step, follow these steps:

In the Flow editor, click Add step.
Select Fetch URL.
In the Fetch selector, choose Standard HTTP or Firecrawl (Web Scraping). If Massive is enabled for your account, you can also choose Massive (Web Render).
In Output variable, enter the name that later steps use to read the result. The default is fetch_result.

Standard HTTP mode

Configure the following fields for a standard HTTP request:

from URL: enter the endpoint URL. You can include flow variables such as {{PRODUCT_ID}}.
using: choose GET, POST, PUT, DELETE, or PATCH. The default is GET.
with body: enter the request payload. This field appears only for POST, PUT, and PATCH. Changing the method to GET or DELETE hides the field without clearing its stored value.
as: choose JSON or Text/HTML for the response format. To receive XML, choose Text/HTML, which stores the response as text.
Headers: add key-value pairs in the advanced Headers settings.

When you choose Text/HTML, Prefer Markdown when available (fallback to HTML) is on by default. Turn it off to receive the page’s HTML or text as the server sends it.

The request URL must meet these requirements:

It uses HTTPS. A request to an http:// URL fails with URL validation failed: HTTPS required for non-localhost URLs.
It points to a public host. Requests to localhost and to private or loopback IP addresses are blocked.
If the URL has no scheme, Runtype adds https://.
Runtype follows up to 5 redirects and checks each redirect target against these rules.

Requests time out after 120 seconds. To change this in a flow definition, set http.timeout in milliseconds.

Use a URL template when a flow variable supplies part of the endpoint:

https://api.example.com/products/{{PRODUCT_ID}}

Replace PRODUCT_ID with the name of the flow variable that contains the product ID.

Firecrawl mode

Choose Firecrawl (Web Scraping) when you need page content in one or more output formats. For API keys and output details, see Firecrawl (web scraping).

Configure the following Firecrawl options:

output as: choose one or more of Markdown, HTML, Raw HTML, Screenshot, Links, or JSON.
Extract main content only: request the page’s main content without its surrounding navigation.
JSON Extraction: when you select JSON, enter a JSON schema or describe the data to extract.
Advanced: turn Cache results on or off, set how long to cache results in milliseconds, and set the country and languages for the request.

To click, type, or wait on the page before Firecrawl scrapes it, set firecrawl.actions in the flow definition. The Flow editor doesn’t show this option.

Massive mode

If Massive is enabled for your account, you can choose Massive (Web Render) when a page blocks or challenges requests from data center IP ranges. Massive can return Markdown, rendered HTML, or raw HTML. For setup and configuration details, see Massive (web rendering).

Make API Call step

Make API Call sends an HTTP request and parses the response as JSON. It uses the same URL requirements and timeout as Fetch URL.

Add a Make API Call step

To add a Make API Call step, follow these steps:

In the Flow editor, click Add step.
Select Make API Call.
Configure the method, URL, request body, and headers.
In Output variable, enter the name that later steps use to read the result. The default is api_result.

Configure the following fields for the step:

request to: choose GET, POST, PUT, DELETE, or PATCH, then enter the endpoint URL.
with body: enter the request body for POST, PUT, or PATCH.
Headers: add key-value pairs in the advanced Headers settings.

Make API Call parses every response as JSON, whatever you choose in the expecting selector. To receive text or XML, use a Fetch URL step with Text/HTML.

Select fields with response mapping

Use responseMapping to store only some fields from the response. Each key is a name in the output, and each value is a path in the JSON response. Set the mapping in a flow definition through the API or SDK. The Flow editor doesn’t show this option.

This mapping stores the user’s name and the ID of the first item:

"responseMapping": {
  "userName": "data.user.name",
  "firstItemId": "items[0].id"
}

Later steps read the mapped fields as {{api_result.userName}} and {{api_result.firstItemId}}. If a path doesn’t exist in the response, the step leaves that key out of the output.

Set headers

Add headers as key-value pairs in either step’s Headers settings. This example uses a managed secret and a flow variable:

Authorization: Bearer {{secret:API_KEY}}
X-Custom-Header: {{CUSTOM_HEADER_VALUE}}

Replace API_KEY with the name of your managed secret and CUSTOM_HEADER_VALUE with the name of a flow variable.

Set the request body

For POST, PUT, or PATCH requests, enter a request body. JSON is a common format:

{
  "customer": "{{CUSTOMER_NAME}}",
  "order": {
    "items": "{{ORDER_ITEMS}}",
    "total": "{{ORDER_TOTAL|0}}"
  }
}

Replace CUSTOMER_NAME, ORDER_ITEMS, and ORDER_TOTAL with the names of flow variables.

When the body starts with { or [, Runtype treats it as JSON:

A value that contains only one template, such as "{{ORDER_ITEMS}}", keeps the variable’s type. If ORDER_ITEMS is an array, the request sends a JSON array, not a string.
To supply a default for a missing variable, add it after a pipe, such as {{ORDER_TOTAL|0}}. The request sends the default as a string, such as "0".
Runtype sets Content-Type: application/json unless you set that header yourself.

Configure authentication

In the Flow editor, send credentials in headers. Store credentials as managed secrets instead of writing them directly in a flow.

Use a bearer token

Send a bearer token in the Authorization header:

Authorization: Bearer {{secret:API_KEY}}

Replace API_KEY with the name of the managed secret that stores the token.

Use Basic auth

Encode the USERNAME:PASSWORD string as base64, then send the result in the Authorization header:

Authorization: Basic {{secret:BASE64_CREDENTIALS}}

Replace BASE64_CREDENTIALS with the name of the managed secret that stores the encoded credentials. In a flow definition, you can use the auth field instead, and Runtype encodes the credentials for you.

Send an API key in a header

Send an API key in a header that the external service accepts:

X-API-Key: {{secret:API_KEY}}

Replace API_KEY with the name of the managed secret that stores the API key.

Set authentication in a flow definition

When you define a step through the API or SDK, you can set an auth object instead of writing the header. The following table lists the auth types that both steps accept:

`type`	Fields	Header that Runtype sends
`bearer`	`token`	`Authorization: Bearer TOKEN`
`basic`	`username`, `password`	`Authorization: Basic` with the encoded credentials
`api-key`	`apiKey`, optional `headerName` (default `X-API-Key`)	The API key in the named header
`custom`	`customHeaders`, an object of header names and values	Each header in `customHeaders`

Each field accepts {{secret:NAME}} references and flow variables.

Define the steps in code

This flow definition calls an API with Basic auth, maps two fields, and continues with a default value if the request fails:

{
  "type": "api-call",
  "name": "Get user",
  "config": {
    "http": {
      "url": "https://api.example.com/users/{{USER_ID}}",
      "method": "GET",
      "timeout": 30000
    },
    "auth": {
      "type": "basic",
      "username": "{{secret:API_USER}}",
      "password": "{{secret:API_PASSWORD}}"
    },
    "responseMapping": {
      "userName": "data.user.name",
      "plan": "data.user.plan"
    },
    "outputVariable": "api_result",
    "errorHandling": { "onError": "continue" },
    "defaultValue": { "data": { "user": { "name": "Unknown", "plan": "free" } } }
  }
}

Replace USER_ID with a flow variable, and API_USER and API_PASSWORD with the names of your managed secrets. A Fetch URL step uses "type": "fetch-url" and accepts the same http, auth, outputVariable, errorHandling, and defaultValue fields, plus responseType (json or text). For how to create and update flows from code, see Manage flows as code.

Handle responses

Each step stores its result under the configured Output variable. Fetch URL stores either a parsed JSON value or text. Make API Call stores the full parsed JSON response unless responseMapping selects fields. The output variable holds only the response body. The HTTP status code isn’t stored in it.

Reference a full result or a nested property in a later step:

{{api_result}}
{{api_result.data.id}}

Replace api_result with your output variable and data.id with the path to a property in the response.

Handle errors

A request fails when the server returns a status outside the 2xx range, the request times out, the URL fails validation, or a referenced secret doesn’t exist. To choose what happens next, open the step’s error handling control and choose a mode:

Continue on error: the flow keeps going. The step writes its default value to the output variable, or null if no default is set. For a Make API Call step with responseMapping, Runtype applies the mapping to the default value. With no default value, that output is an empty object.
Stop on error: the run ends with an error.
Use fallbacks: Runtype retries the step. Add one Retry per attempt, each with an optional Delay in milliseconds. If every retry fails, Runtype marks the step as failed, the flow keeps going, and the output variable isn’t set.

A step that you add in the Flow editor starts with Stop on error. A step in a flow definition that has no errorHandling field continues on error. You can set the default value only in a flow definition, with the defaultValue field.

When a step continues on error, add a Conditional Logic step to check the output before you use it. For how error handling works across all step types, see Error handling.

Troubleshoot requests

These errors are the most common:

URL validation failed: the URL uses http:// or points to localhost or a private IP address. Use a public HTTPS endpoint.
Redirect blocked or Too many redirects: the server redirected to a URL that fails these rules, or redirected more than 5 times. Request the final URL directly.
requires secrets that are not configured: a {{secret:NAME}} reference names a secret that doesn’t exist. Add the secret in Settings > Secrets.
HTTP followed by a status code: the server rejected the request. The error message includes the response body, which usually names the problem.
Timeout: the server didn’t respond within the timeout. Increase http.timeout in the flow definition, or check whether the endpoint is slow.

For more diagnosis steps, see HTTP request failure.

Build dynamic URLs

Build a URL from flow variables when the endpoint depends on earlier data:

https://api.example.com/{{RESOURCE}}/{{ITEM_ID}}?filter={{SEARCH_TERM}}

Replace RESOURCE, ITEM_ID, and SEARCH_TERM with the names of flow variables.

Rate-limit requests

Add a Wait Until step between requests when an external API limits request frequency. Use this sequence:

Add a Fetch URL or Make API Call step.
Add a Wait Until step and enter 1000 milliseconds for a one-second pause.
Add the next Fetch URL or Make API Call step.

Test a step

Test an HTTP step with the inputs from the most recent flow run. Run the flow at least once before you test a step. The test uses the current step configuration, including unsaved edits, and does not save its output to run history.

To test a step, follow these steps:

Run the flow once.
Open the step’s action menu.
Choose Test with last run.
Review Previous output and New output in the test panel.
If Runtype asks you to confirm side effects, review the warning and click Run anyway.

Apply best practices

Apply these practices when you build HTTP steps:

Validate responses: check that the output variable is set, isn’t null, and has the fields that you expect before you use it.
Handle errors: choose an error handling mode for each step, and use conditional logic to branch on a failed request.
Protect secrets: reference credentials with {{secret:SECRET_NAME}} instead of hardcoding them.
Choose the right step: use Fetch URL for direct HTTP requests or page content, and use Make API Call for JSON APIs. To collect every page of a paginated API, use Paginate API. To wait until an endpoint reports a status, use Wait Until polling.

Next steps

Continue with these guides:

Using transform-data steps: process fetched data.
Using conditional steps: branch on response values.
Flow variables and templates: build dynamic requests.
Managing secrets: store credentials for HTTP requests.
Manage flows as code: define HTTP steps through the API or SDK.
Calling TypeSafe directly: call a judgment model from a Make API Call step and branch on its answer.
Using crawl, search, and paginate-api steps: crawl a website, run an AI search, and page through an API.
Using tool-call steps: call a saved or built-in Tool from a Flow step.