> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.runtype.com/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.runtype.com/_mcp/server.

# Execute a synchronous site crawl

POST https://api.runtype.com/v1/runtime/crawl
Content-Type: application/json

Hosted fulfillment of the runtime SiteCrawler seam. Executes the same crawl capability the `crawl` flow step uses (Cloudflare Browser Rendering: submit, poll to completion, paginate results) and resolves once with the collected pages. Synchronous only: the async crawl_pending pause machinery is not available on this endpoint. Consumed by standalone `@runtypelabs/runtime` deployments via `HostedSiteCrawler`. Requires an Enterprise plan: ineligible accounts receive 403 (code `BYOC_PLAN_REQUIRED`).

Reference: https://docs.runtype.com/api-reference/runtype-api/self-hosting/execute-a-synchronous-site-crawl

## Authentication

- `Authorization` header (bearer token, required) — API key or Clerk session token

## Servers

- `https://api.runtype.com` (Production, default)
- `https://api.runtype-staging.com` (Staging)

## Request

### Body (application/json)

This endpoint expects an object.

- `url` (string, required) — Fully-resolved seed URL, passed through to the crawl service untouched. Note the platform crawl service drops query strings from the seed URL and follows absolute links only (service behavior, inherited by design).
- `limits` (V1RuntimeCrawlPostRequestBodyContentApplicationJsonSchemaLimits, optional)
- `options` (V1RuntimeCrawlPostRequestBodyContentApplicationJsonSchemaOptions, optional)

## Response

### 200

Crawl completed

- `metadata` (V1RuntimeCrawlPostResponsesContentApplicationJsonSchemaMetadata, required) — Implementation metadata folded into the runtime step’s result metadata.
- `pages` (list of V1RuntimeCrawlPostResponsesContentApplicationJsonSchemaPagesItems, required) — All collected result pages; the first entry is the terminal status page (the crawl step output’s `result` field).

## Errors

### 400 Bad Request Error

Invalid request, or Browser Rendering credentials are not configured

- `code` (enum, required) — Stable machine-readable failure code.
  - Allowed values: `CREDENTIALS_MISSING`, `CRAWL_FAILED`, `CRAWL_TIMEOUT`, `CRAWL_PENDING_UNSUPPORTED`
- `error` (string, required) — Human-readable failure message.

### 401 Unauthorized Error

Unauthorized

- `error` (string, required)
- `code` (string, optional) — Stable error code. `insufficient_scope` marks an API key that lacks a scope this request needs.
- `details` (list of ErrorDetailsItems, optional)
- `requiredScopes` (list of string, optional) — On `insufficient_scope`: every API key scope this request needs.

### 402 Payment Required Error

Monthly spend limit exceeded

- `error` (string, required)
- `code` (string, optional) — Stable error code. `insufficient_scope` marks an API key that lacks a scope this request needs.
- `details` (list of ErrorDetailsItems, optional)
- `requiredScopes` (list of string, optional) — On `insufficient_scope`: every API key scope this request needs.

### 403 Forbidden Error

Insufficient permissions, or the account is not on an Enterprise plan (code `BYOC_PLAN_REQUIRED`)

- `error` (string, required)
- `code` (string, optional) — Stable error code. `insufficient_scope` marks an API key that lacks a scope this request needs.
- `details` (list of ErrorDetailsItems, optional)
- `requiredScopes` (list of string, optional) — On `insufficient_scope`: every API key scope this request needs.

### 429 Too Many Requests Error

Daily execution limit or rate limit exceeded

- `error` (string, required)
- `code` (string, optional) — Stable error code. `insufficient_scope` marks an API key that lacks a scope this request needs.
- `details` (list of ErrorDetailsItems, optional)
- `requiredScopes` (list of string, optional) — On `insufficient_scope`: every API key scope this request needs.

### 500 Internal Server Error

Internal server error

- `error` (string, required)
- `code` (string, optional) — Stable error code. `insufficient_scope` marks an API key that lacks a scope this request needs.
- `details` (list of ErrorDetailsItems, optional)
- `requiredScopes` (list of string, optional) — On `insufficient_scope`: every API key scope this request needs.

### 501 Not Implemented Error

The crawl requested an async continuation, which the synchronous endpoint does not support

- `code` (enum, required) — Stable machine-readable failure code.
  - Allowed values: `CREDENTIALS_MISSING`, `CRAWL_FAILED`, `CRAWL_TIMEOUT`, `CRAWL_PENDING_UNSUPPORTED`
- `error` (string, required) — Human-readable failure message.

### 502 Bad Gateway Error

The crawl failed upstream (submit error, non-completed terminal status, …)

- `code` (enum, required) — Stable machine-readable failure code.
  - Allowed values: `CREDENTIALS_MISSING`, `CRAWL_FAILED`, `CRAWL_TIMEOUT`, `CRAWL_PENDING_UNSUPPORTED`
- `error` (string, required) — Human-readable failure message.

### 504 Gateway Timeout Error

The crawl did not reach a terminal status within completionTimeoutMs

- `code` (enum, required) — Stable machine-readable failure code.
  - Allowed values: `CREDENTIALS_MISSING`, `CRAWL_FAILED`, `CRAWL_TIMEOUT`, `CRAWL_PENDING_UNSUPPORTED`
- `error` (string, required) — Human-readable failure message.

## Types

### V1RuntimeCrawlPostRequestBodyContentApplicationJsonSchemaLimits

- `completionTimeoutMs` (double, optional) — Overall completion timeout for the crawl job (default 120000, floored at pollIntervalMs).
- `depth` (double, optional) — Crawl depth from the seed URL (min-1 normalized server-side).
- `maxPages` (double, optional) — Maximum number of pages to crawl (min-1 normalized server-side).
- `pollIntervalMs` (double, optional) — Poll interval for the server-side completion loop (default 2000, floor 1).

### V1RuntimeCrawlPostRequestBodyContentApplicationJsonSchemaOptions

- `authenticate` (V1RuntimeCrawlPostRequestBodyContentApplicationJsonSchemaOptionsAuthenticate, optional)
- `cookies` (list of map from string to any, optional)
- `formats` (list of string, optional)
- `gotoOptions` (map from string to any, optional)
- `jsonOptions` (V1RuntimeCrawlPostRequestBodyContentApplicationJsonSchemaOptionsJsonOptions, optional)
- `maxAge` (integer, optional)
- `modifiedSince` (string, optional)
- `options` (V1RuntimeCrawlPostRequestBodyContentApplicationJsonSchemaOptionsOptions, optional)
- `rejectRequestPattern` (V1RuntimeCrawlPostRequestBodyContentApplicationJsonSchemaOptionsRejectRequestPattern, optional)
- `rejectResourceTypes` (list of string, optional)
- `render` (boolean, optional)
- `setExtraHTTPHeaders` (map from string to string, optional)
- `source` (string, optional)
- `userAgent` (string, optional)
- `waitForSelector` (string, optional)

### V1RuntimeCrawlPostResponsesContentApplicationJsonSchemaMetadata

Implementation metadata folded into the runtime step’s result metadata.

- `crawlId` (string, required)
- `credentialSource` (enum, required)
  - Allowed values: `user`, `platform`
- `pagesFetched` (double, required)
- `resultPagesTruncated` (boolean, required)
- `status` (string, required)

### V1RuntimeCrawlPostResponsesContentApplicationJsonSchemaPagesItems

- `status` (string, required) — Status of the crawl job at the time this page was fetched.
- `browserSecondsUsed` (double, optional) — Browser rendering seconds consumed.
- `cursor` (string, optional) — Cursor for the next result page (absent on last).
- `finished` (double, optional) — Number of pages that finished crawling.
- `records` (list of map from string to any, optional) — Crawled page records in this result page.
- `total` (double, optional) — Total pages discovered by the crawl.

### ErrorDetailsItems

- `message` (string, required)
- `code` (string, optional) — Stable Runtype validation code (e.g. MISSING_REQUIRED_FIELD, INVALID_ENUM).
- `path` (string, optional) — Dotted JSON path to the offending field.
- `suggestion` (string, optional) — Actionable hint (allowed values, accepted range, …).

### V1RuntimeCrawlPostRequestBodyContentApplicationJsonSchemaOptionsAuthenticate

- `password` (string, optional)
- `username` (string, optional)

### V1RuntimeCrawlPostRequestBodyContentApplicationJsonSchemaOptionsJsonOptions

- `prompt` (string, optional)
- `schema` (map from string to any, optional)

### V1RuntimeCrawlPostRequestBodyContentApplicationJsonSchemaOptionsOptions

- `allowExternalLinks` (boolean, optional)
- `allowSubdomains` (boolean, optional)
- `excludePatterns` (list of string, optional)
- `ignoreSitemap` (boolean, optional)
- `includePatterns` (list of string, optional)
- `sitemapOnly` (boolean, optional)

### V1RuntimeCrawlPostRequestBodyContentApplicationJsonSchemaOptionsRejectRequestPattern

- string
- list of string

## Examples

**Request**

```json
{
  "url": "string"
}
```

**Response**

```json
{
  "metadata": {
    "crawlId": "string",
    "credentialSource": "user",
    "pagesFetched": 1.1,
    "resultPagesTruncated": true,
    "status": "string"
  },
  "pages": [
    {
      "status": "string",
      "browserSecondsUsed": 1.1,
      "cursor": "string",
      "finished": 1.1,
      "records": [
        {}
      ],
      "total": 1.1
    }
  ]
}
```

**SDK Code**

```python
import requests

url = "https://api.runtype.com/v1/runtime/crawl"

payload = { "url": "string" }
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.json())
```

```javascript
const url = 'https://api.runtype.com/v1/runtime/crawl';
const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: '{"url":"string"}'
};

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}
```

```go
package main

import (
	"fmt"
	"strings"
	"net/http"
	"io"
)

func main() {

	url := "https://api.runtype.com/v1/runtime/crawl"

	payload := strings.NewReader("{\n  \"url\": \"string\"\n}")

	req, _ := http.NewRequest("POST", url, payload)

	req.Header.Add("Authorization", "Bearer <token>")
	req.Header.Add("Content-Type", "application/json")

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(res)
	fmt.Println(string(body))

}
```

```ruby
require 'uri'
require 'net/http'

url = URI("https://api.runtype.com/v1/runtime/crawl")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Post.new(url)
request["Authorization"] = 'Bearer <token>'
request["Content-Type"] = 'application/json'
request.body = "{\n  \"url\": \"string\"\n}"

response = http.request(request)
puts response.read_body
```

```java
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;

HttpResponse<String> response = Unirest.post("https://api.runtype.com/v1/runtime/crawl")
  .header("Authorization", "Bearer <token>")
  .header("Content-Type", "application/json")
  .body("{\n  \"url\": \"string\"\n}")
  .asString();
```

```php
<?php
require_once('vendor/autoload.php');

$client = new \GuzzleHttp\Client();

$response = $client->request('POST', 'https://api.runtype.com/v1/runtime/crawl', [
  'body' => '{
  "url": "string"
}',
  'headers' => [
    'Authorization' => 'Bearer <token>',
    'Content-Type' => 'application/json',
  ],
]);

echo $response->getBody();
```

```csharp
using RestSharp;

var client = new RestClient("https://api.runtype.com/v1/runtime/crawl");
var request = new RestRequest(Method.POST);
request.AddHeader("Authorization", "Bearer <token>");
request.AddHeader("Content-Type", "application/json");
request.AddParameter("application/json", "{\n  \"url\": \"string\"\n}", ParameterType.RequestBody);
IRestResponse response = client.Execute(request);
```

```swift
import Foundation

let headers = [
  "Authorization": "Bearer <token>",
  "Content-Type": "application/json"
]
let parameters = ["url": "string"] as [String : Any]

let postData = JSONSerialization.data(withJSONObject: parameters, options: [])

let request = NSMutableURLRequest(url: NSURL(string: "https://api.runtype.com/v1/runtime/crawl")! as URL,
                                        cachePolicy: .useProtocolCachePolicy,
                                    timeoutInterval: 10.0)
request.httpMethod = "POST"
request.allHTTPHeaderFields = headers
request.httpBody = postData as Data

let session = URLSession.shared
let dataTask = session.dataTask(with: request as URLRequest, completionHandler: { (data, response, error) -> Void in
  if (error != nil) {
    print(error as Any)
  } else {
    let httpResponse = response as? HTTPURLResponse
    print(httpResponse)
  }
})

dataTask.resume()
```