Model routing and data flow

Every model call you make follows a data path: it leaves Runtype, passes through zero or more gateways, and reaches the company that runs the model. This page explains how Runtype chooses that path, how bring your own key (BYOK) changes it, and which companies can process your prompts.

How a model name becomes a provider

With Require your own model keys enabled, every route requires a selected organization connection. Runtype platform keys and platform fallback are unavailable, even when a saved provider setting normally permits them. See Require your own model keys.

The routing behavior in this section assumes workspace ZDR is disabled. With ZDR enabled, supported routed models use Vercel AI Gateway regardless of native BYOK selection. Explicit native routes and custom gateway origins are refused. Every Gateway call enforces zero data retention; fallback attempts do not relax the policy. GPT-Live voice is the one exception: it runs only through a direct OpenAI connection whose OpenAI organization was detected as ZDR. See Require zero data retention for inference.

A gateway is a service that routes and meters requests in front of a model provider. The gateway sees the request, and the provider runs the inference. Runtype accepts three model-name forms, and the form determines whether Runtype or you choose the provider and the gateway.

The following table compares the three model-name forms:

The dashboard model picker uses routed names. Routed names remain stable as providers add or retire model versions. Flow and Agent templates use routed names for this reason.

For a routed name, the model family sets the first provider that Runtype tries and the fallback it uses when the first provider is unavailable to your account. The following table shows the usual route for each family:

Some models don’t follow their family’s usual route. For example, gemini-3-flash runs on Google with no fallback, and grok-4-fast runs on Vercel AI Gateway with no xAI fallback. Some models also pass through the Cloudflare AI gateway before they reach the provider.

When your organization selects its own key for Anthropic, OpenAI, or xAI, routed names for that provider’s models call the provider directly, and the gateway isn’t used. You don’t need to switch to the provider-pinned name. For how to select a key, see Which key pays for the call.

To choose the provider explicitly, use the provider-pinned form on a prompt step or an Agent. To keep a call on a gateway, use the gateway-pinned form. A gateway-pinned name always stays on the named gateway, even when your organization selected its own key for the model’s provider. Set either form through the Runtype API, the SDK, the Runtype MCP server, or a code-authored Flow.

Provider-native tools pin the provider

Some built-in tools run inside a specific provider’s API, for example builtin:anthropic_web_search and builtin:openai_web_search. When a step with a routed model name uses one of these tools, the call goes directly to that tool’s provider and skips the gateway. Google Search grounding (builtin:google_search) goes directly to Google when your account has a Google key, and through Vercel AI Gateway otherwise.

A single call reaches only one provider. If a step uses provider-native tools from more than one provider, the provider of the first Anthropic, OpenAI, or xAI tool in the step’s tool list serves the call. Google Search grounding serves the call only when the step has no tools from those providers. The other provider’s tools don’t run. Runtype reports a PROVIDER_TOOLS_MIXED_OWNERS warning when it validates a Flow or Agent with this setup. To use tools from two providers, put them on separate prompt steps, each with a model that the tool’s provider serves. For the list of tools, see Built-in tools.

Which key pays for the call

Each model call runs on one of two kinds of key. A platform key is a key that Runtype holds and bills to your Runtype plan. A connection is your organization’s own provider key (BYOK). Runtype chooses the key per provider, not per model, so one choice applies to every model from that provider.

Only an organization admin can choose which key the organization uses for a provider. Adding a connection requires the Team plan or higher. On a lower plan, the API returns 402 with the code BYOK_NOT_AVAILABLE. An API key that changes provider keys needs the INTEGRATIONS:WRITE permission. For more information, see Billing and plans and Team members and permissions.

The following table shows which key types each model provider supports:

In an organization, saving or rotating a connection in the dashboard only stores it. Runtype starts using a connection after an admin selects Use for the organization next to it in Manage connections. The connection then shows the Used organization-wide badge. To return a provider to the platform key, an admin clicks Use Platform for organization. For the full steps, see Select a connection for the organization.

Through the API, send a PUT request to /v1/provider-keys/organization-policy with the provider field and a selection field. Set selection to an object with a connectionId, or to "platform" to use the platform key. Through MCP, call the select_organization_provider_credential tool. One API path also changes the key in use: when you create an OpenAI, Anthropic, Google, or xAI connection with isDefault set to true, Runtype selects that connection for the organization at the same time.

A key that you select for an individual model doesn’t override the organization’s choice for that model’s provider. When a routed Claude, GPT, or Grok name runs on Vercel AI Gateway, the call uses the organization’s choice for Vercel AI Gateway. A selected Vercel connection bills your Vercel account. The platform key bills your Runtype plan.

For an account-wide view, open Settings > Models and select the Providers tab. A provider whose calls use your own key shows the Organization key or Your key badge. In an organization, a provider whose calls use the platform key shows the Platform key badge. Each provider that is in use also shows how many Flows and Agents use it.

Personal accounts

On a personal account, add your key in Settings > Models > Providers. If you have several keys for one provider, click Set as default next to the key to use. The key then shows the Default badge. Runtype uses your key for that provider’s calls, and uses the platform key for providers where you have no key.

If calls fail after you select a connection

When your organization uses a connection and that connection becomes unavailable, calls to that provider fail. Runtype doesn’t switch them to the platform key. To restore calls, do one of the following:

• Rotate the connection’s key if the provider revoked it or it expired.
• Select a different connection with Use for the organization.
• For providers that support a platform key, click Use Platform for organization.

Provider failover and the uptime tradeoff

The same model can be available from multiple providers. Claude models, for example, are served by Anthropic directly, Amazon Bedrock, and Vertex AI. A gateway can fail over among them during an outage. Your prompt can then reach AWS or Google when you expect Anthropic.

Runtype pins gateway traffic for Anthropic models to Anthropic’s first-party API. Silent cross-cloud failover is disabled.

Provider pinning creates the following tradeoff:

• Predictable data path: Your build determines which companies can process a prompt. The path does not change during load or an incident.
• No cross-cloud continuity: During an Anthropic outage, an Anthropic request fails instead of being silently served from another cloud.

Runtype uses a predictable data path instead of invisible cross-provider failover. To add resilience, configure an explicit fallback chain. The flow records when it uses the chain.

When a model hosted on Cloudflare Workers AI has no available capacity, the call fails right away instead of waiting in a capacity queue. The run then moves to its fallback chain immediately, either the automatic recovery chain on platform keys or the fallbacks you configured on the step. The flow records the fallback.

Configure your own failover

Prompt steps support fallback chains that run entries in order until one succeeds. Each entry can retry the same model, use a different model, or return a fixed message. In the dashboard, select Use fallbacks on the step, then choose what starts the chain: The step errors, The reply is empty, or both. A third trigger, a call that runs longer than a duration you set, is available only through the API, the SDK, or a code-authored Flow. For more information, see Using prompt steps.

A cross-provider fallback, such as Anthropic Claude first and a Google or OpenAI model second, gives you continuity without silent gateway failover. You define the provider choices in your Flow, and each run records when it uses a fallback. For batch-specific guidance, see Handling batch failures.

Automatic recovery on platform keys

A prompt step with no error-handling setting of its own can still recover from a provider failure. This applies only to a call that Runtype funds. Runtype adds a recovery chain to the step when all of the following are true:

• The step has no error-handling setting.
• The model call ran on a platform key.
• The provider returned a transient failure, such as an overload, a rate limit, or a response stream that stopped producing text, or it refused the model for your account.
• The call returned no text before it failed.

The added chain tries the same model on another provider that serves it. Its last entry is a smaller model from a different provider. When no other provider serves that model, the chain retries the same call once before the model swap. For a refusal, the chain only tries the same model on other providers and never switches to a smaller model. If no provider serves the model to you, the call fails with the provider’s message. The run records each attempt, so the Fallback field names the model that produced the answer.

A call on your own key doesn’t qualify. This covers a connection selected with Use for the organization and a personal key. Runtype doesn’t move such a call onto a platform key. That would bill Runtype for a call you chose to fund. It would also send the prompt to a provider outside the path your selection defines. In a Flow, a prompt step on your own key with no error-handling setting fails on the first transient provider failure, and the flow records the step as failed.

Set Use fallbacks on every prompt step on your own key that must survive a provider incident. A chain you configure runs on platform keys and on your own keys alike. Each entry resolves its provider and key the same way the first attempt did.

Default retries for Agents

An Agent with no error-handling setting of its own retries a failed model call twice with the same model, first after 5 seconds and then after 10 seconds. These retries apply on platform keys and on your own keys. On a platform key, the Agent can also use the automatic recovery chain. To replace this default, configure error handling on the Agent.

Who can see your data, based on your choices

Runtype sends requests to a model or tool provider only when your build selects it. No background pool of vendors processes your content.

The following table shows which companies process prompt content for each build choice:

These choices have three consequences:

• Provider selection controls the path. If your build does not select a provider, that provider does not process the request. Pinning prevents a gateway from selecting another provider for the request.
• BYOK changes the provider relationship. When a call uses your key, the provider’s terms with you govern that content, not the terms between Runtype and the provider.
• Self-hosted isolation depends on telemetry. The self-hosted runtime includes an opt-in telemetry reporter that is disabled by default. When enabled, lifecycle events sent to Runtype can include step output and error messages, as well as counts and timings.

Inspect a run’s route

You can inspect the route after a run. Open a run from the logs, as described in Working with logs. The execution detail includes the following fields:

• Model: the model that ran the step.
• Model Resolved: the configured and provider-specific names when they differ.
• Fallback: the fallback model that produced the answer when a fallback chain runs.

Next steps

Use these guides to configure routing and inspect model calls:

• Platform keys vs. bring your own key (BYOK): compare key options
• Connecting AI model providers: add provider keys
• Using prompt steps: configure fallback models and triggers
• Prompt caching: review how routing affects cost and latency
• Available AI models: find the models you can use