Settings

Model routing and data flow

Every model call you make follows a data path: it leaves Runtype, passes through zero or more gateways, and reaches the company that runs the model. This page explains how Runtype chooses that path, how bring your own key (BYOK) changes it, and which companies can process your prompts.

How a model name becomes a provider

Runtype accepts two model-name forms. The form determines whether Runtype or you choose the provider.

The following table compares the two model-name forms:

Model nameExampleWho picks the provider
Routed nameclaude-sonnet-4-5Runtype, using the routing configuration
Provider-pinned nameanthropic:claude-sonnet-4-5You, by selecting the provider directly

The dashboard model picker uses routed names. Routed names remain stable as providers add or retire model versions. Flow and Agent templates use routed names for this reason.

For the Claude, GPT, and Grok families, the routed name resolves to the Vercel AI Gateway unless your organization has selected that model’s first-party provider as its BYOK authority. When it has, the routed name calls the first-party API directly and the gateway is not used. Without that selection, the model’s first-party provider is the fallback when the gateway is unavailable to your account. Gemini text families resolve the other way round: the routed name calls the Google API first, and the Vercel AI Gateway is the fallback when your account has no Google credential. Gemini image-generation models (for example gemini-2.5-flash-image and gemini-3-pro-image) are the exception: their routed name always resolves to the Vercel AI Gateway and has no Google fallback. Some models route through the Cloudflare AI Gateway or run on Cloudflare Workers AI instead. A gateway routes and meters requests in front of the provider. It sees the request, and the provider runs the inference.

To choose the provider explicitly, use the provider-pinned form. Use it on a prompt step or an Agent. Define the model through the Runtype API, SDK, Runtype MCP server, or a code-authored Flow.

Which key pays for the call

Runtype resolves credential authority per provider, not per model. Open Settings > Models > Providers to view or change it.

Saving or rotating an organization connection only stores the credential. This behavior applies to OpenAI, Anthropic, Google, xAI, Mixlayer, Vercel AI Gateway, Amazon Bedrock, Vertex AI, Vertex AI (Claude), and OpenAI-compatible endpoints. Select Use for the organization to make one connection the provider-wide BYOK authority. OpenAI, Anthropic, Google, xAI, Mixlayer, and Vercel AI Gateway also support Platform Key for Runtype-managed custody. Amazon Bedrock, Vertex AI, Vertex AI (Claude), and OpenAI-compatible endpoints support BYOK only. Individual model bindings are compatibility references. They do not override the organization-wide choice. If your organization requires BYOK and its selected connection becomes unavailable, calls fail instead of switching silently to a platform credential.

Personal accounts and providers that remain member-scoped keep member-scoped compatibility behavior. Runtype prefers the member’s configured provider credential and may use an allowed platform credential when no member credential resolves. In Providers, shared organization connections appear separately from personal keys. The view distinguishes a stored connection from the connection selected for organization-wide use.

Platform keys exist for Anthropic, OpenAI, Google, xAI, Mixlayer, and Cloudflare Workers AI, plus the Vercel and Cloudflare AI gateways. Amazon Bedrock, Vertex AI, Vertex AI (Claude), and OpenAI-compatible endpoints support BYOK only, so model calls to these providers do not use a Runtype credential. These providers are still organization-owned connections even though they have no Platform Key option.

Selecting Use for the organization for a model’s first-party provider is what sends its routed names to that provider directly, billed to your own account. You do not need to pin the provider-specific model name. A routed name still resolves to Vercel AI Gateway when no such selection exists, and then uses the organization’s selected Vercel authority: a selected Vercel connection bills your Vercel account, while Platform Key uses a Runtype-managed Vercel credential. An explicitly gateway-pinned name always stays on the gateway.

Provider failover and the uptime tradeoff

The same model can be available from multiple providers. Claude models, for example, are served by Anthropic directly, AWS Bedrock, and Google Vertex AI. A gateway can fail over among them during an outage. Your prompt can then reach AWS or Google when you expect Anthropic.

Runtype pins gateway traffic for Anthropic models to Anthropic’s first-party API. Silent cross-cloud failover is disabled.

Provider pinning creates the following tradeoff:

  • Predictable data path: Your build determines which companies can process a prompt. The path does not change during load or an incident.
  • No cross-cloud continuity: During an Anthropic outage, an Anthropic request fails instead of being silently served from another cloud.

Runtype uses a predictable data path instead of invisible cross-provider failover. To add resilience, configure an explicit fallback chain. The flow records when it uses the chain.

Configure your own failover

Prompt steps support fallback chains that run entries in order until one succeeds. Each entry can retry the same model, use a different model, or return a fixed message. Choose what triggers the chain: a step error, an empty reply, or a call that exceeds the duration you set.

A cross-provider fallback, such as Anthropic Claude first and a Google or OpenAI model second, gives you continuity without silent gateway failover. You define the provider choices in your Flow, and each run records when it uses a fallback. For batch-specific guidance, see Handling batch failures.

Automatic recovery on platform keys

A prompt step with no error-handling setting of its own can still recover from a provider failure. This applies only to a call that Runtype funds. Runtype adds a recovery chain to the step when all of the following are true:

  • The step has no error-handling setting.
  • The model call ran on a Runtype platform key.
  • The provider returned a transient failure, such as an overload, a rate limit, or a response stream that stopped producing text, or it refused the model for your account (an entitlement error). A refusal is handled differently from a transient failure: it only tries the same model on another provider, never a smaller substitute, so a model no provider will serve you still fails with the provider’s message instead of quietly answering as something else.
  • The call returned no text before it failed.

The added chain tries the same model on another provider that serves it. Its last entry is a smaller model from a different provider. When no other provider serves that model, the chain retries the same call once before the model swap. The run records each attempt, so the Fallback field names the model that produced the answer.

A call on your own key doesn’t qualify. This covers an organization connection selected under Use for the organization and a personal provider key. Runtype doesn’t move such a call onto a platform credential. That would bill Runtype for a call you chose to fund. It would also send the prompt to a provider outside the path your selection defines. A BYOK prompt step with no error-handling setting fails on the first transient provider failure, and the flow records the step as failed.

Set Use fallbacks on every BYOK prompt step that must survive a provider incident. A chain you configure runs on platform keys and on your own keys alike. Each entry resolves its provider and credential the same way the first attempt did.

Who can see your data, based on your choices

Runtype sends requests to a model or tool provider only when your build selects it. No background pool of vendors processes your content.

The following table shows which companies process prompt content for each build choice:

What your build doesWho processes your prompt content
Runs a model with a platform keyRuntype infrastructure, the gateway in the path, and the selected model provider
Runs a model with your BYOK keyThe same path. The model provider bills you and processes the request under your agreement
Calls a built-in tool, such as web search, scraping, or a code sandboxThe tool provider, and only the query, URL, or code that you send
Runs the self-hosted runtime with your own keysYour infrastructure and model provider. The model request goes directly to the provider.

These choices have three consequences:

  • Provider selection controls the path. If your build does not select a provider, that provider does not process the request. Pinning prevents a gateway from selecting another provider for the request.
  • BYOK changes the provider relationship. When a call uses your key, the provider’s terms with you govern that content, not the terms between Runtype and the provider.
  • Self-hosted isolation depends on telemetry. The self-hosted runtime includes an opt-in telemetry reporter that is disabled by default. When enabled, lifecycle events sent to Runtype can include step output and error messages, as well as counts and timings.

Inspect a run’s route

You can inspect the route after a run. Open a run from Logs. The execution detail includes the following fields:

  • Model: the model that ran the step.
  • Model resolved: the configured and provider-specific names when they differ.
  • Fallback: the fallback model that produced the answer when a fallback chain runs.

For an account-wide view, open Settings > Models and select the Providers tab. Each provider shows whether calls use your key or the Runtype plan key. It also links to the Flows and Agents that use the provider.

Next steps

Use these guides to configure routing and inspect model calls: