Connecting AI model providers | Runtype Documentation

Your Runtype account includes platform keys, so you can use popular AI models without adding provider accounts. Connect your own keys in Settings if you want dedicated rate limits, direct provider billing, or the provider’s full model catalog.

For a full comparison, see Platform Keys vs. Bring Your Own Key (BYOK).

Before you begin

To connect your own keys, you need the following:

A Team plan or higher. The Launch plan uses platform keys only, and adding a key returns an error. For plan details, see Billing and plans.
An account with the AI provider that you want to connect, with permission to create API keys.
In an organization, the organization admin role. Only admins can add, change, delete, or select the organization’s connections. For roles, see Team members and permissions.

Use platform keys

Your account includes platform keys for models from OpenAI, Anthropic, Google, xAI, Vercel AI Gateway, Cloudflare Workers AI, and open-source models through Mixlayer (SDK). You can build Flows and Agents without additional setup. For the models that each key covers, see Available AI models.

Supported providers

You can connect your own keys for these providers:

Direct providers: OpenAI, Anthropic, Google, and xAI.
Inference providers: Tinfoil, Amazon Bedrock, Vertex AI, Vertex AI (Claude), and Mixlayer (SDK).
Gateway providers: Vercel AI Gateway.
Custom inference endpoints: Any OpenAI-compatible (Chat Completions or Responses) endpoint, through Custom inference endpoint.

Cloudflare Workers AI models run on platform keys only. You don’t add a key for them.

Add a provider

To add a provider, start with a model or a provider connection. You can connect several providers and choose a model from any of them for each Flow step or Agent.

Start from a model

To add a model and its key, follow these steps:

On Settings > Models, open the Models tab.
Click Add model and browse or search the catalog.
Select a model.
If the model needs a key that you haven’t added, enter it when prompted.
In an organization, click Use for the organization to confirm. The new connection becomes the organization’s connection for that provider.

Start from a provider

To add a provider key first, follow these steps:

On Settings > Models, open the Providers tab.
On the provider card, click Add your own key. For Vertex AI, Vertex AI (Claude), Amazon Bedrock, and Custom inference endpoint, click Configure. In an organization, click Add connection on any card.
Enter the key details.
Click Save API Key. In an organization, click Save connection. For Vertex AI, Vertex AI (Claude), Amazon Bedrock, and Custom inference endpoint, click Save Configuration.
In an organization, select the connection for the organization. For the steps, see Select a connection for the organization.
Enable the models that you want to use.

Select a connection for the organization

In an organization, a connection that you save on the Providers tab isn’t used until an organization admin selects it. After an admin selects it, every teammate’s runs, plus scheduled and API-triggered runs, use it. Saving or rotating a connection doesn’t change which connection the organization uses.

To select a connection, follow these steps:

On Settings > Models, open the Providers tab.
On the provider card, click Manage connections.
Next to the connection, click Use for the organization.
Click Use this connection.

Get provider credentials

Create credentials in each provider’s dashboard, then add them in Settings > Models on the Providers tab.

OpenAI

To add an OpenAI key, follow these steps:

Sign in to OpenAI Platform.
On API keys, create a secret key.
Copy the key.
On the OpenAI card in Runtype, click Add your own key and paste the key.

Anthropic

To add an Anthropic key, follow these steps:

Sign in to Anthropic Console.
On API Keys, create a key.
Copy the key.
On the Anthropic card in Runtype, click Add your own key and paste the key.

Google

To add a Google key, follow these steps:

Sign in to Google AI Studio.
Create an API key and copy it.
On the Google card in Runtype, click Add your own key and paste the key.

xAI

To add an xAI key, follow these steps:

Sign in to xAI Console.
Generate an API key and copy it.
On the xAI card in Runtype, click Add your own key and paste the key.

Tinfoil

Tinfoil has no card on the Providers tab. You add the key when you add a Tinfoil model.

To add a Tinfoil key, follow these steps:

Sign in to Tinfoil Platform.
Open API Keys.
Create a key and copy it.
On Settings > Models, open the Models tab and click Add model.
Select a Tinfoil model and paste the key when prompted.

Mixlayer (SDK)

Mixlayer models use platform keys by default. To use a dedicated Mixlayer key, follow these steps:

Create an account on Mixlayer.
On the Mixlayer API keys page, generate a key.
Copy the key.
On the Mixlayer (SDK) card in Runtype, click Add your own key and paste the key.

Vercel AI Gateway

Vercel AI Gateway models use platform keys by default. To use your own Vercel account and limits, add your own key.

To add a Vercel AI Gateway key, follow these steps:

Open Vercel AI Gateway.
Create an API key.
On the Vercel AI Gateway card in Runtype, click Add your own key and paste the key.

Amazon Bedrock

To add Amazon Bedrock credentials, follow these steps:

Create an AWS access key with Bedrock permissions.
On the Amazon Bedrock card in Runtype, click Configure.
Enter the value in AWS Access Key ID.
Enter the value in AWS Secret Access Key.
Select a region in AWS Region.
Click Save Configuration.

Vertex AI

Vertex AI works with your own credentials only. Use Vertex AI for Gemini models and Vertex AI (Claude) for Claude models. Each is a separate connection, and each connection can use its own service account, project, and region.

To add a Vertex AI connection, follow these steps:

Create a Google Cloud service account with the roles/aiplatform.user role.
On the Vertex AI card for Gemini models, or the Vertex AI (Claude) card for Claude models, click Configure.
Click Upload JSON File and choose the service account key file.
Enter the project ID in GCP Project ID and the region in Region.
Click Save Configuration.

Custom inference endpoints

A custom inference endpoint is any server that speaks the OpenAI Chat Completions or Responses API. This includes Groq, Fireworks, Perplexity, Ollama, vLLM, LiteLLM, and hosted gateways such as NVIDIA (https://integrate.api.nvidia.com/v1). Custom inference endpoints work with your own credentials only. They have no platform key. In the API, MCP, and SDK, the provider ID is generic-openai.

Supported API format

Each custom inference endpoint connection speaks one API format. Choose it in API format in the dialog, or set it in the connection’s apiFormat setting:

`apiFormat`	Request path
`chat-completions` (default)	`POST {base URL}/chat/completions`
`responses`	`POST {base URL}/responses`

Model discovery uses GET {base URL}/models for both formats. The Anthropic Messages format (/messages) isn’t supported on this connector.

To use the Responses format, choose Responses in API format. Through the API, set settings.apiFormat to responses when you create or update the connection (POST /v1/provider-keys), or pass api_format: "responses" to the create_provider_key or update_provider_key MCP tool. On the Responses format, Runtype sends store: false on every request and never references a stored response, so the endpoint doesn’t need to keep any state between requests.

The following table answers the common questions about what Runtype sends:

Question	Answer
Can I use the Responses API?	Yes. Choose Responses in API format, or set the connection’s `apiFormat` to `responses`.
Can I set a reasoning effort, such as `medium`?	Yes, after you declare the model as a reasoning model. Runtype sends `reasoning_effort` on Chat Completions and `reasoning.effort` on Responses. See Reasoning models.
Does Runtype send `max_tokens` on every request?	No. Runtype sends `max_tokens` only when the agent or step sets Max output tokens. The model’s own Max Output Tokens setting is the highest limit that editors offer.
Can I turn off streaming?	No. Runtype always requests a streamed response (`stream: true`), so the endpoint must support streaming. Supports streaming is saved model information only.

The dialog shows the full request URL and the request body shape for the chosen format under Base URL, so you can check them before you save.

The base URL must meet these requirements:

It uses HTTPS.
It points to a publicly reachable hostname. Runtype rejects localhost, private IP addresses, and cloud metadata hosts. To use a server that runs on your own machine or private network, such as Ollama or vLLM, expose it through a public HTTPS tunnel or proxy first.
It ends at the API root. Runtype adds /v1 when the URL doesn’t already end with it, so enter the URL up to and including /v1. For example, https://api.groq.com/openai/v1.

To add a custom inference endpoint, follow these steps:

Get the endpoint base URL and API key from your provider.
On the Custom inference endpoint card in Runtype, click Configure.
Enter the base URL and API key.
In API format, keep Chat Completions or choose Responses.
Add the models that you want to enable with one of these methods:
- Auto-discover: Click Discover Models. Runtype requests the endpoint’s /v1/models list and shows the returned models.
- Add manually: If the endpoint doesn’t support /v1/models, enter each model ID.
Optional: Expand a model’s Advanced Settings to change its defaults.
Click Save Configuration.

On plans without BYOK, the dashboard locks Custom inference endpoint and prompts you to add a payment method.

Keep your API keys private. Don’t share them publicly or commit them to version control.

Model settings

Each model that you add has the following settings under Advanced Settings:

Setting	Default	When to change it
Max Output Tokens	32768	Lower it to the endpoint’s real output limit. Editors offer values up to this limit. Runtype doesn’t send it on every request
Response Formats	text, JSON	Remove JSON if the endpoint doesn’t support JSON output
Input Cost / 1k Tokens	0	Enter your provider’s price to see accurate costs in usage reports
Output Cost / 1k Tokens	0	Enter your provider’s price to see accurate costs in usage reports
Supports streaming	Selected	Saved as model information only. Runtype always streams, so clearing it doesn’t turn streaming off
Reasoning model	Off	Turn it on if the endpoint accepts a reasoning effort for the model

With the default costs of 0, usage and cost reports show $0 for these models.

Troubleshoot model discovery

Discovery waits 10 seconds for a response and doesn’t follow redirects. The endpoint must return an OpenAI-style list: a JSON object with a data array of models, each with an id.

The following errors are the most common:

Only HTTPS URLs are supported, Localhost URLs are not allowed, or Private/internal IP addresses are not allowed: The base URL doesn’t meet the base URL requirements. Use a public HTTPS URL.
Models endpoint returned 401 or 403: The provider rejected the API key. Check the key and its permissions.
Models endpoint returned 404: The base URL path is wrong. Enter the URL up to and including /v1.
redirects are not followed: The endpoint redirects to another address. Enter the final address as the base URL.
Invalid JSON response from models endpoint or Response missing expected “data” array: The endpoint doesn’t return an OpenAI-style model list. Add the model IDs manually.
An aborted-request error after about 10 seconds: The endpoint didn’t answer in time. Try again, or add the model IDs manually.

Reasoning models on custom inference endpoints

Runtype has no catalog entry for a model served from a custom endpoint, so it can’t tell whether the endpoint accepts reasoning_effort for that model. To declare a reasoning model, follow these steps:

On the Custom inference endpoint card, click Manage keys. In an organization, click Manage connections.
Next to the key, click Edit key.
Expand the model’s Advanced Settings.
Turn on Reasoning model.
Choose a Default reasoning effort. Runtype sends this value when an agent or flow step sets no reasoning of its own. Provider default (send nothing) sends no value.
Click Update Configuration.

Through the API, set reasoning on the model entry when you sync models, or settings.customModel.reasoning on the model configuration: { "supported": true, "defaultEffort": "none" }. The sync_provider_models and create_model_config MCP tools accept the same declaration as reasoning_supported and reasoning_default_effort. A default effort without reasoning_supported: true is rejected.

The declaration is the only opt-in. Runtype never sends reasoning_effort to a model that isn’t declared, or that’s declared with supported: false, even when an agent or step sets a reasoning effort. On a declared model, Runtype chooses the value as follows:

reasoning: false on the agent or step sends reasoning_effort: "none".
A reasoning effort set on the agent or step (reasoning.reasoningEffort) is sent as given.
Otherwise, Runtype sends the model’s default effort, if you set one.

Some providers treat an omitted reasoning_effort as unbounded. Cerebras, for example, lets a Qwen model spend its whole output budget on reasoning and return no text. For such an endpoint, declare the model and set the default effort to none or low. The model’s reasoning streams as reasoning blocks either way, including into a Persona widget that shows reasoning.

On a connection that uses the Responses format, Runtype sends the same value as reasoning.effort, and asks for a reasoning summary (reasoning.summary: "auto") unless the effort is none. A model that isn’t declared gets no reasoning field on either format.

System message position on custom inference endpoints

A conversation can carry a system message after the first user turn. For example, a caller can send per-turn context last so that the leading system prompt stays identical across turns and the provider’s prompt cache keeps working. On a custom inference endpoint, Runtype keeps that message at its position by default, the same as on OpenAI and xAI.

Some chat templates reject a system message that isn’t first. Runtype automatically moves such messages into the leading system prompt for Mistral- and Gemma-family model IDs (mistral, mixtral, ministral, magistral, codestral, and gemma). For any other model on an endpoint that rejects them, declare the opt-out on the model:

Through the API, set midConversationSystemMessages: "hoist" on the model entry when you sync models, or settings.customModel.midConversationSystemMessages on the model configuration.
Through MCP, pass mid_conversation_system_messages: "hoist" to sync_provider_models or create_model_config. The field applies only to generic-openai models. Any other provider rejects it.

The dashboard has no toggle for this setting. "preserve" is accepted as an explicit value and matches the default.

Enable models

To choose the models that appear in model pickers, follow these steps:

On Settings > Models, open the Models tab.
Click Add model.
Select the models that you want to use.
Confirm that the enabled models appear in model pickers across Runtype, including Flows, Agents, and the Playground.

Enable only the models that you use. This keeps model pickers shorter and more focused.

Connect a custom inference endpoint through the API, MCP, or SDK

You can connect a custom inference endpoint without the dashboard. Use the Runtype API, the Runtype MCP server, or the Runtype SDK. To add a key for another provider, such as OpenAI or Anthropic, send the same create request with that provider’s ID in provider.

Creating a provider key, discovering or syncing models, and creating a model configuration require BYOK on your plan. Without it, these calls return 402 with code BYOK_NOT_AVAILABLE. In an organization, creating, changing, syncing, deleting, and selecting connections require the organization admin role. Use a Runtype API key with the INTEGRATIONS:WRITE scope for provider keys and the MODELS:WRITE scope for model configurations. For more information, see Managing API keys.

The setup has three steps. Create a generic-openai provider key with the endpoint’s base URL. Discover its models or supply a manual list. Sync the models that you want to enable. In an organization, the create request returns a connection ID, and an organization admin selects that connection in a separate call.

A sync request sets the full model list for the connection. Runtype removes any previously synced model that you leave out, and an empty models array removes every model. Flows and Agents that use a removed model stop working. Include every model that you want to keep in each sync.

The following TypeScript SDK sample creates the key, discovers models, and syncs the model list:

TypeScript SDK

import { RuntypeClient } from '@runtypelabs/sdk'
const client = new RuntypeClient({
  apiKey: process.env.YOUR_RUNTYPE_API_KEY!,
})
const key = await client.providerKeys.create({
  provider: 'generic-openai',
  name: 'customer-support-endpoint',
  apiKey: process.env.YOUR_PROVIDER_API_KEY!,
  settings: {
    baseUrl: process.env.YOUR_OPENAI_COMPATIBLE_BASE_URL!,
  },
})
const { models } = await client.providerKeys.discoverModelsForKey(key.id)
await client.providerKeys.syncModels(
  key.id,
  models.map((model) => ({ id: model.id }))
)

In an organization, an admin can then select the connection for the organization. Call selectOrganizationProviderCredential with provider: 'generic-openai' and selection: { connectionId: key.id }.

Set these environment variables before you run the sample:

YOUR_RUNTYPE_API_KEY: Your Runtype API key.
YOUR_PROVIDER_API_KEY: Your custom inference endpoint’s API key.
YOUR_OPENAI_COMPATIBLE_BASE_URL: The provider endpoint base URL.

The following cURL commands create the key, discover models, and sync the model list. In an organization, the final command selects the connection for the organization:

cURL

# 1. Create the provider key
curl -X POST https://api.runtype.com/v1/provider-keys \
  -H "Authorization: Bearer YOUR_RUNTYPE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "generic-openai",
    "name": "customer-support-endpoint",
    "apiKey": "YOUR_PROVIDER_API_KEY",
    "settings": {
      "baseUrl": "YOUR_OPENAI_COMPATIBLE_BASE_URL"
    }
  }'
# The first request returns the credential ID used by the following commands.
CREDENTIAL_ID=YOUR_CREDENTIAL_ID
# 2. Discover models from the endpoint's /v1/models
curl -X POST \
  "https://api.runtype.com/v1/provider-keys/${CREDENTIAL_ID}/discover-models" \
  -H "Authorization: Bearer YOUR_RUNTYPE_API_KEY"
# 3. Sync the full list of models that you want to enable
curl -X POST \
  "https://api.runtype.com/v1/provider-keys/${CREDENTIAL_ID}/sync-models" \
  -H "Authorization: Bearer YOUR_RUNTYPE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "models": [
      {"id": "YOUR_MODEL_ID"}
    ]
  }'
# 4. Organization only: select the connection for the organization
curl -X PUT https://api.runtype.com/v1/provider-keys/organization-policy \
  -H "Authorization: Bearer YOUR_RUNTYPE_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"provider\": \"generic-openai\",
    \"selection\": {
      \"connectionId\": \"${CREDENTIAL_ID}\"
    }
  }"

Replace the placeholders in the cURL commands with these values:

YOUR_RUNTYPE_API_KEY: Your Runtype API key.
YOUR_PROVIDER_API_KEY: Your custom inference endpoint’s API key.
YOUR_OPENAI_COMPATIBLE_BASE_URL: The provider endpoint base URL.
YOUR_CREDENTIAL_ID: The providerKey.id value returned by the first request. Assign it to CREDENTIAL_ID in the cURL commands.
YOUR_MODEL_ID: A model ID that the endpoint serves.

Through MCP, call the create_provider_key, discover_provider_models, and sync_provider_models tools for the same sequence. Then call select_organization_provider_credential to select the connection for the organization. To register one custom model without discovery, call create_model_config with provider: "generic-openai", model_id, and base_url.

Use a synced model

Synced models appear in model pickers across Runtype. Through the API, the SDK, or MCP, reference a synced model as generic-openai: followed by the model ID that the endpoint serves. For example, set model on an agent or prompt step to generic-openai:llama-3.3-70b.

Rotate or change a key

To replace a key with a new one, follow these steps:

On Settings > Models, open the Providers tab.
On the provider card, click Manage keys. In an organization, click Manage connections.
Next to the key, click Edit key.
Enter the new key.
Click Update Key. In an organization, click Update connection. For Vertex AI, Vertex AI (Claude), Amazon Bedrock, and Custom inference endpoint, click Update Configuration.

Rotating an organization connection doesn’t change which connection the organization uses. Through the API, send a PATCH request to /v1/provider-keys/YOUR_CREDENTIAL_ID with the new apiKey.

If you have several personal keys for one provider, click Set as default next to the key that you want Runtype to use.

With Require your own model keys disabled, to move an organization back to the platform key for OpenAI, Anthropic, Google, xAI, Mixlayer (SDK), or Vercel AI Gateway, an admin clicks Use Platform on the provider card, or Use Platform for organization under Manage connections. Then the admin clicks Use the plan key. Through the API, send the organization-policy request with "selection": "platform".

Understand BYOK costs

With BYOK, your provider bills you directly. Runtype doesn’t add a separate charge. Check your provider’s billing dashboard for exact costs.

A call on your own key doesn’t fall back to a platform key when the provider fails. On platform keys, a prompt step with no error handling of its own can recover from a transient provider failure, often by retrying on another provider. On your own key, that prompt step fails on the first transient provider error. Agents retry a failed call twice on either kind of key. To keep BYOK prompt steps running through provider incidents, set Use fallbacks on the step. For more information, see Automatic recovery on platform keys.

Remove a provider

Deleting a custom inference endpoint key or connection also removes every model that you added through it. Before you delete it, switch any Flows or Agents that use those models to another model.

To remove a provider key or connection, follow these steps:

On Settings > Models, open the Providers tab.
On the provider card, click Manage keys. In an organization, click Manage connections.
Next to the key, click Delete key. In an organization, click Delete connection.
Type the key or connection name to confirm.
Click Delete key. In an organization, click Delete connection.

In an organization, if you remove the connection that the organization uses for OpenAI, Anthropic, Google, xAI, Mixlayer (SDK), or Vercel AI Gateway, an admin first selects another organization connection. If Require your own model keys is disabled, the admin can instead select the platform key to keep related Flows and Agents running on platform keys.

Tinfoil, Amazon Bedrock, Vertex AI, Vertex AI (Claude), and custom inference endpoints have no platform key. Before you remove the connection that the organization uses for one of these, an admin selects another connection. Otherwise, calls to those models fail.

In a personal account, models from providers with platform keys move back to platform keys after you delete your key. If a model is available only through your key, choose a different model in your Flows and Agents before you delete the key.

Next steps

Continue with one of these guides:

Managing AI models: set a default model and manage enabled models.
What is the Playground?: test connected models and settings.
What are Flows?: use connected models in repeatable workflows.
What are Agents?: use connected models in agent workflows.
Model routing and data flow: see which provider serves each model and who can see your prompts.
Integrations overview: see each integration, where to connect it, and where you can use it.

Provider changes with BYOK-only enabled

When Require your own model keys is enabled, select another organization connection before removing the selected one. Selecting a platform key is refused. To return to platform credentials, an admin must first turn off that restriction in Settings > Models.