> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.runtype.com/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.runtype.com/_mcp/server.

# Connecting AI model providers

Your Runtype account includes platform keys, so you can use popular AI models without adding provider accounts. Connect your own keys in **Settings** if you want dedicated rate limits, direct provider billing, or the provider's full model catalog.

For a full comparison, see [Platform Keys vs. Bring Your Own Key (BYOK)](/user-guide/getting-started/platform-keys-vs-byok).

## Before you begin

To connect your own keys, you need the following:

* A Team plan or higher. The Launch plan uses platform keys only, and adding a key returns an error. For plan details, see [Billing and plans](/user-guide/settings/billing-and-plans).
* An account with the AI provider that you want to connect, with permission to create API keys.
* In an organization, the organization admin role. Only admins can add, change, delete, or select the organization's connections. For roles, see [Team members and permissions](/user-guide/settings/team-members-and-permissions).

## Use platform keys

Your account includes platform keys for models from OpenAI, Anthropic, Google, xAI, Vercel AI Gateway, Cloudflare Workers AI, and open-source models through Mixlayer (SDK). You can build Flows and Agents without additional setup. For the models that each key covers, see [Available AI models](/user-guide/settings/available-ai-models).

## Supported providers

You can connect your own keys for these providers:

* **Direct providers**: OpenAI, Anthropic, Google, and xAI.
* **Inference providers**: Tinfoil, Amazon Bedrock, Vertex AI, Vertex AI (Claude), and Mixlayer (SDK).
* **Gateway providers**: Vercel AI Gateway.
* **Custom inference endpoints**: Any OpenAI-compatible (Chat Completions or Responses) endpoint, through **Custom inference endpoint**.

Cloudflare Workers AI models run on platform keys only. You don't add a key for them.

## Add a provider

To add a provider, start with a model or a provider connection. You can connect several providers and choose a model from any of them for each Flow step or Agent.

### Start from a model

To add a model and its key, follow these steps:

1. On **Settings > Models**, open the **Models** tab.
2. Click **Add model** and browse or search the catalog.
3. Select a model.
4. If the model needs a key that you haven't added, enter it when prompted.
5. In an organization, click **Use for the organization** to confirm. The new connection becomes the organization's connection for that provider.

### Start from a provider

To add a provider key first, follow these steps:

1. On **Settings > Models**, open the **Providers** tab.
2. On the provider card, click **Add your own key**. For Vertex AI, Vertex AI (Claude), Amazon Bedrock, and Custom inference endpoint, click **Configure**. In an organization, click **Add connection** on any card.
3. Enter the key details.
4. Click **Save API Key**. In an organization, click **Save connection**. For Vertex AI, Vertex AI (Claude), Amazon Bedrock, and Custom inference endpoint, click **Save Configuration**.
5. In an organization, select the connection for the organization. For the steps, see [Select a connection for the organization](#select-a-connection-for-the-organization).
6. Enable the models that you want to use.

### Select a connection for the organization

In an organization, a connection that you save on the **Providers** tab isn't used until an organization admin selects it. After an admin selects it, every teammate's runs, plus scheduled and API-triggered runs, use it. Saving or rotating a connection doesn't change which connection the organization uses.

To select a connection, follow these steps:

1. On **Settings > Models**, open the **Providers** tab.
2. On the provider card, click **Manage connections**.
3. Next to the connection, click **Use for the organization**.
4. Click **Use this connection**.

## Get provider credentials

Create credentials in each provider's dashboard, then add them in **Settings > Models** on the **Providers** tab.

### OpenAI

To add an OpenAI key, follow these steps:

1. Sign in to [OpenAI Platform](https://platform.openai.com).
2. On **API keys**, create a secret key.
3. Copy the key.
4. On the **OpenAI** card in Runtype, click **Add your own key** and paste the key.

### Anthropic

To add an Anthropic key, follow these steps:

1. Sign in to [Anthropic Console](https://console.anthropic.com).
2. On **API Keys**, create a key.
3. Copy the key.
4. On the **Anthropic** card in Runtype, click **Add your own key** and paste the key.

### Google

To add a Google key, follow these steps:

1. Sign in to [Google AI Studio](https://aistudio.google.com).
2. Create an API key and copy it.
3. On the **Google** card in Runtype, click **Add your own key** and paste the key.

### xAI

To add an xAI key, follow these steps:

1. Sign in to [xAI Console](https://console.x.ai).
2. Generate an API key and copy it.
3. On the **xAI** card in Runtype, click **Add your own key** and paste the key.

### Tinfoil

Tinfoil has no card on the **Providers** tab. You add the key when you add a Tinfoil model.

To add a Tinfoil key, follow these steps:

1. Sign in to [Tinfoil Platform](https://tinfoil.sh/).
2. Open **API Keys**.
3. Create a key and copy it.
4. On **Settings > Models**, open the **Models** tab and click **Add model**.
5. Select a Tinfoil model and paste the key when prompted.

### Mixlayer (SDK)

Mixlayer models use platform keys by default. To use a dedicated Mixlayer key, follow these steps:

1. Create an account on [Mixlayer](https://mixlayer.com).
2. On the [Mixlayer API keys page](https://console.mixlayer.com/app/api-keys), generate a key.
3. Copy the key.
4. On the **Mixlayer (SDK)** card in Runtype, click **Add your own key** and paste the key.

### Vercel AI Gateway

Vercel AI Gateway models use platform keys by default. To use your own Vercel account and limits, add your own key.

To add a Vercel AI Gateway key, follow these steps:

1. Open [Vercel AI Gateway](https://vercel.com/docs/ai-gateway).
2. Create an API key.
3. On the **Vercel AI Gateway** card in Runtype, click **Add your own key** and paste the key.

### Amazon Bedrock

To add Amazon Bedrock credentials, follow these steps:

1. Create an AWS access key with Bedrock permissions.
2. On the **Amazon Bedrock** card in Runtype, click **Configure**.
3. Enter the value in **AWS Access Key ID**.
4. Enter the value in **AWS Secret Access Key**.
5. Select a region in **AWS Region**.
6. Click **Save Configuration**.

### Vertex AI

Vertex AI works with your own credentials only. Use **Vertex AI** for Gemini models and **Vertex AI (Claude)** for Claude models. Each is a separate connection, and each connection can use its own service account, project, and region.

To add a Vertex AI connection, follow these steps:

1. Create a Google Cloud service account with the `roles/aiplatform.user` role.
2. On the **Vertex AI** card for Gemini models, or the **Vertex AI (Claude)** card for Claude models, click **Configure**.
3. Click **Upload JSON File** and choose the service account key file.
4. Enter the project ID in **GCP Project ID** and the region in **Region**.
5. Click **Save Configuration**.

### Custom inference endpoints

A custom inference endpoint is any server that speaks the OpenAI Chat Completions or Responses API. This includes Groq, Fireworks, Perplexity, Ollama, vLLM, LiteLLM, and hosted gateways such as NVIDIA (`https://integrate.api.nvidia.com/v1`). Custom inference endpoints work with your own credentials only. They have no platform key. In the API, MCP, and SDK, the provider ID is `generic-openai`.

#### Supported API format

Each custom inference endpoint connection speaks one API format. Choose it in **API format** in the dialog, or set it in the connection's `apiFormat` setting:

| `apiFormat`                  | Request path                       |
| ---------------------------- | ---------------------------------- |
| `chat-completions` (default) | `POST {base URL}/chat/completions` |
| `responses`                  | `POST {base URL}/responses`        |

Model discovery uses `GET {base URL}/models` for both formats. The Anthropic Messages format (`/messages`) isn't supported on this connector.

To use the Responses format, choose **Responses** in **API format**. Through the API, set `settings.apiFormat` to `responses` when you create or update the connection (`POST /v1/provider-keys`), or pass `api_format: "responses"` to the `create_provider_key` or `update_provider_key` MCP tool. On the Responses format, Runtype sends `store: false` on every request and never references a stored response, so the endpoint doesn't need to keep any state between requests.

The following table answers the common questions about what Runtype sends:

| Question                                         | Answer                                                                                                                                                                                                                    |
| ------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Can I use the Responses API?                     | Yes. Choose **Responses** in **API format**, or set the connection's `apiFormat` to `responses`.                                                                                                                          |
| Can I set a reasoning effort, such as `medium`?  | Yes, after you declare the model as a reasoning model. Runtype sends `reasoning_effort` on Chat Completions and `reasoning.effort` on Responses. See [Reasoning models](#reasoning-models-on-custom-inference-endpoints). |
| Does Runtype send `max_tokens` on every request? | No. Runtype sends `max_tokens` only when the agent or step sets **Max output tokens**. The model's own **Max Output Tokens** setting is the highest limit that editors offer.                                             |
| Can I turn off streaming?                        | No. Runtype always requests a streamed response (`stream: true`), so the endpoint must support streaming. **Supports streaming** is saved model information only.                                                         |

The dialog shows the full request URL and the request body shape for the chosen format under **Base URL**, so you can check them before you save.

The base URL must meet these requirements:

* It uses HTTPS.
* It points to a publicly reachable hostname. Runtype rejects `localhost`, private IP addresses, and cloud metadata hosts. To use a server that runs on your own machine or private network, such as Ollama or vLLM, expose it through a public HTTPS tunnel or proxy first.
* It ends at the API root. Runtype adds `/v1` when the URL doesn't already end with it, so enter the URL up to and including `/v1`. For example, `https://api.groq.com/openai/v1`.

To add a custom inference endpoint, follow these steps:

1. Get the endpoint base URL and API key from your provider.
2. On the **Custom inference endpoint** card in Runtype, click **Configure**.
3. Enter the base URL and API key.
4. In **API format**, keep **Chat Completions** or choose **Responses**.
5. Add the models that you want to enable with one of these methods:
   * **Auto-discover**: Click **Discover Models**. Runtype requests the endpoint's `/v1/models` list and shows the returned models.
   * **Add manually**: If the endpoint doesn't support `/v1/models`, enter each model ID.
6. Optional: Expand a model's **Advanced Settings** to change its defaults.
7. Click **Save Configuration**.

On plans without BYOK, the dashboard locks **Custom inference endpoint** and prompts you to add a payment method.

Keep your API keys private. Don't share them publicly or commit them to version control.

#### Model settings

Each model that you add has the following settings under **Advanced Settings**:

| Setting                     | Default    | When to change it                                                                                                             |
| --------------------------- | ---------- | ----------------------------------------------------------------------------------------------------------------------------- |
| **Max Output Tokens**       | 32768      | Lower it to the endpoint's real output limit. Editors offer values up to this limit. Runtype doesn't send it on every request |
| **Response Formats**        | text, JSON | Remove JSON if the endpoint doesn't support JSON output                                                                       |
| **Input Cost / 1k Tokens**  | 0          | Enter your provider's price to see accurate costs in usage reports                                                            |
| **Output Cost / 1k Tokens** | 0          | Enter your provider's price to see accurate costs in usage reports                                                            |
| **Supports streaming**      | Selected   | Saved as model information only. Runtype always streams, so clearing it doesn't turn streaming off                            |
| **Reasoning model**         | Off        | Turn it on if the endpoint accepts a reasoning effort for the model                                                           |

With the default costs of 0, usage and cost reports show \$0 for these models.

#### Troubleshoot model discovery

Discovery waits 10 seconds for a response and doesn't follow redirects. The endpoint must return an OpenAI-style list: a JSON object with a `data` array of models, each with an `id`.

The following errors are the most common:

* **Only HTTPS URLs are supported**, **Localhost URLs are not allowed**, or **Private/internal IP addresses are not allowed**: The base URL doesn't meet the base URL requirements. Use a public HTTPS URL.
* **Models endpoint returned 401** or **403**: The provider rejected the API key. Check the key and its permissions.
* **Models endpoint returned 404**: The base URL path is wrong. Enter the URL up to and including `/v1`.
* **redirects are not followed**: The endpoint redirects to another address. Enter the final address as the base URL.
* **Invalid JSON response from models endpoint** or **Response missing expected "data" array**: The endpoint doesn't return an OpenAI-style model list. Add the model IDs manually.
* An aborted-request error after about 10 seconds: The endpoint didn't answer in time. Try again, or add the model IDs manually.

#### Reasoning models on custom inference endpoints

Runtype has no catalog entry for a model served from a custom endpoint, so it can't tell whether the endpoint accepts `reasoning_effort` for that model. To declare a reasoning model, follow these steps:

1. On the **Custom inference endpoint** card, click **Manage keys**. In an organization, click **Manage connections**.
2. Next to the key, click **Edit key**.
3. Expand the model's **Advanced Settings**.
4. Turn on **Reasoning model**.
5. Choose a **Default reasoning effort**. Runtype sends this value when an agent or flow step sets no reasoning of its own. **Provider default (send nothing)** sends no value.
6. Click **Update Configuration**.

Through the API, set `reasoning` on the model entry when you sync models, or `settings.customModel.reasoning` on the model configuration: `{ "supported": true, "defaultEffort": "none" }`. The `sync_provider_models` and `create_model_config` MCP tools accept the same declaration as `reasoning_supported` and `reasoning_default_effort`. A default effort without `reasoning_supported: true` is rejected.

The declaration is the only opt-in. Runtype never sends `reasoning_effort` to a model that isn't declared, or that's declared with `supported: false`, even when an agent or step sets a reasoning effort. On a declared model, Runtype chooses the value as follows:

* `reasoning: false` on the agent or step sends `reasoning_effort: "none"`.
* A reasoning effort set on the agent or step (`reasoning.reasoningEffort`) is sent as given.
* Otherwise, Runtype sends the model's default effort, if you set one.

Some providers treat an omitted `reasoning_effort` as unbounded. Cerebras, for example, lets a Qwen model spend its whole output budget on reasoning and return no text. For such an endpoint, declare the model and set the default effort to `none` or `low`. The model's reasoning streams as reasoning blocks either way, including into a Persona widget that shows reasoning.

On a connection that uses the Responses format, Runtype sends the same value as `reasoning.effort`, and asks for a reasoning summary (`reasoning.summary: "auto"`) unless the effort is `none`. A model that isn't declared gets no `reasoning` field on either format.

#### System message position on custom inference endpoints

A conversation can carry a `system` message after the first user turn. For example, a caller can send per-turn context last so that the leading system prompt stays identical across turns and the provider's prompt cache keeps working. On a custom inference endpoint, Runtype keeps that message at its position by default, the same as on OpenAI and xAI.

Some chat templates reject a `system` message that isn't first. Runtype automatically moves such messages into the leading system prompt for Mistral- and Gemma-family model IDs (`mistral`, `mixtral`, `ministral`, `magistral`, `codestral`, and `gemma`). For any other model on an endpoint that rejects them, declare the opt-out on the model:

* Through the API, set `midConversationSystemMessages: "hoist"` on the model entry when you sync models, or `settings.customModel.midConversationSystemMessages` on the model configuration.
* Through MCP, pass `mid_conversation_system_messages: "hoist"` to `sync_provider_models` or `create_model_config`. The field applies only to `generic-openai` models. Any other provider rejects it.

The dashboard has no toggle for this setting. `"preserve"` is accepted as an explicit value and matches the default.

## Enable models

To choose the models that appear in model pickers, follow these steps:

1. On **Settings > Models**, open the **Models** tab.
2. Click **Add model**.
3. Select the models that you want to use.
4. Confirm that the enabled models appear in model pickers across Runtype, including Flows, Agents, and the Playground.

Enable only the models that you use. This keeps model pickers shorter and more focused.

## Connect a custom inference endpoint through the API, MCP, or SDK

You can connect a custom inference endpoint without the dashboard. Use the Runtype API, the Runtype MCP server, or the Runtype SDK. To add a key for another provider, such as OpenAI or Anthropic, send the same create request with that provider's ID in `provider`.

Creating a provider key, discovering or syncing models, and creating a model configuration require BYOK on your plan. Without it, these calls return `402` with code `BYOK_NOT_AVAILABLE`. In an organization, creating, changing, syncing, deleting, and selecting connections require the organization admin role. Use a Runtype API key with the `INTEGRATIONS:WRITE` scope for provider keys and the `MODELS:WRITE` scope for model configurations. For more information, see [Managing API keys](/user-guide/settings/managing-api-keys).

The setup has three steps. Create a `generic-openai` provider key with the endpoint's base URL. Discover its models or supply a manual list. Sync the models that you want to enable. In an organization, the create request returns a connection ID, and an organization admin selects that connection in a separate call.

> **Warning**
>
> A sync request sets the full model list for the connection. Runtype removes any previously synced model that you leave out, and an empty `models` array removes every model. Flows and Agents that use a removed model stop working. Include every model that you want to keep in each sync.

The following TypeScript SDK sample creates the key, discovers models, and syncs the model list:

**`TypeScript SDK`**

```typescript title="TypeScript SDK"
import { RuntypeClient } from '@runtypelabs/sdk'

const client = new RuntypeClient({
  apiKey: process.env.YOUR_RUNTYPE_API_KEY!,
})

const key = await client.providerKeys.create({
  provider: 'generic-openai',
  name: 'customer-support-endpoint',
  apiKey: process.env.YOUR_PROVIDER_API_KEY!,
  settings: {
    baseUrl: process.env.YOUR_OPENAI_COMPATIBLE_BASE_URL!,
  },
})

const { models } = await client.providerKeys.discoverModelsForKey(key.id)
await client.providerKeys.syncModels(
  key.id,
  models.map((model) => ({ id: model.id }))
)
```

In an organization, an admin can then select the connection for the organization. Call `selectOrganizationProviderCredential` with `provider: 'generic-openai'` and `selection: { connectionId: key.id }`.

Set these environment variables before you run the sample:

* `YOUR_RUNTYPE_API_KEY`: Your Runtype API key.
* `YOUR_PROVIDER_API_KEY`: Your custom inference endpoint's API key.
* `YOUR_OPENAI_COMPATIBLE_BASE_URL`: The provider endpoint base URL.

The following cURL commands create the key, discover models, and sync the model list. In an organization, the final command selects the connection for the organization:

**`cURL`**

```bash title="cURL"
# 1. Create the provider key
curl -X POST https://api.runtype.com/v1/provider-keys \
  -H "Authorization: Bearer YOUR_RUNTYPE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "generic-openai",
    "name": "customer-support-endpoint",
    "apiKey": "YOUR_PROVIDER_API_KEY",
    "settings": {
      "baseUrl": "YOUR_OPENAI_COMPATIBLE_BASE_URL"
    }
  }'

# The first request returns the credential ID used by the following commands.
CREDENTIAL_ID=YOUR_CREDENTIAL_ID

# 2. Discover models from the endpoint's /v1/models
curl -X POST \
  "https://api.runtype.com/v1/provider-keys/${CREDENTIAL_ID}/discover-models" \
  -H "Authorization: Bearer YOUR_RUNTYPE_API_KEY"

# 3. Sync the full list of models that you want to enable
curl -X POST \
  "https://api.runtype.com/v1/provider-keys/${CREDENTIAL_ID}/sync-models" \
  -H "Authorization: Bearer YOUR_RUNTYPE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "models": [
      {"id": "YOUR_MODEL_ID"}
    ]
  }'

# 4. Organization only: select the connection for the organization
curl -X PUT https://api.runtype.com/v1/provider-keys/organization-policy \
  -H "Authorization: Bearer YOUR_RUNTYPE_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"provider\": \"generic-openai\",
    \"selection\": {
      \"connectionId\": \"${CREDENTIAL_ID}\"
    }
  }"
```

Replace the placeholders in the cURL commands with these values:

* `YOUR_RUNTYPE_API_KEY`: Your Runtype API key.
* `YOUR_PROVIDER_API_KEY`: Your custom inference endpoint's API key.
* `YOUR_OPENAI_COMPATIBLE_BASE_URL`: The provider endpoint base URL.
* `YOUR_CREDENTIAL_ID`: The `providerKey.id` value returned by the first request. Assign it to `CREDENTIAL_ID` in the cURL commands.
* `YOUR_MODEL_ID`: A model ID that the endpoint serves.

Through MCP, call the `create_provider_key`, `discover_provider_models`, and `sync_provider_models` tools for the same sequence. Then call `select_organization_provider_credential` to select the connection for the organization. To register one custom model without discovery, call `create_model_config` with `provider: "generic-openai"`, `model_id`, and `base_url`.

### Use a synced model

Synced models appear in model pickers across Runtype. Through the API, the SDK, or MCP, reference a synced model as `generic-openai:` followed by the model ID that the endpoint serves. For example, set `model` on an agent or prompt step to `generic-openai:llama-3.3-70b`.

## Rotate or change a key

To replace a key with a new one, follow these steps:

1. On **Settings > Models**, open the **Providers** tab.
2. On the provider card, click **Manage keys**. In an organization, click **Manage connections**.
3. Next to the key, click **Edit key**.
4. Enter the new key.
5. Click **Update Key**. In an organization, click **Update connection**. For Vertex AI, Vertex AI (Claude), Amazon Bedrock, and Custom inference endpoint, click **Update Configuration**.

Rotating an organization connection doesn't change which connection the organization uses. Through the API, send a `PATCH` request to `/v1/provider-keys/YOUR_CREDENTIAL_ID` with the new `apiKey`.

If you have several personal keys for one provider, click **Set as default** next to the key that you want Runtype to use.

With **Require your own model keys** disabled, to move an organization back to the platform key for OpenAI, Anthropic, Google, xAI, Mixlayer (SDK), or Vercel AI Gateway, an admin clicks **Use Platform** on the provider card, or **Use Platform for organization** under **Manage connections**. Then the admin clicks **Use the plan key**. Through the API, send the organization-policy request with `"selection": "platform"`.

## Understand BYOK costs

With BYOK, your provider bills you directly. Runtype doesn't add a separate charge. Check your provider's billing dashboard for exact costs.

A call on your own key doesn't fall back to a platform key when the provider fails. On platform keys, a prompt step with no error handling of its own can recover from a transient provider failure, often by retrying on another provider. On your own key, that prompt step fails on the first transient provider error. Agents retry a failed call twice on either kind of key. To keep BYOK prompt steps running through provider incidents, set **Use fallbacks** on the step. For more information, see [Automatic recovery on platform keys](/user-guide/settings/model-routing-and-data-flow#automatic-recovery-on-platform-keys).

## Remove a provider

Deleting a custom inference endpoint key or connection also removes every model that you added through it. Before you delete it, switch any Flows or Agents that use those models to another model.

To remove a provider key or connection, follow these steps:

1. On **Settings > Models**, open the **Providers** tab.
2. On the provider card, click **Manage keys**. In an organization, click **Manage connections**.
3. Next to the key, click **Delete key**. In an organization, click **Delete connection**.
4. Type the key or connection name to confirm.
5. Click **Delete key**. In an organization, click **Delete connection**.

In an organization, if you remove the connection that the organization uses for OpenAI, Anthropic, Google, xAI, Mixlayer (SDK), or Vercel AI Gateway, an admin first selects another organization connection. If **Require your own model keys** is disabled, the admin can instead select the platform key to keep related Flows and Agents running on platform keys.

Tinfoil, Amazon Bedrock, Vertex AI, Vertex AI (Claude), and custom inference endpoints have no platform key. Before you remove the connection that the organization uses for one of these, an admin selects another connection. Otherwise, calls to those models fail.

In a personal account, models from providers with platform keys move back to platform keys after you delete your key. If a model is available only through your key, choose a different model in your Flows and Agents before you delete the key.

## Next steps

Continue with one of these guides:

* [Managing AI models](/user-guide/settings/managing-ai-models): set a default model and manage enabled models.
* [What is the Playground?](/user-guide/playground/what-is-the-playground): test connected models and settings.
* [What are Flows?](/user-guide/flows/what-are-flows): use connected models in repeatable workflows.
* [What are Agents?](/user-guide/agents/what-are-agents): use connected models in agent workflows.
* [Model routing and data flow](/user-guide/settings/model-routing-and-data-flow): see which provider serves each model and who can see your prompts.
* [Integrations overview](/user-guide/integrations/integrations-overview): see each integration, where to connect it, and where you can use it.

### Provider changes with BYOK-only enabled

When **Require your own model keys** is enabled, select another organization connection
before removing the selected one. Selecting a platform key is refused. To return to
platform credentials, an admin must first turn off that restriction in **Settings > Models**.