> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.runtype.com/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.runtype.com/_mcp/server.

# Prompt caching

Prompt caching reuses a stable prefix of your prompt across requests. Runtype manages provider-specific cache settings, so you do not configure cache keys or breakpoints. A cache read can reduce input cost and time to first token.

## How prompt caching works

Your prompt often starts with stable content, such as the system prompt, tool definitions, and previous conversation turns. A provider that supports prompt caching stores that prefix after the first request and reuses it on later requests.

Runtype handles each provider's caching requirements for you. The provider reports cache reads and cache writes with each response.

Prompt caching does not store model responses. The model runs on every request and returns a fresh response. Caching reduces only the cost and latency of processing the input.

Provider caches have their own retention rules. A cache opt-out does not delete earlier
cache entries or saved application data. For the relationship between caching, inference
ZDR, and storage, see [Data retention](/user-guide/settings/data-retention#model-providers-and-connected-services).

Runtype recognizes prompt-cache usage for the Anthropic, OpenAI, and DeepSeek model families, including Anthropic and OpenAI calls made with platform keys. Usage estimates for these families apply the provider's discounted cached-input rate. For other providers, usage estimates price cached input at the regular input rate when Runtype has no verified cached-input rate for the model.

## When caching helps

Prompt caching helps when you send the same prefix more than once. Runtype turns on provider caching automatically for these request patterns:

* **Agent loops:** Your Agent resends its system prompt, tools, and growing conversation history on each turn. Later turns can read the earlier prefix from cache.
* **Conversations with history:** When you send previous messages, each request reuses the conversation prefix that remains unchanged.
* **Batch and Eval fan-outs:** When you run a batch or Eval over one Flow or Agent version, the records share the same system prompt and tool definitions. Later records can read that shared prefix from cache.
* **Repeated single-shot requests:** When you repeat a request with a stable system prompt or tools, the provider can reuse that prefix. Runtype does not request caching when the request has no stable system prompt or tools.

A one-time request has no cross-request cache read. For providers that charge a cache-write premium, Runtype does not request caching when a request has no reusable prefix. Providers with automatic caching can still reuse a repeated, byte-identical prefix.

### Batch warm-up

For a batch with more than one record, Runtype runs one record first to write the shared prefix to the cache, and then runs the remaining records in parallel. This adds the latency of one record before the batch fans out.

To start all records at once, set `skipCacheWarm` to `true` in the batch `options` object. Use this option when records share no stable prefix or when throughput matters more than cache reads. The same option is available in the TypeScript SDK `BatchBuilder` and in the execution options of a schedule. For more information, see [Running flows in batch](/user-guide/flows/running-flows-in-batch).

## Write cache-friendly prompts

To get cache reads, keep the reusable prefix byte-for-byte identical across requests. A value that changes on every run changes the prefix that follows it and prevents later requests from reading that prefix from cache.

Follow these guidelines when you write prompts:

* Put large, stable context, such as style guides, schemas, and examples, in the system prompt. Vary only the user prompt.
* Give steps that perform the same task an identical system prompt, so they share one cache entry.
* Move per-run values into the user prompt or a later step. Do not place any `_now`, `_execution`, or `_schedule` variable, such as `{{_now}}` or `{{_execution.id}}`, in the system prompt. If any of these variables appears anywhere in the system prompt, the provider cannot reuse the system prompt from cache.
* Expect other variables in the system prompt, such as `{{_user.id}}` or a previous step's output, to create a separate cache entry for each distinct value. They do not stop caching.

For the full list of system variables, see [Flow variables and templates](/user-guide/flows/flow-variables-and-templates).

When you save a Flow, Runtype reports a non-blocking `CACHE_VOLATILE_SYSTEM_PROMPT` recommendation for per-run variables in a system prompt. The recommendation does not block saving or change Flow behavior. Runtype checks top-level prompt steps only. Prompt steps inside a conditional step are not checked. For more information, see [Flow validation warnings](/user-guide/flows/flow-validation-warnings).

## Turn off prompt caching

Prompt caching needs no configuration. To turn it off for one request, set `cache` to `false`. Use this opt-out when requests must not share provider cache state or when you compare cached and uncached behavior.

For a `POST /v1/dispatch` request, set `cache` in the `options` object as shown in this JSON fragment:

```json
{
  "options": {
    "cache": false
  }
}
```

For a `POST /v1/agents/{id}/execute` request, set the top-level `cache` field as shown in this JSON fragment:

```json
{
  "cache": false
}
```

The TypeScript SDK accepts the same field: pass `cache: false` to the `agents.execute` method, or include it in the options you pass to the `withOptions` method of a `FlowBuilder`. The Runtype MCP server `dispatch` and `execute_agent` tools accept a `cache` parameter.

The opt-out applies per request. Batch, Eval, and schedule runs have no `cache` option. For these runs, use `skipCacheWarm` to skip the warm-up record.

## View cache activity

To inspect one run, open **Logs**, select the run, and select a prompt step. The step details show cache read and cache write token counts when the provider reports them. A nonzero cache read count confirms that the provider served part of the input from cache.

To review aggregate activity, open **Settings > Usage**. The **Estimated cache savings** tile shows the estimated savings and the cache hit rate. The **Prompt cache** card shows cache read and cache write token totals. The savings estimate is net of any cache-write premium the provider charges. Both appear only when the selected period contains cache activity. In the **Cost by AI model** table, a percent-cached badge appears next to each model that served input from cache. For more information, see [Usage data](/user-guide/settings/usage-data).

To check caching from code, call [`GET /v1/runs/{executionId}`](/api-reference/runtype-api/runs/read-one-run) and read the `cacheReadTokens` and `cacheWriteTokens` fields. Both are subsets of `inputTokens`.

## Troubleshoot missing cache reads

If the cache read count stays at zero, check the following causes:

* **The stable prefix is short.** Providers cache only prefixes above a minimum length. Runtype does not request caching for a prefix it estimates is under that minimum, so a short system prompt with few tools often does not cache.
* **Requests are too far apart.** A cache entry typically expires about 5 minutes after its last use. Requests with the same prefix that arrive further apart do not read from cache.
* **The system prompt changes on every run.** Move per-run variables out of the system prompt, as described in [Write cache-friendly prompts](#write-cache-friendly-prompts).
* **The request turned caching off.** Check whether the request sets `cache` to `false`.
* **The provider is not supported.** Runtype recognizes cache usage for the Anthropic, OpenAI, and DeepSeek model families.
* **Runtype paused caching for the workload.** For more information, see [Why caching stops for a workload](#why-caching-stops-for-a-workload).

### Why caching stops for a workload

Runtype tracks cache reads for each Flow version and each Agent. If at least 20 requests in a 24-hour window request caching and fewer than 5% of them produce a cache read, Runtype stops requesting caching for that workload for 24 hours. This avoids repeated cache-write premiums for a prefix that is rarely reused. Providers with automatic caching can still reuse an identical prefix during the pause.

After you fix the prompt, caching restarts as follows:

* **Agent:** Caching restarts when the 24-hour pause ends. Publishing a new Agent version does not reset the pause.
* **Flow:** Publish a new Flow version to restart caching immediately. Otherwise, caching restarts when the 24-hour pause ends.

## Next steps

Use these guides to continue:

* [Available AI models](/user-guide/settings/available-ai-models): choose a model that fits your workload.
* [Working with logs](/user-guide/logs/working-with-logs): inspect cache token counts for a run.
* [Running an eval](/user-guide/evals/running-an-eval): run an Eval whose cases share a cached prefix.
* [Automating batch processing](/user-guide/schedules/automating-batch-processing): run batches on a schedule.
* [Conversation history in an embedded widget](/developer-guides/guides/client-conversation-history): send previous messages so later requests reuse the conversation prefix.