> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.runtype.com/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.runtype.com/_mcp/server.

# Building retrieval that works

Use retrieval to give an AI model relevant context for each request. For retrieval-augmented generation (RAG), retrieve documents with Records, vector Flow steps, or the **Semantic Search** tool. Start with the smallest design that fits your corpus. Add specialized infrastructure only after evals show that a less complex design no longer meets your needs. Measure retrieval quality and answer quality separately.

This guide calls the vector store that Runtype provides with Records the built-in Records vector store. The **Vector Search** step lists it as **PostgreSQL (pgvector)**, and the **Semantic Search** tool lists it as **Platform Records (pgvector)**.

## Right-size retrieval before adding vectors

Choose the least complex retrieval method for your data. Use the following options:

* **Use Records for structured lookups.** Use Records when your queries map to metadata fields. Add a **Get Record** or **List Records** step with a `recordFilter`. A **Get Record** step returns one object. A **List Records** step returns an array.
* **Pass small context directly.** If the relevant material fits in the prompt context, pass it to a **Run Task** step instead of adding an index.
* **Use web search for public content.** Add the **Exa** tool to an Agent, or use Exa as the provider for an **AI Search** step. For details, see [Exa (web search)](/user-guide/integrations/exa-web-search).

Add vector search when your corpus or query language prevents structured matching. Use it when your queries describe a concept instead of using the document's exact words.

## Choose a step or a tool

You can run vector search in two ways:

* **Use a Vector Search step** when the Flow must search on every run. You set the query, the embedding model, and the result settings.
* **Use the Semantic Search tool** when an Agent or model decides when to search. You set a display name, the **Provider**, and a similarity threshold (default `0.3`). With **Platform Records (pgvector)**, you also select a record type. The model sets `limit` from 1 to 20 (default 5) on each call.

The **Semantic Search** tool always embeds queries with `text-embedding-3-small`, and it has no embedding model setting. Embed any index that this tool searches with `text-embedding-3-small`. For setup, see [Semantic Search](/user-guide/tools/built-in-tools#semantic-search).

## Combine vector and keyword search

Vector similarity matches concepts, but it can miss exact identifiers such as part numbers, error codes, dates, proper nouns, acronyms, and other domain terms.

The built-in Records vector store and Cloudflare Vectorize provide semantic search only. For exact-token queries against Records, run a **Vector Search** step and a **List Records** step with a `recordFilter` that uses the `contains` operator. Then merge the two result sets in a **Run Code** step.

The **Vector Search** step shows hybrid search settings when you select **Weaviate**. However, a Flow **Vector Search** step that uses Weaviate fails when the Flow runs. To search Weaviate, use the **Semantic Search** tool in an Agent. The tool does not support hybrid search.

## Filter metadata before ranking

Extract structured metadata when you ingest a document and store it with the vector. Include fields such as type, source, date, owner, or product area when your queries use them.

Apply filters inside the search so similarity ranking considers only the relevant results. Post-filtering a broad result set wastes the result limit on results that are out of scope. Each vector store supports different filters:

* **Built-in Records vector store.** Use `recordType` to scope searches. This store ignores `metadataFilters`.
* **Cloudflare Vectorize.** Use `metadataFilters` for exact-match filters, or scope searches by namespace.

For exact Records lookups, use `recordFilter` with a **Get Record** or **List Records** step. For more filter details, see [Filtering and searching records](/user-guide/records/filtering-and-searching-records).

## Chunk documents by structure

Split documents into coherent chunks before you generate embeddings. Use these guidelines:

* **Aim for 200 to 300 tokens per chunk.** Split at sentence boundaries.
* **Overlap adjacent chunks.** Include a sentence or two from the preceding chunk when a fact can span the boundary.
* **Use the document's structure.** Split at headings, sections, or list items instead of using a fixed character count.
* **Normalize entities before grouping.** Treat names such as "Acme Inc." and "Acme" as the same entity before you aggregate results.

## Build the ingest Flow

With the built-in Records vector store, a **Store Vector** step adds an embedding to a Record that already exists. It does not create Records. If a Record does not exist, the step reports `Record not found` for that item. If the ID template is empty, every item fails with `ID template is required for pgvector destination`, and the step fails.

To index chunks into the built-in Records vector store, add these steps in order:

1. **Split the source.** Use a **Run Task** or **Run Code** step to split each document into an array of chunks.
2. **Create one Record per chunk.** Add an **Upsert Record** step in batch mode. Store the chunk text and its metadata in each Record, and set a `recordType`, for example `kb_chunk`. The step returns the Records in input order, each with its `id`.
3. **Pair each chunk with its Record ID.** Use a **Run Code** step to build an array where each item has the Record `id` and the chunk `text`.
4. **Generate the embeddings.** Add a **Generate Embedding** step with `inputMode: "batch"`. Set `inputVariable` to the paired array, `itemAlias` to `item`, and `textTemplate` to `{{item.text}}`. Each output item keeps the fields of its input item.
5. **Store the embeddings.** Add a **Store Vector** step with `destination: "pgvector"` and `inputMode: "batch"`. Set **Store vectors from** (`vectorsSource`) to the **Generate Embedding** output variable, and set **with ID template** (`idTemplate`) to `{{item.id}}`. The step fails if **Store vectors from** is empty.

The `maxLength` setting on **Generate Embedding** truncates long input. It does not split input into chunks. Use the same embedding model for indexing and for queries.

## Query the index

The following **Vector Search** step config searches the chunks from the ingest Flow:

**`Vector Search step config`**

```json title="Vector Search step config"
{
  "query": "{{userMessage}}",
  "recordType": "kb_chunk",
  "embeddingModel": "text-embedding-3-small",
  "limit": 5,
  "outputVariable": "hits"
}
```

With the built-in Records vector store, each result has the following shape:

**`One result`**

```json title="One result"
{
  "id": "record_01jd8x2k4f7m9q3v5w6y8z0a1b",
  "score": 0.82,
  "similarity": 0.82,
  "name": "refund-policy-chunk-3",
  "type": "kb_chunk",
  "metadata": {
    "text": "Customers can request a refund within 30 days of purchase.",
    "source": "refund-policy.md"
  }
}
```

The result has no separate text field. Store the chunk text in the Record metadata at ingest, so that the model receives content and not only IDs and scores. To pass the results to the model, reference `{{hits}}` in the prompt of the **Run Task** step that generates the answer.

## Add reranking or query rewriting

Reranking and query rewriting are patterns built from existing steps, not dedicated steps. Add them only when an eval shows that precision or recall is the bottleneck:

* **Rerank the candidates.** Retrieve a wider set. Then use a **Run Task** step with a lower-cost model to score the candidates against the query and keep the highest-scoring results.
* **Rewrite or decompose the query.** For a multi-part or vague question, use a **Run Task** step before the search. Rewrite the query or split it into separate searches. Merge the results before you generate the answer.

## Keep retrieved context minimal

More context can reduce answer quality. Remove distractors and keep only the results that answer the question:

* **Remove similar but incorrect chunks.** Tighten `limit` and remove near-duplicate chunks so the model receives fewer confusing results.
* **Limit marginal context.** A large context window does not make broad retrieval safe. Increase the retrieved set only when an eval shows that recall needs improvement.

Use the `limit` and `threshold` settings on the **Vector Search** step to control the result set. The step first selects the top `limit` results (default 5). Then it removes results with a similarity lower than `threshold`, so a high `threshold` can return fewer results than `limit`. Start with a narrow `limit`, then increase it when your evals show missing relevant results.

## If results are empty or sparse

A search that returns no results usually reports no error. Check the following:

* **The embedding model matches.** With the built-in Records vector store, a search matches only Records embedded with the same model as the query. If you do not set `embeddingModel`, the step uses `text-embedding-3-small`.
* **The record type matches.** Set `recordType` to the same type that the ingest Flow used.
* **The threshold is not too high.** Lower `threshold`, or remove it, and run the search again.
* **The Records have embeddings.** Check the **Store Vector** step output from the ingest run for items that failed.

## Separate retrieval and generation evals

Retrieval quality and generation quality measure different failure points. Build two [Eval](/user-guide/evals/what-are-evals) suites so you can isolate them:

1. **Measure retrieval quality.** Create cases with expected chunks and check that the search returns them in the top results.
2. **Measure generation quality with fixed context.** Give the model the correct chunks and check the answer without changing retrieval.

Bootstrap cases with synthetic query-to-chunk pairs. Generate a question for each chunk, then paraphrase the questions so they test meaning instead of the document's exact wording. Include **distractor cases** where a similar but incorrect chunk must not be returned. Use a fixed golden set of about 50 labeled query-to-chunk pairs from your corpus to compare retrieval changes.

After retrieval feeds an **Agent**, measure end-task success and token use. Ranking position matters less than whether the Agent answers correctly with a reasonable number of tokens.

## Keep the index fresh

For a changing corpus, use a Schedule to rerun the ingest Flow. Re-embed new and changed documents with the same model that you use for queries. Give each chunk a stable Record ID, and set the **Store Vector** ID template to that ID. You can then re-ingest one item without rebuilding the entire index.

## Next steps

Use these guides to apply the retrieval patterns:

* [What are Records?](/user-guide/records/what-are-records): understand the structured store behind filter-based retrieval.
* [Filtering and searching records](/user-guide/records/filtering-and-searching-records): configure `recordFilter` and semantic search.
* [Using record steps (get/list/upsert)](/user-guide/flows/using-record-steps-upsert-retrieve): create the Records that the ingest Flow embeds.
* [Built-in tools](/user-guide/tools/built-in-tools#semantic-search): configure the **Semantic Search** tool.
* [Exa (web search)](/user-guide/integrations/exa-web-search): search public web content.
* [Weaviate (vector search)](/user-guide/integrations/weaviate-vector-search): connect a Weaviate instance.
* [Designing evals that work](/user-guide/evals/designing-evals-that-work): build and evaluate retrieval test cases.
* [What are Schedules?](/user-guide/schedules/what-are-schedules): rerun an ingest Flow on a cadence.
* [Choosing between a flow and an agent](/user-guide/flows/choosing-between-a-flow-and-an-agent): choose what consumes the retrieved context.