Records

Building retrieval that works

Use retrieval to give an AI model relevant context for each request. For retrieval-augmented generation (RAG), retrieve documents with Records, vector Flow steps, or the Semantic Search tool. Start with the smallest design that fits your corpus. Then measure retrieval and answer quality separately.

Right-size retrieval before adding vectors

Choose the least complex retrieval method for your data. Use the following options:

  • Use Records for structured lookups. Use Records when your queries map to metadata fields. Add a get-record or list-records step with a recordFilter. A get-record step returns one object. A list-records step returns an array.
  • Pass small context directly. If the relevant material fits in the prompt context, pass it to a Prompt step instead of adding an index.
  • Start with keyword or web search when it fits. Use a recordFilter for structured lookups or Exa for web search. Add vector search after an eval shows that semantic matching improves the results.

Add vector search when your corpus or query language prevents structured matching. Use it when your queries describe a concept instead of using the document’s exact words.

Vector similarity matches concepts, but it can miss exact identifiers such as part numbers, error codes, dates, proper nouns, acronyms, and other domain terms.

Use hybrid search when exact tokens matter. A Vector Search step supports hybridSearch and hybridAlpha with Weaviate. hybridAlpha controls the balance between keyword and vector search. The built-in pgvector store provides semantic search only. For exact-token queries against pgvector, run a semantic vector-search step and a list-records lookup with a recordFilter, then merge the results.

Filter metadata before ranking

Extract structured metadata when you ingest a document and store it with the vector. Include fields such as type, source, date, owner, or product area when your queries use them.

Apply filters inside the search so similarity ranking considers only the relevant records. With the built-in Record store, use recordType to scope searches. With Weaviate or Vectorize, use metadataFilters. For exact Records lookups, use recordFilter with get-record or list-records. Post-filtering a broad result set wastes the result limit on records that are out of scope.

Chunk documents by structure

Split documents into coherent chunks before you generate embeddings. Use these guidelines:

  • Aim for 200 to 300 tokens per chunk. Split at sentence boundaries.
  • Overlap adjacent chunks. Include a sentence or two from the preceding chunk when a fact can span the boundary.
  • Use the document’s structure. Split at headings, sections, or list items instead of using a fixed character count.
  • Normalize entities before grouping. Treat names such as “Acme Inc.” and “Acme” as the same entity before you aggregate results.

Build chunking in the ingest Flow. Use a Prompt or transform-data step to split the source. Then use a generate-embedding step to create embeddings and a store-vector step to write them to the selected vector store. Use the same embeddingModel for indexing and query embeddings.

Add reranking or query rewriting

Reranking and query rewriting are patterns built from existing steps, not dedicated steps. Add them only when an eval shows that precision or recall is the bottleneck:

  • Rerank the candidates. Retrieve a wider set. Then use a Prompt step with a lower-cost model to score the candidates against the query and keep the highest-scoring results.
  • Rewrite or decompose the query. For a multi-part or vague question, use a Prompt step before the search. Rewrite the query or split it into separate searches. Merge the results before you generate the answer.

Keep retrieved context minimal

More context can reduce answer quality. Remove distractors and keep only the records that answer the question:

  • Remove similar but incorrect chunks. Tighten limit and remove near-duplicate chunks so the model receives fewer confusing results.
  • Limit marginal context. A large context window does not make broad retrieval safe. Increase the retrieved set only when an eval shows that recall needs improvement.

Use the limit and threshold settings on the vector-search step to control the result set. Start with a narrow limit, then increase it when your evals show missing relevant results.

Separate retrieval and generation evals

Retrieval quality and generation quality measure different failure points. Build two Eval suites so you can isolate them:

  1. Measure retrieval quality. Create cases with expected chunks and check that the search returns them in the top results.
  2. Measure generation quality with fixed context. Give the model the correct chunks and check the answer without changing retrieval.

Bootstrap cases with synthetic query-to-chunk pairs. Generate a question for each chunk, then paraphrase the questions so they test meaning instead of the document’s exact wording. Include distractor cases where a similar but incorrect chunk must not be returned. Use a golden set of 50 to 90 judgments from your corpus to compare retrieval changes.

After retrieval feeds an Agent, measure end-task success and token use. Ranking position matters less than whether the Agent answers correctly with a reasonable number of tokens.

Keep the index fresh

For a changing corpus, use a Schedule to rerun the ingest Flow. Re-embed new and changed documents with the same model that you use for queries. Give each document or chunk a stable name so you can re-ingest one item without rebuilding the entire index.

Treat Graph RAG as a retrieval technique, not a requirement to run a graph database. Add specialized infrastructure only after evals show that a less specialized design no longer meets your retrieval requirements.

Next steps

Use these guides to apply the retrieval patterns: