Using prompt steps

Prompt steps call AI models to generate text, analyze data, or answer questions. They’re the core building block of AI workflows.

Add a prompt step

1. In the Flow editor, click Add Step
2. Select Run Task for a single-shot prompt, or Run Agent for a conversational step that keeps message history
3. Configure:
   • Name: Descriptive label (e.g., “Generate summary”)
   • Model: Select an AI model
   • Prompt: Write your prompt text
4. Click Save Step

Writing prompts

Use natural language to instruct the AI. Be specific about what you want:

Weak:

Summarize this.

Better:

Summarize the following customer email in 2-3 sentences.
Focus on the main request and any urgency indicators.
Email: {{email_text}}

Using template variables

Reference Flow inputs and previous step outputs with {{variable}} syntax:

You are a customer support Agent for {{company}}.
Customer message: {{customer_message}}
Customer tier: {{customer_data.tier}}
Previous interactions: {{customer_data.interaction_count}}
Provide a helpful, professional response.

Variable names come from the inputs field when triggering the Flow or from the outputVariable field of previous steps.

Model selection

Different models have different strengths. Use the routed model format (e.g., claude-sonnet-4-6, gpt-5.4-mini, gemini-3-1-flash-lite) so your Flows stay current as models are updated. Choose based on task complexity, speed requirements, and cost.

Parameters

Fine-tune model behavior:

Temperature

Controls randomness (0-2):

• 0-0.3: Deterministic, consistent (good for classification, extraction)
• 0.7: Balanced (default for most tasks)
• 0.9-1.0: Creative, varied (good for content generation)
• 1.0-2.0: Highly creative (use with caution, can produce less coherent output)

Claude 5 models (for example claude-sonnet-5) and GPT-5 family models (for example gpt-5, gpt-5-mini) do not support the temperature parameter. Any value you set here is ignored for those models. Use Top P instead to control output variation on models that do not support temperature.

Max tokens

Limits response length. Set conservatively to control costs.

System prompt

Optional instructions that set the AI’s behavior for the entire conversation. Use for role definition or global rules.

Advanced sampling parameters

Open the model’s advanced settings for finer control over how the model selects tokens. Each parameter shows Use model default until you turn it on, so you only override what you set deliberately.

Not every provider supports every parameter. The editor shows only the parameters the selected model’s provider accepts — for example, Anthropic models expose Top P and Top K but not the penalties or seed. Leave a parameter on Use model default to use the provider’s own recommended value.

Reasoning (extended thinking)

Some models can spend extra effort working through a problem before they answer — OpenAI’s GPT-5 and o-series, Anthropic’s Claude 4 models, Google’s Gemini 2.5 and 3, and reasoning models from xAI, Together AI, and Groq. When you select a reasoning-capable model, the editor shows a reasoning control. Non-reasoning models hide it entirely.

Turn reasoning on to let the model think before responding. Depending on the provider, you can also tune how much effort it spends:

Higher effort and larger budgets improve results on hard tasks (complex reasoning, multi-step math, code) at the cost of more tokens and slower responses. For simple extraction or classification, leave reasoning off.

Test prompts in the Playground before adding to Flows. This lets you iterate quickly without running the entire Flow each time.

Accessing output

Each prompt step has an outputVariable in its configuration. Reference that variable in later steps:

{{summary_result}}

If the prompt returns JSON (responseFormat: json), access specific properties with dot notation:

{{extract_data.customerName}}
{{extract_data.orderAmount}}

The variable name comes from the step’s outputVariable field, not the step name.

Error handling

Prompt steps can fail if:

• API quota exceeded
• Model rate limits hit
• Invalid API keys
• Prompt too long for model context

Each prompt-step card has an error-handling control that sets what happens when the step fails:

• Continue on error skips the failed step and continues running the rest of the Flow.
• Stop on error halts the Flow immediately.
• Use fallbacks tries a chain of alternatives before giving up.

A fallback chain runs each fallback in order until one succeeds. Each fallback is one of three types:

• Retry re-runs the same model.
• Different model switches to a backup model, with an optional temperature and max tokens override.
• Fixed message returns a pre-written reply without calling a model. Use this as the last fallback so the step always produces a reply.

You also choose what triggers the chain. The step errors runs fallbacks when the step fails outright. The reply is empty runs them when a model finishes successfully but returns no visible text. You can enable both triggers at once.

Tool call strategy

When a prompt step has tools attached, the Tool call strategy dropdown in the tools panel controls whether the model is forced to use a tool:

Keep the strategy on Auto for almost every prompt step. Use Required only when you need exactly one forced tool call: set Max tool calls to 1. Do not combine Required with more than one allowed tool call, or the model can never produce a text answer and the step output comes back empty.

Best practices

• Be specific: Clear instructions produce better results
• Provide examples: Show the AI what you want with examples
• Structure output: Ask for JSON or specific formats when needed
• Keep context concise: Don’t include unnecessary information in prompts
• Test variations: Use Evals to compare different prompts and models

Next steps

• Flow variables and templates for advanced variable usage
• Using transform-data steps to process AI outputs
• Running an Eval to test prompt variations