Calling TypeSafe directly

TypeSafe System One is an API that answers classification and scoring questions about a piece of data and returns a probability for each answer. In this recipe, you call it from a Make API Call step to route a support ticket, then branch on the answer with a Conditional Logic step. A judgment takes about 100 milliseconds and costs a fraction of a cent, so only the tickets that need a larger model reach one.

Use this recipe when you hold a TypeSafe API key and want TypeSafe’s own wire format, or when you need to pin a model version. Otherwise use the builtin:typesafe_judge tool. It runs the same judgment on a Vercel AI Gateway credential, either a Runtype platform key or your organization’s own gateway key, and needs no TypeSafe API key. For what a judgment is and how the tool works, see Judgments.

Before you begin

You need the following before you start:

• A flow open in the Flow editor.
• A TypeSafe API key, stored as a managed secret named TYPESAFE_API_KEY. The direct API needs an early-access key, which you request from TypeSafe. If you don’t have one, use the builtin:typesafe_judge tool instead. For how to store the key, see Managing secrets.
• If you create or run the flow with a Runtype API key, a key with the SECRETS:READ permission. Without it, Runtype rejects the request with a 403 response, because the Make API Call step references a secret.
• A record to run the flow on, with subject, body, and customer_tier in its metadata. For more information, see Creating and managing records.
• The action that the flow takes for each route and for tickets over your escalation threshold.

Select the state fields

A judgment is metered on input tokens only. The state, the instructions, and the criteria all count, and the state is usually the largest part. Select the fields that the questions read instead of sending a whole record. This transform-data step builds one object from a support ticket:

const metadata = input._record?.metadata ?? {}
return {
  subject: metadata.subject ?? '',
  body: metadata.body ?? '',
  customer_tier: metadata.customer_tier ?? 'standard',
}

Set Output variable to ticket.

The state plus the longest question must fit in about 32,000 tokens, roughly 150,000 characters. TypeSafe System One rejects a larger request with a 422 response.

Call TypeSafe System One

Add a Make API Call step, then configure these fields:

• Method: POST.
• URL: https://api.typesafe.ai/v1/systemone.
• Headers: Content-Type: application/json, and Authorization: Bearer {{secret:TYPESAFE_API_KEY}}.
• Output variable: judgment.
• Error handling: Stop on error.

This request body sends the ticket as the state and asks two questions about it:

{
  "state": "{{ticket}}",
  "questions": {
    "route": {
      "type": "choice",
      "instructions": "Read the support ticket and choose the queue that handles it. Choose other when none of the named queues fit.",
      "criteria": {
        "billing": "Invoices, charges, refunds, payment methods, or plan pricing.",
        "technical": "Errors, outages, failed integrations, or unexpected product behavior.",
        "account": "Sign-in problems, seats, permissions, or organization settings.",
        "other": "Anything that none of the other labels describe."
      }
    },
    "needs_human": {
      "type": "noul",
      "instructions": "Estimate the probability that this support ticket needs a human agent rather than an automated reply.",
      "criteria": {
        "true": "The customer reports data loss, disputes a charge, threatens to cancel, or asks something the product documentation does not answer.",
        "false": "The request is routine, and a documented answer resolves it."
      }
    }
  }
}

The {{ticket}} placeholder is the entire value of state, so the step sends the ticket as a JSON object. A placeholder mixed into a longer string is serialized into that string instead, and the state then arrives as text. Give each object-valued field a placeholder of its own.

The state field accepts a string, a JSON object, a JSON array, or null. Prefer named JSON fields when the context has several parts, because a question can then name the field that it reads.

Send a model field to request a specific model. The default is jev-latest, an alias that moves as TypeSafe releases versions. To keep your thresholds stable, set model to a pinned version such as jev-1.13.0. The builtin:typesafe_judge tool defaults to typesafe-ai/jev, which is also a moving alias.

Write the questions

The questions field is a map of name to question, and it can’t be empty. The name is for your flow. TypeSafe System One doesn’t send the name to the model, so instructions and the criteria carry the whole meaning of the question.

Ask independent questions over the same state in one request. They run together and can’t read each other’s answers, so a routing question and an escalation question cost one call instead of two. Keep each question to one narrow decision, and give a choice question a row for the cases that match no other label.

The following table lists the three question types:

Every question needs a type. Give each question instructions as well, because the model doesn’t see the question’s name. The instructions field and every description accept text or JSON.

Read the answer

The response carries the model that answered, one answer per question name, and the token usage:

{
  "model": "jev-1.13.0",
  "answers": {
    "route": {
      "type": "choice",
      "choice": "billing",
      "confidence": 0.39,
      "probabilities": {
        "billing": 0.6,
        "technical": 0.3,
        "account": 0.08,
        "other": 0.02
      }
    },
    "needs_human": {
      "type": "noul",
      "noul": 0.18
    }
  },
  "usage": {
    "input_tokens": 412,
    "output_tokens": 6
  }
}

Record the model value when you tune a set of questions against it, because jev-latest resolves to a different version as TypeSafe releases one.

The following table lists what each answer carries:

Keep the quantities apart

A probability isn’t a decision. Your condition sets the threshold at which the flow acts. The noul probability, a label’s probability, the margin between the top two labels, and a score are different quantities. For how to keep them apart, see Keep the five quantities apart.

A choice or score answer from TypeSafe’s API also carries a confidence field, which TypeSafe computes from the distribution with its own formula. Branch on probabilities and noul instead.

Branch on the answer

Add a conditional step. A condition is JavaScript that runs after Runtype substitutes the {{...}} references, so put quotes around a placeholder that you compare with a string, and leave a numeric one bare:

'{{judgment.answers.route.choice}}' === 'billing' && {{judgment.answers.needs_human.noul}} < 0.6

Add one branch for each of billing, technical, and account, each with the same needs_human clause. The other label and any ticket at or over the threshold fall through to Otherwise, where a larger model drafts a reply for a person to review.

The threshold 0.6 is in the condition, not in the request, so you can change it without calling the model again.

Store the raw answers

To change a threshold later without paying for inference again, write the answers to a record. Record filters read top-level metadata keys only, so a filter can’t read a nested value such as answers.needs_human.noul. Flatten the values you filter on into top-level fields first. This transform-data step builds that object:

const answers = input.judgment.answers
return {
  route: answers.route.choice,
  route_probability: answers.route.probabilities[answers.route.choice],
  needs_human: answers.needs_human.noul,
  judged_by: input.judgment.model,
  raw_answers: answers,
}

Set Output variable to judgment_fields. Then add an upsert-record step and set its source variable to judgment_fields. A filter such as needs_human greater than 0.6 then reads the stored row, and re-thresholding is a query instead of a second call to the model. The raw_answers field keeps the full distributions for a later question that the flat fields don’t answer.

Test the flow

Run the flow on one record, for example with a batch run over a record type that holds a sample ticket. Then open the run in Logs and check these results:

• The Make API Call step’s output holds answers.route.choice and answers.needs_human.noul.
• The branch that ran matches the route and the threshold.
• The stored record has route and needs_human as top-level metadata fields.

For how to read a run’s step results, see Debugging flows.

Handle errors

The Make API Call step uses Stop on error because an HTTP step that continues on error writes its default value, and the branch then reads a judgment that no model produced.

A step has one error-handling mode, so Use fallbacks replaces Stop on error. When a retry fallback runs out of attempts, the flow continues to the next step without a judgment variable, and every branch condition then fails. To retry a 429 or a 529 inside the flow, choose Use fallbacks, add a retry fallback with a delay, and add a step after the request that checks for a missing judgment and stops or escalates. Otherwise keep Stop on error and retry the whole run.

When the step fails, its error in the run results shows the HTTP status and TypeSafe’s response body. Read the body to tell a malformed question from an over-budget request.

The following table lists the responses to handle:

A TypeSafe API key allows about 1,200 requests per minute and 250,000 tokens per second. When you run the flow over many records, keep the request rate under that quota. For the current limits, see TypeSafe’s models page.

Next steps

• Judgments: run the same judgment with the builtin:typesafe_judge tool and a versioned question set.
• Using conditional steps: write the branch conditions that read an answer.
• Using fetch-url and api-call steps: configure headers, request bodies, and response mapping.
• Managing secrets: store the TypeSafe API key.
• Creating and editing flows: build and import flows.