> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.runtype.com/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.runtype.com/_mcp/server.

# Get eval suite

GET https://api.runtype.com/v1/eval/suites/{id}

Get an eval suite including its cases and latest run + score.

Reference: https://docs.runtype.com/api-reference/runtype-api/evals/get-eval-suite

## Authentication

- `Authorization` header (bearer token, required) — API key or Clerk session token

## Servers

- `https://api.runtype.com` (Production, default)
- `https://api.runtype-staging.com` (Staging)

## Request

### Path parameters

- `id` (string, required)

## Response

### 200

Eval suite returned

- `agentId` (string, required, nullable)
- `baselineBatchExecutionId` (string, required, nullable)
- `caseCount` (integer, required)
- `cases` (list of EvalCase, required)
- `createdAt` (string, required)
- `description` (string, required, nullable)
- `enabledCaseCount` (integer, required)
- `flowId` (string, required, nullable)
- `graders` (list of any, required) — Ordered GraderConfig[] applied to every case.
- `id` (string, required)
- `identityDefaults` (EvalSuiteDetailIdentityDefaults, required, nullable) — Default identity (endUser / tenant) cases run as unless they set their own; null when unset.
- `isDefault` (boolean, required)
- `judgeAgreement` (EvalSuiteDetailJudgeAgreement, required, nullable) — The judge-trust corpus (E6 calibration): how often the AI grader agrees with your reviews, surfaced as a plain fraction. Null until at least one verdict has been reviewed (POST /eval/scores/\{scoreId}/review).
- `lastModifiedSource` (string, required, nullable) — Provenance of the last Definition write. 'sdk' / 'terraform' mean the suite is managed in code (converged by ensure on deploy).
- `latestRun` (EvalSuiteLatestRun, required)
- `name` (string, required)
- `recordedToolMode` (enum, required) — How captured cases replay: "next_step" (default when null) or "continue".
  - Allowed values: `next_step`, `continue`
- `recordedToolUnmatchedPolicy` (enum, required) — Unmatched-call policy for "continue"-mode replays: "fail" (default when null) or "stub".
  - Allowed values: `fail`, `stub`
- `targetName` (string, required, nullable) — Display name of the target flow/agent; null when the target row is gone.
- `targetType` (enum, required)
  - Allowed values: `flow`, `agent`
- `updatedAt` (string, required)
- `warnings` (list of EvalSuiteDetailWarningsItems, optional) — Non-blocking findings about this suite. `AGENT_NOT_DEPLOYED` means the target agent has no runnable `live` version, so a run that names no version selector refuses once the live default is on for the organization.

## Errors

### 401 Unauthorized Error

Unauthorized

- `error` (string, required)
- `code` (string, optional) — Stable error code. `insufficient_scope` marks an API key that lacks a scope this request needs.
- `details` (list of ErrorDetailsItems, optional)
- `requiredScopes` (list of string, optional) — On `insufficient_scope`: every API key scope this request needs.

### 403 Forbidden Error

Forbidden

- `error` (string, required)
- `code` (string, optional) — Stable error code. `insufficient_scope` marks an API key that lacks a scope this request needs.
- `details` (list of ErrorDetailsItems, optional)
- `requiredScopes` (list of string, optional) — On `insufficient_scope`: every API key scope this request needs.

### 404 Not Found Error

Not found

- `error` (string, required)
- `code` (string, optional) — Stable error code. `insufficient_scope` marks an API key that lacks a scope this request needs.
- `details` (list of ErrorDetailsItems, optional)
- `requiredScopes` (list of string, optional) — On `insufficient_scope`: every API key scope this request needs.

### 500 Internal Server Error

Internal server error

- `error` (string, required)
- `code` (string, optional) — Stable error code. `insufficient_scope` marks an API key that lacks a scope this request needs.
- `details` (list of ErrorDetailsItems, optional)
- `requiredScopes` (list of string, optional) — On `insufficient_scope`: every API key scope this request needs.

## Types

### EvalCase

- `createdAt` (string, required)
- `enabled` (boolean, required)
- `graders` (list of any, required, nullable) — Per-case graders that REPLACE the suite graders for this case, or null when the case uses the suite graders.
- `id` (string, required)
- `name` (string, required)
- `notes` (string, required, nullable)
- `origin` (string, required)
- `suiteId` (string, required)
- `updatedAt` (string, required)
- `expected` (any, optional)
- `input` (any, optional) — \{ variables?, messages?, toolMocks?, endUser?, tenant?, record? } — what the case feeds the target, the identity it runs as, and the record it runs on.

### EvalSuiteDetailIdentityDefaults

Default identity (endUser / tenant) cases run as unless they set their own; null when unset.

- `endUser` (EvalSuiteDetailIdentityDefaultsEndUser, optional) — The end user (the person) the turn runs as. Exposed to tools as _endUser.*, drives memory-profile sharding, and satisfies an end-user-isolated tenancyStrategy. Distinct from _user (the Builder User).
- `tenant` (EvalSuiteDetailIdentityDefaultsTenant, optional) — The product tenant (a customer of your product, under your Builder Org) the turn runs within. Exposed to tools as _tenant.*.

### EvalSuiteDetailJudgeAgreement

The judge-trust corpus (E6 calibration): how often the AI grader agrees with your reviews, surfaced as a plain fraction. Null until at least one verdict has been reviewed (POST /eval/scores/\{scoreId}/review).

- `agree` (integer, required) — Reviewed AI-grader verdicts the human agreed with.
- `total` (integer, required) — All reviewed AI-grader verdicts across the suite's runs.

### EvalSuiteLatestRun

- `completedAt` (string, required, nullable)
- `passedCases` (integer, required, nullable)
- `platformAbortedCases` (integer, required, nullable) — Cases the platform aborted before the target could answer. They run no graders and are counted in neither half of score, so totalCases minus this is the denominator score was taken over.
- `runId` (string, required)
- `score` (double, required, nullable) — Suite score of the run, 0..1; null while unscored (in flight / unscoreable).
- `startedAt` (string, required)
- `status` (string, required)
- `totalCases` (integer, required, nullable)

### EvalSuiteDetailWarningsItems

- `code` (string, required)
- `message` (string, required)
- `agentId` (string, optional)

### ErrorDetailsItems

- `message` (string, required)
- `code` (string, optional) — Stable Runtype validation code (e.g. MISSING_REQUIRED_FIELD, INVALID_ENUM).
- `path` (string, optional) — Dotted JSON path to the offending field.
- `suggestion` (string, optional) — Actionable hint (allowed values, accepted range, …).

### EvalSuiteDetailIdentityDefaultsEndUser

The end user (the person) the turn runs as. Exposed to tools as _endUser.*, drives memory-profile sharding, and satisfies an end-user-isolated tenancyStrategy. Distinct from _user (the Builder User).

- `id` (string, required)
- `email` (string, optional)

### EvalSuiteDetailIdentityDefaultsTenant

The product tenant (a customer of your product, under your Builder Org) the turn runs within. Exposed to tools as _tenant.*.

- `id` (string, required)

## Examples

**Response**

```json
{
  "agentId": "string",
  "baselineBatchExecutionId": "string",
  "caseCount": 1,
  "cases": [
    {
      "createdAt": "string",
      "enabled": true,
      "graders": [
        null
      ],
      "id": "string",
      "name": "string",
      "notes": "string",
      "origin": "string",
      "suiteId": "string",
      "updatedAt": "string",
      "input": null
    }
  ],
  "createdAt": "string",
  "description": "string",
  "enabledCaseCount": 1,
  "flowId": "string",
  "graders": [
    null
  ],
  "id": "string",
  "identityDefaults": {
    "endUser": {
      "id": "string",
      "email": "string"
    },
    "tenant": {
      "id": "string"
    }
  },
  "isDefault": true,
  "judgeAgreement": {
    "agree": 1,
    "total": 1
  },
  "lastModifiedSource": "string",
  "latestRun": {
    "completedAt": "string",
    "passedCases": 1,
    "platformAbortedCases": 1,
    "runId": "string",
    "score": 1.1,
    "startedAt": "string",
    "status": "string",
    "totalCases": 1
  },
  "name": "string",
  "recordedToolMode": "next_step",
  "recordedToolUnmatchedPolicy": "fail",
  "targetName": "string",
  "targetType": "flow",
  "updatedAt": "string",
  "warnings": [
    {
      "code": "string",
      "message": "string",
      "agentId": "string"
    }
  ]
}
```

**SDK Code**

```python
import requests

url = "https://api.runtype.com/v1/eval/suites/id"

headers = {"Authorization": "Bearer <token>"}

response = requests.get(url, headers=headers)

print(response.json())
```

```javascript
const url = 'https://api.runtype.com/v1/eval/suites/id';
const options = {method: 'GET', headers: {Authorization: 'Bearer <token>'}};

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}
```

```go
package main

import (
	"fmt"
	"net/http"
	"io"
)

func main() {

	url := "https://api.runtype.com/v1/eval/suites/id"

	req, _ := http.NewRequest("GET", url, nil)

	req.Header.Add("Authorization", "Bearer <token>")

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(res)
	fmt.Println(string(body))

}
```

```ruby
require 'uri'
require 'net/http'

url = URI("https://api.runtype.com/v1/eval/suites/id")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Get.new(url)
request["Authorization"] = 'Bearer <token>'

response = http.request(request)
puts response.read_body
```

```java
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;

HttpResponse<String> response = Unirest.get("https://api.runtype.com/v1/eval/suites/id")
  .header("Authorization", "Bearer <token>")
  .asString();
```

```php
<?php
require_once('vendor/autoload.php');

$client = new \GuzzleHttp\Client();

$response = $client->request('GET', 'https://api.runtype.com/v1/eval/suites/id', [
  'headers' => [
    'Authorization' => 'Bearer <token>',
  ],
]);

echo $response->getBody();
```

```csharp
using RestSharp;

var client = new RestClient("https://api.runtype.com/v1/eval/suites/id");
var request = new RestRequest(Method.GET);
request.AddHeader("Authorization", "Bearer <token>");
IRestResponse response = client.Execute(request);
```

```swift
import Foundation

let headers = ["Authorization": "Bearer <token>"]

let request = NSMutableURLRequest(url: NSURL(string: "https://api.runtype.com/v1/eval/suites/id")! as URL,
                                        cachePolicy: .useProtocolCachePolicy,
                                    timeoutInterval: 10.0)
request.httpMethod = "GET"
request.allHTTPHeaderFields = headers

let session = URLSession.shared
let dataTask = session.dataTask(with: request as URLRequest, completionHandler: { (data, response, error) -> Void in
  if (error != nil) {
    print(error as Any)
  } else {
    let httpResponse = response as? HTTPURLResponse
    print(httpResponse)
  }
})

dataTask.resume()
```