> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.runtype.com/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.runtype.com/_mcp/server.

# Update eval suite

PATCH https://api.runtype.com/v1/eval/suites/{id}
Content-Type: application/json

Update a suite’s Definition (name, description, graders) or state (default flag, baseline run pin). The target is immutable — create a new suite to evaluate a different flow/agent. Definition edits stamp provenance; a graders change invalidates the config-as-code content hash so the next ensure re-converges.

Reference: https://docs.runtype.com/api-reference/runtype-api/evals/update-eval-suite

## Authentication

- `Authorization` header (bearer token, required) — API key or Clerk session token

## Servers

- `https://api.runtype.com` (Production, default)
- `https://api.runtype-staging.com` (Staging)

## Request

### Path parameters

- `id` (string, required)

### Body (application/json)

This endpoint expects an object.

- `baselineBatchExecutionId` (string, optional, nullable) — Pin a run of this suite as the comparison baseline; null clears the pin.
- `description` (string, optional, nullable)
- `graders` (list of V1EvalSuitesIdPatchRequestBodyContentApplicationJsonSchemaGradersItems, optional)
- `identityDefaults` (V1EvalSuitesIdPatchRequestBodyContentApplicationJsonSchemaIdentityDefaults, optional, nullable) — Default identity (endUser / tenant) every case runs as unless the case sets its own; replaces the stored value. null clears it.
- `isDefault` (boolean, optional)
- `name` (string, optional)
- `recordedToolMode` (enum, optional) — How captured (saved_from_run) cases replay. "next_step" captures the first tool intent and stops; "continue" serves recorded toolMocks and runs past the first action. null resets to the default (next_step).
  - Allowed values: `next_step`, `continue`
- `recordedToolUnmatchedPolicy` (enum, optional) — For "continue"-mode replays, the unmatched-call policy: "fail" (divergence is the signal) or "stub" (canned success). null resets to the default (fail).
  - Allowed values: `fail`, `stub`

## Response

### 200

Suite updated

- `agentId` (string, required, nullable)
- `baselineBatchExecutionId` (string, required, nullable)
- `caseCount` (integer, required)
- `createdAt` (string, required)
- `description` (string, required, nullable)
- `enabledCaseCount` (integer, required)
- `flowId` (string, required, nullable)
- `graders` (list of any, required) — Ordered GraderConfig[] applied to every case.
- `id` (string, required)
- `identityDefaults` (EvalSuiteSummaryIdentityDefaults, required, nullable) — Default identity (endUser / tenant) cases run as unless they set their own; null when unset.
- `isDefault` (boolean, required)
- `lastModifiedSource` (string, required, nullable) — Provenance of the last Definition write. 'sdk' / 'terraform' mean the suite is managed in code (converged by ensure on deploy).
- `latestRun` (EvalSuiteLatestRun, required)
- `name` (string, required)
- `recordedToolMode` (enum, required) — How captured cases replay: "next_step" (default when null) or "continue".
  - Allowed values: `next_step`, `continue`
- `recordedToolUnmatchedPolicy` (enum, required) — Unmatched-call policy for "continue"-mode replays: "fail" (default when null) or "stub".
  - Allowed values: `fail`, `stub`
- `targetName` (string, required, nullable) — Display name of the target flow/agent; null when the target row is gone.
- `targetType` (enum, required)
  - Allowed values: `flow`, `agent`
- `updatedAt` (string, required)
- `warnings` (list of EvalSuiteSummaryWarningsItems, optional) — Non-blocking findings about this suite. `AGENT_NOT_DEPLOYED` means the target agent has no runnable `live` version, so a run that names no version selector refuses once the live default is on for the organization.

## Errors

### 400 Bad Request Error

Invalid request

- `error` (string, required)
- `code` (string, optional) — Stable error code. `insufficient_scope` marks an API key that lacks a scope this request needs.
- `details` (list of ErrorDetailsItems, optional)
- `requiredScopes` (list of string, optional) — On `insufficient_scope`: every API key scope this request needs.

### 401 Unauthorized Error

Unauthorized

- `error` (string, required)
- `code` (string, optional) — Stable error code. `insufficient_scope` marks an API key that lacks a scope this request needs.
- `details` (list of ErrorDetailsItems, optional)
- `requiredScopes` (list of string, optional) — On `insufficient_scope`: every API key scope this request needs.

### 403 Forbidden Error

Forbidden

- `error` (string, required)
- `code` (string, optional) — Stable error code. `insufficient_scope` marks an API key that lacks a scope this request needs.
- `details` (list of ErrorDetailsItems, optional)
- `requiredScopes` (list of string, optional) — On `insufficient_scope`: every API key scope this request needs.

### 404 Not Found Error

Not found

- `error` (string, required)
- `code` (string, optional) — Stable error code. `insufficient_scope` marks an API key that lacks a scope this request needs.
- `details` (list of ErrorDetailsItems, optional)
- `requiredScopes` (list of string, optional) — On `insufficient_scope`: every API key scope this request needs.

### 500 Internal Server Error

Internal server error

- `error` (string, required)
- `code` (string, optional) — Stable error code. `insufficient_scope` marks an API key that lacks a scope this request needs.
- `details` (list of ErrorDetailsItems, optional)
- `requiredScopes` (list of string, optional) — On `insufficient_scope`: every API key scope this request needs.

## Types

### V1EvalSuitesIdPatchRequestBodyContentApplicationJsonSchemaGradersItems

- `kind` (string, required)

### V1EvalSuitesIdPatchRequestBodyContentApplicationJsonSchemaIdentityDefaults

Default identity (endUser / tenant) every case runs as unless the case sets its own; replaces the stored value. null clears it.

- `endUser` (V1EvalSuitesIdPatchRequestBodyContentApplicationJsonSchemaIdentityDefaultsEndUser, optional) — The end user (the person) the turn runs as. Exposed to tools as _endUser.*, drives memory-profile sharding, and satisfies an end-user-isolated tenancyStrategy. Distinct from _user (the Builder User).
- `tenant` (V1EvalSuitesIdPatchRequestBodyContentApplicationJsonSchemaIdentityDefaultsTenant, optional) — The product tenant (a customer of your product, under your Builder Org) the turn runs within. Exposed to tools as _tenant.*.

### EvalSuiteSummaryIdentityDefaults

Default identity (endUser / tenant) cases run as unless they set their own; null when unset.

- `endUser` (EvalSuiteSummaryIdentityDefaultsEndUser, optional) — The end user (the person) the turn runs as. Exposed to tools as _endUser.*, drives memory-profile sharding, and satisfies an end-user-isolated tenancyStrategy. Distinct from _user (the Builder User).
- `tenant` (EvalSuiteSummaryIdentityDefaultsTenant, optional) — The product tenant (a customer of your product, under your Builder Org) the turn runs within. Exposed to tools as _tenant.*.

### EvalSuiteLatestRun

- `completedAt` (string, required, nullable)
- `passedCases` (integer, required, nullable)
- `platformAbortedCases` (integer, required, nullable) — Cases the platform aborted before the target could answer. They run no graders and are counted in neither half of score, so totalCases minus this is the denominator score was taken over.
- `runId` (string, required)
- `score` (double, required, nullable) — Suite score of the run, 0..1; null while unscored (in flight / unscoreable).
- `startedAt` (string, required)
- `status` (string, required)
- `totalCases` (integer, required, nullable)

### EvalSuiteSummaryWarningsItems

- `code` (string, required)
- `message` (string, required)
- `agentId` (string, optional)

### ErrorDetailsItems

- `message` (string, required)
- `code` (string, optional) — Stable Runtype validation code (e.g. MISSING_REQUIRED_FIELD, INVALID_ENUM).
- `path` (string, optional) — Dotted JSON path to the offending field.
- `suggestion` (string, optional) — Actionable hint (allowed values, accepted range, …).

### V1EvalSuitesIdPatchRequestBodyContentApplicationJsonSchemaIdentityDefaultsEndUser

The end user (the person) the turn runs as. Exposed to tools as _endUser.*, drives memory-profile sharding, and satisfies an end-user-isolated tenancyStrategy. Distinct from _user (the Builder User).

- `id` (string, required)
- `email` (string, optional)

### V1EvalSuitesIdPatchRequestBodyContentApplicationJsonSchemaIdentityDefaultsTenant

The product tenant (a customer of your product, under your Builder Org) the turn runs within. Exposed to tools as _tenant.*.

- `id` (string, required)

### EvalSuiteSummaryIdentityDefaultsEndUser

The end user (the person) the turn runs as. Exposed to tools as _endUser.*, drives memory-profile sharding, and satisfies an end-user-isolated tenancyStrategy. Distinct from _user (the Builder User).

- `id` (string, required)
- `email` (string, optional)

### EvalSuiteSummaryIdentityDefaultsTenant

The product tenant (a customer of your product, under your Builder Org) the turn runs within. Exposed to tools as _tenant.*.

- `id` (string, required)

## Examples

**Request**

```json
{}
```

**Response**

```json
{
  "agentId": "string",
  "baselineBatchExecutionId": "string",
  "caseCount": 1,
  "createdAt": "string",
  "description": "string",
  "enabledCaseCount": 1,
  "flowId": "string",
  "graders": [
    null
  ],
  "id": "string",
  "identityDefaults": {
    "endUser": {
      "id": "string",
      "email": "string"
    },
    "tenant": {
      "id": "string"
    }
  },
  "isDefault": true,
  "lastModifiedSource": "string",
  "latestRun": {
    "completedAt": "string",
    "passedCases": 1,
    "platformAbortedCases": 1,
    "runId": "string",
    "score": 1.1,
    "startedAt": "string",
    "status": "string",
    "totalCases": 1
  },
  "name": "string",
  "recordedToolMode": "next_step",
  "recordedToolUnmatchedPolicy": "fail",
  "targetName": "string",
  "targetType": "flow",
  "updatedAt": "string",
  "warnings": [
    {
      "code": "string",
      "message": "string",
      "agentId": "string"
    }
  ]
}
```

**SDK Code**

```python
import requests

url = "https://api.runtype.com/v1/eval/suites/id"

payload = {}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.patch(url, json=payload, headers=headers)

print(response.json())
```

```javascript
const url = 'https://api.runtype.com/v1/eval/suites/id';
const options = {
  method: 'PATCH',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: '{}'
};

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}
```

```go
package main

import (
	"fmt"
	"strings"
	"net/http"
	"io"
)

func main() {

	url := "https://api.runtype.com/v1/eval/suites/id"

	payload := strings.NewReader("{}")

	req, _ := http.NewRequest("PATCH", url, payload)

	req.Header.Add("Authorization", "Bearer <token>")
	req.Header.Add("Content-Type", "application/json")

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(res)
	fmt.Println(string(body))

}
```

```ruby
require 'uri'
require 'net/http'

url = URI("https://api.runtype.com/v1/eval/suites/id")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Patch.new(url)
request["Authorization"] = 'Bearer <token>'
request["Content-Type"] = 'application/json'
request.body = "{}"

response = http.request(request)
puts response.read_body
```

```java
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;

HttpResponse<String> response = Unirest.patch("https://api.runtype.com/v1/eval/suites/id")
  .header("Authorization", "Bearer <token>")
  .header("Content-Type", "application/json")
  .body("{}")
  .asString();
```

```php
<?php
require_once('vendor/autoload.php');

$client = new \GuzzleHttp\Client();

$response = $client->request('PATCH', 'https://api.runtype.com/v1/eval/suites/id', [
  'body' => '{}',
  'headers' => [
    'Authorization' => 'Bearer <token>',
    'Content-Type' => 'application/json',
  ],
]);

echo $response->getBody();
```

```csharp
using RestSharp;

var client = new RestClient("https://api.runtype.com/v1/eval/suites/id");
var request = new RestRequest(Method.PATCH);
request.AddHeader("Authorization", "Bearer <token>");
request.AddHeader("Content-Type", "application/json");
request.AddParameter("application/json", "{}", ParameterType.RequestBody);
IRestResponse response = client.Execute(request);
```

```swift
import Foundation

let headers = [
  "Authorization": "Bearer <token>",
  "Content-Type": "application/json"
]
let parameters = [] as [String : Any]

let postData = JSONSerialization.data(withJSONObject: parameters, options: [])

let request = NSMutableURLRequest(url: NSURL(string: "https://api.runtype.com/v1/eval/suites/id")! as URL,
                                        cachePolicy: .useProtocolCachePolicy,
                                    timeoutInterval: 10.0)
request.httpMethod = "PATCH"
request.allHTTPHeaderFields = headers
request.httpBody = postData as Data

let session = URLSession.shared
let dataTask = session.dataTask(with: request as URLRequest, completionHandler: { (data, response, error) -> Void in
  if (error != nil) {
    print(error as Any)
  } else {
    let httpResponse = response as? HTTPURLResponse
    print(httpResponse)
  }
})

dataTask.resume()
```