Submit eval | Runtype Documentation

curl -X POST https://api.runtype.com/v1/eval/submit \
     -H "Authorization: <apiKey>" \
     -H "Content-Type: application/json" \
     -d '{
  "evalConfig": {
    "overrides": {
      "steps": {
        "step_01jv0000000000000000000001": {
          "maxTokens": 2000,
          "model": "gpt-4o",
          "temperature": 0.7
        }
      }
    }
  },
  "evalName": "Ad-hoc Data Test",
  "flowId": "flow_01jv0000000000000000000001",
  "options": {
    "chunkSize": 200,
    "recordConcurrency": 25
  },
  "records": [
    {
      "metadata": {
        "company": "Acme Corp",
        "email": "customer1@example.com",
        "plan": "enterprise"
      },
      "name": "Test Customer 1",
      "type": "customer"
    },
    {
      "metadata": {
        "company": "TechStart Inc",
        "email": "customer2@example.com",
        "plan": "growth"
      },
      "name": "Test Customer 2",
      "type": "customer"
    },
    {
      "metadata": {
        "company": "MegaSoft LLC",
        "email": "customer3@example.com",
        "plan": "starter"
      },
      "name": "Test Customer 3",
      "type": "customer"
    }
  ]
}'

Submit a flow or agent eval. Hosted targets support overrides and multi-model requests; delegated external agents support one baseline config without overrides and not /eval/stream.

Authentication

AuthorizationBearer

API key or Clerk session token

Request

This endpoint expects an object.

agentobjectOptional

Version selector for an agentId target. Supply at most one of alias or versionId; the run resolves it once and every case executes that version.

evalConfigobjectOptional

evalConfigslist of objectsOptional

Response

Eval submitted

object

Errors

400

Bad Request Error

401

Unauthorized Error

402

Payment Required Error

403

Forbidden Error

500

Internal Server Error

1	{
2	"evalGroupId": "string",
3	"submissions": [
4	{
5	"batchExecutionId": "string",
6	"evalName": "string",
7	"status": "string"
8	}
9	],
10	"totalEvals": 1
11	}