Submit eval | Runtype Documentation

curl -X POST https://api.runtype.com/v1/eval/submit \
     -H "Authorization: <apiKey>" \
     -H "Content-Type: application/json" \
     -d '{
  "evalConfig": {
    "stepOverrides": {
      "step_01jv0000000000000000000001": {
        "maxTokens": 2000,
        "model": "gpt-4o",
        "temperature": 0.7
      }
    }
  },
  "evalName": "Ad-hoc Data Test",
  "flowId": "flow_01jv0000000000000000000001",
  "options": {
    "chunkSize": 200,
    "recordConcurrency": 25
  },
  "records": [
    {
      "metadata": {
        "company": "Acme Corp",
        "email": "customer1@example.com",
        "plan": "enterprise"
      },
      "name": "Test Customer 1",
      "type": "customer"
    },
    {
      "metadata": {
        "company": "TechStart Inc",
        "email": "customer2@example.com",
        "plan": "growth"
      },
      "name": "Test Customer 2",
      "type": "customer"
    },
    {
      "metadata": {
        "company": "MegaSoft LLC",
        "email": "customer3@example.com",
        "plan": "starter"
      },
      "name": "Test Customer 3",
      "type": "customer"
    }
  ]
}'

Submit a flow or agent for evaluation with model/parameter overrides. Supports both single eval and multi-model eval requests.

Authentication

AuthorizationBearer

API key or Clerk session token

Request

This endpoint expects an object.

Response

Eval submitted

object

Errors

400

Bad Request Error

401

Unauthorized Error

402

Payment Required Error

403

Forbidden Error

500

Internal Server Error

1	{
2	"evalGroupId": "string",
3	"submissions": [
4	{
5	"batchExecutionId": "string",
6	"evalName": "string",
7	"status": "string"
8	}
9	],
10	"totalEvals": 1
11	}