Submit eval | Runtype Documentation

curl -X POST https://api.runtype.com/v1/eval/submit \
     -H "Authorization: <apiKey>" \
     -H "Content-Type: application/json" \
     -d '{
  "evalConfig": {
    "overrides": {
      "steps": {
        "step_01jv0000000000000000000001": {
          "maxTokens": 2000,
          "model": "gpt-4o",
          "temperature": 0.7
        }
      }
    }
  },
  "evalName": "Ad-hoc Data Test",
  "flowId": "flow_01jv0000000000000000000001",
  "options": {
    "chunkSize": 200,
    "recordConcurrency": 25
  },
  "records": [
    {
      "metadata": {
        "company": "Acme Corp",
        "email": "customer1@example.com",
        "plan": "enterprise"
      },
      "name": "Test Customer 1",
      "type": "customer"
    },
    {
      "metadata": {
        "company": "TechStart Inc",
        "email": "customer2@example.com",
        "plan": "growth"
      },
      "name": "Test Customer 2",
      "type": "customer"
    },
    {
      "metadata": {
        "company": "MegaSoft LLC",
        "email": "customer3@example.com",
        "plan": "starter"
      },
      "name": "Test Customer 3",
      "type": "customer"
    }
  ]
}'

Submit a flow or agent eval. Hosted targets support overrides and multi-model requests; delegated external agents support one baseline config without overrides and not /eval/stream.

Authentication

AuthorizationBearer

API key or Clerk session token

Request

This endpoint expects an object.

agentobjectOptional

Version selector for an agentId target. Supply at most one of alias or versionId; the run resolves it once and every case executes that version.

evalConfigobjectOptional

evalConfigslist of objectsOptional

inputsmap from strings to any or nullOptional

Flow input variables applied to every record execution, as on POST /batch/flow: they override same-named record metadata at the top level and leave _record.metadata unchanged. Flow targets only.

Response

Eval submitted

object

Errors

400

Bad Request Error

401

Unauthorized Error

402

Payment Required Error

403

Forbidden Error

422

Unprocessable Entity Error

500

Internal Server Error

503

Service Unavailable Error