For AI agents: a documentation index is available at the root level at /llms.txt and /llms-full.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.
Dashboard
User GuideDeveloper GuidesAPI Reference
User GuideDeveloper GuidesAPI Reference
    • Overview
  • Runtype API
      • POSTAnalyze step results
      • GETList eval batches
      • POSTCompare evals
      • POSTCompare record across evals
      • GETGet eval group
      • STREAMStream eval execution
      • POSTSubmit eval
      • GETGet eval results
Dashboard
LogoLogo
Runtype APIEvals

Submit eval

POST
/v1/eval/submit
POST
/v1/eval/submit
$curl -X POST https://api.runtype.com/v1/eval/submit \
>     -H "Authorization: <apiKey>" \
>     -H "Content-Type: application/json" \
>     -d '{
>  "evalConfig": {
>    "stepOverrides": {
>      "step_01jv0000000000000000000001": {
>        "maxTokens": 2000,
>        "model": "gpt-4o",
>        "temperature": 0.7
>      }
>    }
>  },
>  "evalName": "Ad-hoc Data Test",
>  "flowId": "flow_01jv0000000000000000000001",
>  "options": {
>    "chunkSize": 200,
>    "recordConcurrency": 25
>  },
>  "records": [
>    {
>      "metadata": {
>        "company": "Acme Corp",
>        "email": "customer1@example.com",
>        "plan": "enterprise"
>      },
>      "name": "Test Customer 1",
>      "type": "customer"
>    },
>    {
>      "metadata": {
>        "company": "TechStart Inc",
>        "email": "customer2@example.com",
>        "plan": "growth"
>      },
>      "name": "Test Customer 2",
>      "type": "customer"
>    },
>    {
>      "metadata": {
>        "company": "MegaSoft LLC",
>        "email": "customer3@example.com",
>        "plan": "starter"
>      },
>      "name": "Test Customer 3",
>      "type": "customer"
>    }
>  ]
>}'
1{
2  "evalGroupId": "string",
3  "submissions": [
4    {
5      "batchExecutionId": "string",
6      "evalName": "string",
7      "status": "string"
8    }
9  ],
10  "totalEvals": 1
11}

Submit a flow or agent for evaluation with model/parameter overrides. Supports both single eval and multi-model eval requests.

Was this page helpful?
Previous

Get eval results

Next
Built with

Authentication

AuthorizationBearer
API key or Clerk session token

Request

This endpoint expects an object.

Response

Eval submitted
object
OR
object

Errors

400
Bad Request Error
401
Unauthorized Error
402
Payment Required Error
403
Forbidden Error
500
Internal Server Error