For AI agents: a documentation index is available at the root level at /llms.txt and /llms-full.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.
Dashboard
User GuideDeveloper GuidesAPI Reference
User GuideDeveloper GuidesAPI Reference
    • Overview
  • Runtype API
      • POSTAnalyze step results
      • GETList eval batches
      • POSTCompare evals
      • POSTCompare record across evals
      • GETGet eval group
      • STREAMStream eval execution
      • POSTSubmit eval
      • GETGet eval results
Dashboard
LogoLogo
Runtype APIEvals

Compare evals

POST
/v1/eval/compare
POST
/v1/eval/compare
$curl -X POST https://api.runtype.com/v1/eval/compare \
>     -H "Authorization: <apiKey>" \
>     -H "Content-Type: application/json" \
>     -d '{
>  "evalGroupId": "evalgrp_01jv0000000000000000000001",
>  "stepIds": [
>    "step_01jv0000000000000000000001",
>    "step_01jv0000000000000000000002"
>  ]
>}'
1{
2  "evalGroupId": "string",
3  "evals": [
4    {
5      "batchExecutionId": "string",
6      "metrics": {
7        "avgCostPerRecord": 1.1,
8        "avgDurationPerRecord": 1.1,
9        "failedSteps": 1,
10        "successRate": 1.1,
11        "successfulSteps": 1,
12        "totalCompletionTokens": 1,
13        "totalCost": 1.1,
14        "totalDurationMs": 1.1,
15        "totalPromptTokens": 1,
16        "totalSteps": 1
17      },
18      "evalName": "string"
19    }
20  ]
21}
Compare aggregate metrics across eval batches in a group.
Was this page helpful?
Previous

Compare record across evals

Next
Built with

Authentication

AuthorizationBearer
API key or Clerk session token

Request

This endpoint expects an object.
evalGroupIdstringRequired
stepIdslist of stringsOptional

Response

Comparison returned
evalGroupIdstring
evalslist of objects

Errors

400
Bad Request Error
401
Unauthorized Error
403
Forbidden Error
404
Not Found Error
500
Internal Server Error