Compare record across evals | Runtype Documentation

curl -X POST https://api.runtype.com/v1/eval/compare-record \
     -H "Authorization: <apiKey>" \
     -H "Content-Type: application/json" \
     -d '{
  "evalGroupId": "evalgrp_01jv0000000000000000000001",
  "recordId": "record_01jv0000000000000000000001",
  "stepIds": [
    "step_01jv0000000000000000000001",
    "step_01jv0000000000000000000002"
  ]
}'

{
  "evalGroupId": "string",
  "evals": [
    {
      "batchExecutionId": "string",
      "stepResults": [
        {
          "completionTokens": 1,
          "durationMs": 1,
          "executedAt": "string",
          "modelUsed": "string",
          "output": "string",
          "promptTokens": 1,
          "stepId": "string",
          "stepName": "string",
          "stepResultId": "string",
          "stepType": "string",
          "toolCalls": [
            null
          ],
          "totalCost": 1.1,
          "error": "string"
        }
      ],
      "evalName": "string"
    }
  ],
  "recordId": "string"
}

Compare results for a specific record across eval batches in a group.

Authentication

AuthorizationBearer

API key or Clerk session token

Request

This endpoint expects an object.

evalGroupIdstringRequired

recordIdstringRequired

stepIdslist of stringsOptional

Response

Record comparison returned

evalGroupIdstring

evalslist of objects

recordIdstring

Errors

400

Bad Request Error

401

Unauthorized Error

403

Forbidden Error

404

Not Found Error

500

Internal Server Error