Get persisted grader scores for an eval run

GET

/v1/eval/runs/:batchId/scores

GET

/v1/eval/runs/:batchId/scores

curl https://api.runtype.com/v1/eval/runs/batchId/scores \
     -H "Authorization: <apiKey>"

{
  "cases": [
    {
      "caseId": "string",
      "error": "string",
      "name": "string",
      "nextStepToolCalls": [
        {
          "name": "string",
          "input": null
        }
      ],
      "outcomes": [
        {
          "graderIndex": 1,
          "humanVerdict": "agree",
          "kind": "string",
          "passed": true,
          "scoreId": "string",
          "evidence": [
            {
              "index": 1,
              "kind": "message"
            }
          ],
          "judgeDiagnostics": {
            "attempts": [
              {
                "attempt": 1,
                "durationMs": 1,
                "requestedModel": "string",
                "status": "valid",
                "usageObserved": true,
                "cacheReadTokens": 1,
                "cacheWriteTokens": 1,
                "completionTokens": 1,
                "finishReason": "stop",
                "modelError": "empty_completion",
                "parseError": "empty_completion",
                "promptTokens": 1,
                "provider": "string",
                "reasoningTokens": 1,
                "resolvedModel": "string",
                "responseChars": 1
              }
            ],
            "flowFailure": "deadline_exhausted"
          },
          "reasoning": "string",
          "score": 1.1,
          "severity": "gate",
          "verdict": "pass"
        }
      ],
      "outputExcerpt": "string",
      "passed": true,
      "executionEngine": "runtime",
      "failureKind": "platform_stall"
    }
  ],
  "name": "string",
  "passedCases": 1,
  "platformAbortedCases": 1,
  "runId": "string",
  "score": 1.1,
  "strict": true,
  "suiteId": "string",
  "totalCases": 1,
  "agentTargetAlias": "string",
  "agentTargetResolution": "alias",
  "agentVersionId": "string",
  "caseManifestHash": "string",
  "evaluatorFingerprint": "string"
}

Return the persisted per-case grader scores for a run (a batchExecutions row) — the read side of eval score persistence. Grader outcomes use the same shape as the synchronous POST /eval/run response, so live and historical results render identically. Runs land here from persisted saved-suite runs (POST /eval/run) and completed suite-linked batch runs. Returns an empty case list for a run with no persisted scores (ephemeral/virtual runs, ad-hoc comparisons, legacy evals).

Authentication

AuthorizationBearer

API key or Clerk session token

Path parameters

batchIdstringRequired>=1 character

Response

Per-case grader scores for the run

caseslist of objects

namestring or null

The eval name recorded on the run.

passedCasesinteger

platformAbortedCasesinteger

Cases the platform aborted before the target could answer. They run no graders and are counted in neither half of score, so totalCases minus this is the denominator score was taken over.

runIdstring

The run (batch execution) id these scores belong to.

scoredouble or null

Suite score, 0..1 (passed cases / graded cases). Platform-aborted cases are excluded from both halves; null when the run graded no cases.

strictboolean

The severity semantics the run was graded with (and that these verdicts reproduce): true when a soft grader miss failed its case.

suiteIdstring or null

The eval suite the run executed; null when the suite was since deleted.

totalCasesinteger

agentTargetAliasstring or nullOptional

The release alias the run resolved through, when it resolved through one.

agentTargetResolutionenumOptional

How the run picked its version. Null on a run that predates release aliases.

Allowed values:

agentVersionIdstring or nullOptional

The exact Agent version every case of this run executed. Null for a legacy run.

caseManifestHashstring or nullOptional

SHA-256 over the ordered, normalized case set this run executed.

evaluatorFingerprintstring or nullOptional

SHA-256 over the evaluator configuration this run scored with.

Errors

401

Unauthorized Error

403

Forbidden Error

404

Not Found Error

500

Internal Server Error