Get persisted grader scores for an eval run

GET

/v1/eval/runs/:batchId/scores

GET

/v1/eval/runs/:batchId/scores

$ curl https://api.runtype.com/v1/eval/runs/batchId/scores \
>      -H "Authorization: <apiKey>" \
>      -H "Content-Type: application/json"

1 {
2   "cases": [
3     {
4       "caseId": "case_9f8b7c6d",
5       "name": "Sentiment Analysis Test Case #12",
6       "nextStepToolCalls": [
7         {
8           "name": "sentimentAnalyzer",
9           "input": {
10             "text": "Analyze sentiment of the following review: 'I love this product!'"
11           }
12         }
13       ],
14       "outcomes": [
15         {
16           "graderIndex": 0,
17           "humanVerdict": "agree",
18           "kind": "ai_grader",
19           "passed": true,
20           "scoreId": "score_4a7d2e1f",
21           "evidence": [
22             {
23               "index": 3,
24               "kind": "message"
25             },
26             {
27               "index": 5,
28               "kind": "tool_call"
29             }
30           ],
31           "reasoning": "The model correctly identified the sentiment as positive based on the review text.",
32           "score": 0.95,
33           "severity": "gate",
34           "verdict": "pass"
35         }
36       ],
37       "outputExcerpt": "Sentiment: Positive\nConfidence: 0.95\nThe product review expresses a positive opinion.",
38       "passed": true
39     }
40   ],
41   "name": "Sentiment Analysis Evaluation Run - April 2024",
42   "passedCases": 1,
43   "runId": "run_20240415_001",
44   "score": 1,
45   "strict": true,
46   "suiteId": "suite_sentiment_v2",
47   "totalCases": 1
48 }

Return the persisted per-case grader scores for a run (a batchExecutions row) — the read side of eval score persistence. Grader outcomes use the same shape as the synchronous POST /eval/run response, so live and historical results render identically. Runs land here from persisted saved-suite runs (POST /eval/run) and completed suite-linked batch runs. Returns an empty case list for a run with no persisted scores (ephemeral/virtual runs, ad-hoc comparisons, legacy evals).

Return the persisted per-case grader scores for a run (a `batchExecutions` row) — the read side of eval score persistence. Grader outcomes use the same shape as the synchronous POST /eval/run response, so live and historical results render identically. Runs land here from persisted saved-suite runs (POST /eval/run) and completed suite-linked batch runs. Returns an empty case list for a run with no persisted scores (ephemeral/virtual runs, ad-hoc comparisons, legacy evals).

Authentication

AuthorizationBearer

API key or Clerk session token

Path parameters

batchIdstringRequired>=1 character

Response

Per-case grader scores for the run

caseslist of objects

namestring or null

The eval name recorded on the run.

passedCasesinteger

runIdstring

The run (batch execution) id these scores belong to.

scoredouble or null

Suite score, 0..1 (passed cases / total cases); null when the run has no cases.

strictboolean

The severity semantics the run was graded with (and that these verdicts reproduce): true when a soft grader miss failed its case.

suiteIdstring or null

The eval suite the run executed; null when the suite was since deleted.

totalCasesinteger

Errors

401

Unauthorized Error

403

Forbidden Error

404

Not Found Error

500

Internal Server Error