Runtype APIEvals

Generate eval case proposals

Propose test cases from the target’s definition, fanned out over an explicit diversity matrix (category × persona) and filtered for gradeability. Generated cases land as PROPOSALS for review — never directly in the suite. One daily-eval quota slot per call.

Authentication

AuthorizationBearer
API key or Clerk session token

Path parameters

idstringRequired>=1 character

Request

This endpoint expects an object.
countintegerOptional1-12

How many cases to propose (default 8; 3 for gap-targeted generation).

gapobjectOptional

Generate for one coverage gap: cases exercising this tool or probing this instruction clause (proposals land with source “coverage_gap”). Provide exactly one of toolName/instruction.

hintstringOptional<=2000 characters

Focus hint, e.g. “angry customers” or “non-English input”.

Response

Proposals created
droppedCountinteger

Cases dropped by the gradeability audit (unstable pass bar).

generatedCountinteger
Cases the model produced before filtering.
modelstring
Resolved model id that produced the generation.
proposalslist of objects

Errors

400
Bad Request Error
401
Unauthorized Error
402
Payment Required Error
403
Forbidden Error
404
Not Found Error
429
Too Many Requests Error
500
Internal Server Error
502
Bad Gateway Error