Get eval suite coverage | Runtype Documentation

{
  "definitionHash": "a3f5c9d2e7b14f8a9c3d6e2f1b0a4c7d",
  "instructions": [
    {
      "clause": "The system should validate user input for all required fields.",
      "covered": true,
      "coveredBy": [
        "InputValidationGrader"
      ],
      "polarity": "should"
    }
  ],
  "inventoryDegraded": false,
  "inventoryRefreshed": true,
  "summary": {
    "instructionsCovered": 12,
    "instructionsTotal": 15,
    "toolsCovered": 5,
    "toolsTotal": 7
  },
  "tools": [
    {
      "covered": true,
      "coveredBy": [
        "Case_001",
        "Case_005"
      ],
      "name": "AuthTool",
      "description": "User authentication tool for login and session management."
    }
  ]
}

The coverage meter: which of the target’s tools and instruction clauses the suite’s cases and graders already exercise. Tools come structurally from the definition; instruction clauses come from a cached, definition-hash-invalidated inventory pass. Write-gated because a cache miss spends model tokens.

Authentication

AuthorizationBearer

API key or Clerk session token

Path parameters

idstringRequired>=1 character

Response

Coverage report

definitionHashstring

instructionslist of objects

inventoryDegradedboolean

True when the instruction inventory pass failed; the tools axis is still accurate.

inventoryRefreshedboolean

summaryobject

toolslist of objects

Errors

400

Bad Request Error

401

Unauthorized Error

403

Forbidden Error

404

Not Found Error

500

Internal Server Error

$	curl https://api.runtype.com/v1/eval/suites/id/coverage \
>	-H "Authorization: <apiKey>" \
>	-H "Content-Type: application/json"