Skip to navigation

Ensure eval suite (config-as-code converge)

Idempotently converge a repo-defined eval suite (target + cases + graders) onto the platform. Identity is the suite name + account scope; the target flow/agent is resolved by name. Hash-only probes skip the payload in the steady state (a miss is a normal 200 definitionRequired response, not an error); full requests create the suite or replace the cases ensure owns when the canonical content hash differs. Cases saved from runs, added in the dashboard, or owned by a product are never replaced. Set dryRun to plan without writing (CI drift gate), and onConflict: “overwrite” to converge over dashboard/API edits. Non-executing: never runs the eval. The response always carries the server-computed canonical hash.

Authentication

AuthorizationBearer
API key or Clerk session token

Request

This endpoint expects an object.
namestringRequired>=1 character
contentHashstringOptionalformat: "^[0-9a-f]{64}$"
definitionobjectOptional
dryRunbooleanOptional
expectedRemoteHashstringOptionalformat: "^[0-9a-f]{64}$"
onConflictenumOptional
Allowed values:

Response

Converge result: unchanged | created | updated | definitionRequired (hash miss — retry with the full definition) | plan (dryRun)

object
OR
object
OR
object

Errors

400
Bad Request Error
401
Unauthorized Error
403
Forbidden Error
409
Conflict Error
422
Unprocessable Entity Error
500
Internal Server Error