Ensure eval suite (config-as-code converge)
Idempotently converge a repo-defined eval suite (target + cases + graders) onto the platform. Identity is the suite name + account scope; the target flow/agent is resolved by name. Hash-only probes skip the payload in the steady state (a miss is a normal 200 definitionRequired response, not an error); full requests create the suite or replace the cases ensure owns when the canonical content hash differs. Cases saved from runs, added in the dashboard, or owned by a product are never replaced. Set dryRun to plan without writing (CI drift gate), and onConflict: “overwrite” to converge over dashboard/API edits. Non-executing: never runs the eval. The response always carries the server-computed canonical hash.
Authentication
Request
Response
Converge result: unchanged | created | updated | definitionRequired (hash miss — retry with the full definition) | plan (dryRun)