Update eval suite
Update a suite’s Definition (name, description, graders) or state (default flag, baseline run pin). The target is immutable — create a new suite to evaluate a different flow/agent. Definition edits stamp provenance; a graders change invalidates the config-as-code content hash so the next ensure re-converges.
Authentication
Path parameters
Request
Pin a run of this suite as the comparison baseline; null clears the pin.
How captured (saved_from_run) cases replay. “next_step” captures the first tool intent and stops; “continue” serves recorded toolMocks and runs past the first action. null resets to the default (next_step).
For “continue”-mode replays, the unmatched-call policy: “fail” (divergence is the signal) or “stub” (canned success). null resets to the default (fail).
Response
Ordered GraderConfig[] applied to every case.
Provenance of the last Definition write. ‘sdk’ / ‘terraform’ mean the suite is managed in code (converged by ensure on deploy).
How captured cases replay: “next_step” (default when null) or “continue”.
Unmatched-call policy for “continue”-mode replays: “fail” (default when null) or “stub”.
Display name of the target flow/agent; null when the target row is gone.