> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.runtype.com/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.runtype.com/_mcp/server. # Run simulations with a local coding harness > Connect an installed Claude Code or Codex harness to a Runtype simulation and inspect its tool evidence. Connect an installed coding harness as an external simulation target. Runtype runs the simulated users and grading. Your machine runs the harness and its configured Model Context Protocol (MCP) tools. The CLI manages the local host, device listener, external Agent, and simulation target. You do not need a repository checkout or a separate wrapper process. | Configuration | Exact versions | Verification scope | | ------------------------------------------------ | -------------------------------------------------------------------------------- | --------------------------------------------------------------------------- | | Claude Code, native backend | Claude Code `2.1.314`, Agent SDK `0.3.308` | Installed runtime and local simulation report with a fixture provider | | Codex, native backend | Codex `0.154.0`, app-server protocol | Installed runtime and packed CLI simulation report with a fixture provider. | | Claude Code, Agent Client Protocol (ACP) backend | Claude Code `2.1.314`, `claude-agent-acp` `0.99.1`, Agent SDK `0.3.312`, ACP `1` | Installed adapter and packed CLI simulation report with a fixture provider. | A fresh Codex report also verifies per-turn provenance, the report-wide capture summary, and the declared profile and request-limit qualifications. Installed-runtime checks use local fixtures on macOS. Linux process supervision has separate tests. These checks do not establish staging deployment or live provider compatibility. The host supports macOS and Linux and rejects Windows. Other version combinations are rejected. Codex through ACP and the `workspace` profile are unavailable. ## Prepare the connection Use Node.js 22.12.0 or later and pnpm. 1. Install a CLI release that includes `runtype harness`: **`Install the Runtype CLI`** ```bash title="Install the Runtype CLI" pnpm add --global @runtypelabs/cli ``` 2. Sign in to the Runtype organization that owns the simulation: **`Sign in to Runtype`** ```bash title="Sign in to Runtype" runtype auth login ``` For an API key, grant `AGENTS:READ`, `AGENTS:WRITE`, `DEVICES:READ`, `DEVICES:WRITE`, `DEVICES:EXECUTE`, `SIMULATIONS:READ`, `SIMULATIONS:WRITE`, `SIMULATIONS:RUN`, and `TELEMETRY:WRITE`. Use a production-mode API key, including against a staging API deployment. Test keys cannot authorize device execution or simulation runs, even when the requested permissions include those scopes. Use an API deployment that supports signed harness telemetry. The host validates the per-turn binding and telemetry permission before starting a model turn. An older API deployment cannot run this host. 3. Install the exact harness version from the compatibility table. Authenticate with that harness before connecting. For Codex, install the pinned package: **`Install Codex`** ```bash title="Install Codex" pnpm add --global @openai/codex@0.154.0 codex login ``` For native Codex, the host copies file-backed `auth.json` and `.credentials.json` into an isolated configuration directory. It can also use `OPENAI_API_KEY` or `CODEX_API_KEY`. It does not copy your settings or previous sessions. Keep the matching `codex-code-mode-host` binary installed with Codex. The controlled profile uses Codex Code Mode to call the configured MCP tools. It excludes built-in tool namespaces and disables the in-process fallback. A missing or disabled host fails readiness checks. For Claude Code, use its installer to select `2.1.314` and sign in, or set `ANTHROPIC_API_KEY`. Use `--executable /absolute/path/to/claude` or `--executable /absolute/path/to/codex` to select an installed binary. The Runtype CLI does not silently install or upgrade the harness. 4. Create `mcp.json` with only the servers the simulation may call: **`mcp.json`** ```json title="mcp.json" { "mcpServers": { "fixture": { "url": "http://127.0.0.1:4555/mcp" } } } ``` Remote URLs require HTTPS. HTTP is allowed for loopback servers. Each server can include a `headers` object for authentication. Keep files that contain credentials private. Start the fixture server before checking the harness. > **Warning** > > External tool writes are real. Use read-only tasks or a server where writes are safe. The > controlled profile limits available tools, but it does not turn a write tool into a simulation. > External targets report isolation as `unverifiable`. ## Check and connect the harness 1. Check the installed runtime, authentication, and MCP discovery without starting a model turn: **`Check Claude Code`** ```bash title="Check Claude Code" runtype harness doctor --harness claude --backend native --mcp-config ./mcp.json ``` For Codex, replace `--harness claude` with `--harness codex`. A successful check prints `ready: true`. This confirms local credential detection and MCP discovery. It does not verify credential expiry, access to the requested model, or the model's ability to call tools. 2. Start the connection: **`Connect Claude Code`** ```bash title="Connect Claude Code" runtype harness connect \ --harness claude \ --backend native \ --profile controlled-mcp \ --mcp-config ./mcp.json \ --data-policy redacted \ --name "Read-only MCP harness" ``` Keep this command running. It prints the local registration ID and the paired device, Agent, and target IDs. Save the registration ID for status and cleanup commands. Use `--model MODEL_ID` to request a model supported by that installed harness and account. Omitting the option uses the harness's default model. To limit each session, set `--max-turns` from 1 to 100, with a default of 20. Set `--timeout-ms` from 1000 to 300000 to limit a turn, with a default of 300000. The host selects a free loopback port unless you set `--port`. 3. Use the printed target ID in a simulation variant. Start with one trial and read-only tasks that require the configured MCP tools. Set a low simulation spend limit for the Runtype work. 4. Open the report after the run completes. Inspect **Harness capture** for each turn and **Open trace** for its execution. Verify successful tasks, expected tool calls, and captured results. A completed run or accepted telemetry alone does not prove that the harness used a tool. Unknown outcomes remain unknown. ## Use Claude Code through ACP Install the pinned adapter into a directory you manage outside a project workspace. The adapter is a separate process that translates ACP requests into Claude Agent SDK calls. **`Install the pinned ACP adapter`** ```bash title="Install the pinned ACP adapter" mkdir -p "$HOME/runtype-adapters/claude-acp" pnpm --dir "$HOME/runtype-adapters/claude-acp" add --save-exact @agentclientprotocol/claude-agent-acp@0.99.1 ``` Set `ANTHROPIC_API_KEY`, `ANTHROPIC_AUTH_TOKEN`, or `CLAUDE_CODE_OAUTH_TOKEN` in the local environment. This backend uses an isolated configuration directory. It does not read a local Claude login file or execute `apiKeyHelper`. **`Check the ACP connection`** ```bash title="Check the ACP connection" runtype harness doctor \ --harness claude \ --backend acp \ --adapter "$HOME/runtype-adapters/claude-acp/node_modules/.bin/claude-agent-acp" \ --executable /absolute/path/to/claude \ --mcp-config ./mcp.json ``` Use the same options with `runtype harness connect`. The backend denies permission requests outside the controlled MCP profile. The SDK aggregate can include adapter helper requests, such as title generation, without exposing each request. ACP usage coverage remains partial because those requests are not itemized. ## Choose the capture policy The default `--data-policy redacted` retains allowed content after secret and personal-data redaction. `metrics` omits message, tool-input, tool-output, and native-event content. `full` retains allowed content while still scrubbing known secrets. The stricter host or organization policy wins. A host configured for `full` cannot override an organization policy that permits only metrics. Coverage in the report describes the retained evidence, so missing content is not presented as complete capture. Tool evidence includes stable call identity, names, available input and output, failure details, and known timing. Missing outcomes or durations remain unknown. Coverage distinguishes available, unavailable, disabled, partial, redacted, and truncated fields. **Delivery accepted** confirms that submitted batches were accepted. It does not mean that the backend exposed every field. Native observations are bounded to 128 KiB per event and 2 MiB per turn. Upload batches are bounded to 512 KiB. The local retry spool is bounded to 16 MiB and expires files older than 24 hours when initialized. Upload retries and final delivery waits are bounded. A failed upload can leave partial evidence and retained spool files. Restarting does not guarantee replay of an expired turn binding. Reports include delivered, observed, and dropped event counts and unresolved tools. The report-wide capture summary keeps at most 200 turns and 256 KiB of turn summaries. It states how many summaries were omitted. Inspect individual conversations for per-turn evidence. ## Interpret harness costs **Cost per success** can display a **Harness estimate** when platform-measured cost is unavailable. This requires complete reported cost and at least one successful conversation. Partial reporting shows a known subtotal and coverage instead. Missing usage remains unknown. The estimate describes the harness's reported scope and pricing basis. It is not a subscription charge or a provider invoice. Reported harness costs do not affect platform billing or simulation spend caps. A simulation spend cap does not cap external provider charges or subscription use. CSV and Markdown exports preserve the estimate label and completeness. ## Check status and clean up Inspect the registration from the same API, organization, and user: **`Inspect the connection`** ```bash title="Inspect the connection" runtype harness status YOUR_REGISTRATION_ID --json ``` Status includes whether the local host is online and its telemetry delivery counters. To stop the host and listener while keeping the registration: **`Disconnect the harness`** ```bash title="Disconnect the harness" runtype harness disconnect YOUR_REGISTRATION_ID ``` The command waits for local teardown and registration-lock release. Successful shutdown reports `phase: stopped`. If cleanup fails, inspect the reported stage before reconnecting or removing the registration. Resume the stored configuration with: **`Resume the harness`** ```bash title="Resume the harness" runtype harness connect --registration YOUR_REGISTRATION_ID ``` Resuming restores the registration, not an in-flight provider conversation. Start a new simulation run after an interruption. After disconnecting, remove the registered resources with: **`Remove the registration`** ```bash title="Remove the registration" runtype harness remove YOUR_REGISTRATION_ID --yes ``` Removal deletes the owned target and Agent and revokes the paired device. The private local registration record and retained telemetry files remain under the Runtype configuration directory. Remove those local files separately after you no longer need diagnostics. ## Troubleshoot a connection * If `doctor` rejects a version, install the exact compatible version and pass its executable path. Do not bypass the version check. * If pairing or a run returns a permission error, check the key's effective scopes and environment. A test key is insufficient for this workflow. * If a turn refuses its telemetry binding, check that the API supports harness telemetry and that the key, device, Agent, and simulation belong to the same authorized organization and device owner. * If the device disconnects, later turns fail with a device-unavailable error. Review partial evidence before reconnecting and starting a new run. * If the Agent's protocol, transport, or device changes during a run, the run refuses later turns. Restore the registration or create a new one before rerunning. ## Next steps * [Bring your own agent](/developer-guides/bring-your-own-agent): connect other external Agents. * [Report telemetry from an external agent](/developer-guides/reporting-external-telemetry): inspect the telemetry contract. * [Design a tool surface with simulations](/developer-guides/design-tool-surface-in-simulation): define tasks and compare variants.