Agent mode
Run agents normally (build) or request their planning mode (plan).
agent_mode is a top-level variant axis. Accepts a single value or list of build / plan. It is cross-multiplied with the other axes just like agents and models to define the variants tested.
build(default) runs the agent normally: it executes tools, can read and write, and mutates the workspace.planrequests the agent's native read-only / planning mode. Claude and Codex remain read-only. Cursor starts in its native plan mode, but its headless CLI can request a transition to agent mode and approve that transition without a user present. After transitioning, Cursor can mutate the workspace even though the variant remains labeledplan.
Cursor plan is not currently a strict no-mutation guarantee. Do not use Cursor
plan variants when the evaluation requires enforced read-only behavior. If you
still include them, add tests that fail on file changes or mutating tool calls
and treat any mode transition as a failed plan run.
# Sweep both modes over one agent/model/prompt → 2 variants.
agents: [claude]
models: [claude-opus-4-8]
agent_mode: [build, plan]agent_mode is optional; it defaults to build. When you declare the axis, the mode is emitted into the variant id. This applies to plan and build options. So a [build, plan] sweep produces …::build::my-prompt and …::plan::my-prompt. When the axis is absent nothing is emitted, so experiments that don't use agent_mode keep their original ids (for example claude::anthropic-claude-opus-4-8::my-prompt). Note that authoring agent_mode: build explicitly therefore yields a different id than omitting the axis.
Every test in the flat tests list runs for every variant regardless of mode; the harness does not scope tests by agent_mode. For agents with enforced read-only plan mode, a build-oriented workspace check will fail the plan variant unless you guard it (e.g. [ ! -f out.json ] || <verify out.json>). Analyze plan quality from session data after the run, and split build and plan into separate experiments when they need different test outcomes. For Cursor, also check that the workspace was not changed and no mode transition was approved.