Search documentation

Search the Fumadocs-backed documentation index.

ax run

Inspect and manage runs created by ax experiment run.

CommandUse
run listList local and platform runs.
run viewView a run request roll-up or one run in detail.
run watchPoll a run request until it finishes.
run querySQL scoped to one run.
run rerunRerun failed variants, or re-run tests only.
run cancelCancel queued or in-flight work.
run cleanupRun a previously skipped or failed teardown.
run uploadUpload a local run to the platform.
run cleanRemove orphaned local sandbox containers.

Two id shapes appear below: a run request id covers one ax experiment run submission, and a composite run id (<variant_id>::<hash>) names one run inside it. Platform-backed commands also accept --org <org-id>, and all but run cancel, run cleanup, and run upload accept --json; neither flag is repeated in the tables. Arguments marked required must be supplied; everything else is optional, and flags combine freely unless a row says otherwise.

run list

List runs, one row per run, grouped by status and newest first.

ax run list
Argument / flagWhat it does
--experiment <id-name-or-url>Scope to one experiment by id, exact display name, or details-page URL. Omit to list runs across the org.
--group-by <dimension>Group rows by status (default), experiment, variant, agent, product, environment, or prompt.
--agent / --model / --variant / --product / --environment / --promptDimension filters (repeatable, comma-separated).
--status <status>Filter by queued, waiting-for-ci, running, completed, errored, canceled. failed is a deprecated alias for errored. Default shows everything except canceled.
--triggered-by <user-id>Runs triggered by these users (platform runs only).
--agent-test-awareOnly runs classified as "test aware": the agent noticed it was being tested, or changed its behavior because of it (platform runs only). --test-aware is a deprecated alias.
--since <RFC3339>Status last updated at or after this time.
--experiment-version <v>latest (default) or a version fingerprint (platform runs only).
--scope <scope>local (.axp/runs/, no network), remote, or both (default).
--limit <n>Max remote runs (default 50, platform cap 500).
--output-root <dir>Where to read local runs from (default .).

That classification lands minutes after a run finishes, so --agent-test-aware never matches a run that is still unclassified, and there is no filter for the opposite verdict. Seeing a classification does not require the flag: whenever any listed run carries one, the table gains a trailing AGENT TEST AWARENESS column, and piped output gains three matching columns. --json rows always carry test_aware, behavior_changed, and awareness_cause, each null until the run is classified.

Examples:

# Errored runs for one experiment
ax run list --experiment my-experiment --status errored

# Local runs only, grouped by variant
ax run list --scope local --group-by variant

run view

View a run request roll-up, or one run in detail with its tests or conversation.

ax run view <RUN_REQUEST_ID | RUN_ID>
Argument / flagWhat it does
<RUN_REQUEST_ID | RUN_ID>Required. Request id for the roll-up, or composite run id for one run.
--testsPer-test summary table (name, status, exit, duration). Not with --test.
--test <name>Stdout/stderr detail for one test. Composite run ids only.
--transcriptThe run conversation. Composite run ids only. Not with --tests or --test.
--include reasoningAdd reasoning to a transcript. Dialogue is shown by default. Only with --transcript.
--limit <n>Maximum transcript turns (default 500, maximum 10,000). Only with --transcript.
--scope <scope>Resolve composite ids from local, remote, or both (default).
--webOpen the run page in a browser.
--output-root <dir>Where to read local runs from (default .).

Transcript output adapts to stdout. An interactive terminal shows the run summary and a table. A pipe receives headerless TSV in seq, ts, role, kind, text order, with the run summary and completeness notices on stderr. Pass --json for one structured run document with transcript.turns. Transcripts currently require a platform run.

Run status and test outcome are separate signals: a run whose agent finished but whose tests failed is still completed (with a non-green tests line); only infrastructure failures are errored. For request roll-ups the command exits non-zero when any run ended unsuccessfully (setup failure, timeout, cancel); test failures alone do not change the exit code.

When a platform run contains evidence that the agent is aware it is being tested, it is classified as "test aware" and an agent test awareness row appears with the cause and evidence excerpts. Unclassified runs, and runs where the agent did not notice, look unchanged.

Examples:

# Roll-up for a whole submission
ax run view 01KSDQ...

# One run's failing test, with output
ax run view 'claude::p0::abc123' --test report-exists

# One run's dialogue as structured output
ax run view 'claude::p0::abc123' --transcript --json

# Dialogue and reasoning with a raised turn limit
ax run view 'claude::p0::abc123' --transcript --include reasoning --limit 1000

run watch

Poll a run request until it reaches a terminal status.

ax run watch <RUN_REQUEST_ID>

run query

Run read-only SQL scoped to one run: against uploaded platform data, or a local run with --local. Platform test scripts use the separate test-only ax-run-query command; authentication is supplied automatically and limited to that run.

ax run query <RUN_ID> sql "SELECT * FROM test_analysis"
Argument / flagWhat it does
<RUN_ID>Required. Composite run id, as printed by ax run list.
sql <SQL|@file|->SQL to run: inline text, @file.sql, or - for stdin.
--tables [<NAME>]List queryable tables, or describe one table. --table is an alias.
--localQuery the local run with DuckDB instead of platform ClickHouse.
--limit <n>Max rows for platform queries (default 100, server cap 10000).
--format json|tableForce NDJSON or an ASCII table. Defaults to table on a TTY and json when piped.
--dry-runValidate without executing.
--save <heading>Save the query as an insight. --slug, --description, --label, and --force only apply together with it.
--output-root <dir>Where to read local runs from. Only with --local.

Examples:

# What is queryable
ax run query 'claude::p0::abc123' sql --tables

# Failing tests for one run
ax run query 'claude::p0::abc123' sql \
  "SELECT test_name, exit_code FROM test_analysis WHERE exit_code != 0" --format table

# Conversational evidence inside a platform test (stdout and/or transcript)
ax-run-query "$AX_RUN_ID" sql \
  "SELECT count() AS matches FROM events WHERE source IN ('stdout', 'transcript')"

# Same run, offline
ax run query 'claude::p0::abc123' sql "SELECT * FROM runs" --local

See Tests for a complete transcript assertion.

run rerun

Rerun failed or canceled variants from a prior platform request, or re-run tests only against completed runs with --tests (see Tests).

ax run rerun <RUN_REQUEST_ID>
ax run rerun <RUN_ID> --tests --experiment experiment.yaml
Argument / flagWhat it does
<RUN_OR_REQUEST_ID>...Required. Request id(s) to rerun, or run id(s) to retest with --tests.
--testsRe-run tests only; the agent is not driven again.
--experiment <path>Use this edited YAML instead of the latest registered version.
--variant <id>Limit which variants are retested (exact ids, repeatable).
--flag <name>Override inherited run-request flags. Not with --tests.
--localRetest a local .axp/runs group. Only with --tests.
--run <id> / --env <NAME[=VALUE]> / --env-file <path> / --output-root <dir>Local retest targeting and secrets. Only with --tests --local.

Remote test-only reruns execute against each run's captured sandbox snapshot; if the snapshot was garbage-collected or never captured, the submit is rejected and you should create new runs instead. Tests that query transcript events with $AX_RUN_ID receive the same run-scoped access during a retest. User-triggered test-only reruns do not run the environment's teardown again.

run cancel

Cancel all jobs in a run request, or selected runs within it.

ax run cancel <RUN_REQUEST_ID>
Argument / flagWhat it does
<RUN_REQUEST_ID>Required. Request id, as printed by ax experiment run.
--run <id,...>Cancel only these run ids; default cancels all.

run cleanup

Run a previously skipped or failed declared teardown for one platform run.

ax run cleanup <RUN_ID>
Argument / flagWhat it does
<RUN_ID>Required. Composite run id, as printed by ax run list.

Create a retained platform run with ax experiment run <EXPERIMENT> --no-teardown. When that run finishes, the CLI prints its exact cleanup command. Cleanup runs the declared teardown only. It does not repeat setup, agent execution, or tests, and exits non-zero if teardown fails.

run cleanup operates on the retained platform snapshot for one run. It is different from run clean, which finds orphaned local sandbox containers.

run upload

Upload a local run to the platform so it appears alongside platform runs.

ax run upload [RUN_ID]
Argument / flagWhat it does
[RUN_ID]Run id. Omit to use the most recent completed local group.
--forceRetry an existing upload of the same run.

run clean

List orphaned local sandbox containers, and remove them with --yes.

ax run clean [--yes]
Argument / flagWhat it does
--yesActually remove. Without it, only lists the orphans.