ax run
Inspect and manage runs created by ax experiment run.
| Command | Use |
|---|---|
run list | List local and platform runs. |
run view | View a run request roll-up or one run in detail. |
run watch | Poll a run request until it finishes. |
run query | SQL scoped to one run. |
run rerun | Rerun failed variants, or re-run tests only. |
run cancel | Cancel queued or in-flight work. |
run cleanup | Run a previously skipped or failed teardown. |
run upload | Upload a local run to the platform. |
run clean | Remove orphaned local sandbox containers. |
Two id shapes appear below: a run request id covers one ax experiment run
submission, and a composite run id (<variant_id>::<hash>) names one run
inside it. Platform-backed commands also accept --org <org-id>, and all but
run cancel, run cleanup, and run upload accept --json; neither flag is
repeated in the tables. Arguments marked required must be
supplied; everything else is optional, and flags combine freely unless a row
says otherwise.
run list
List runs, one row per run, grouped by status and newest first.
ax run list| Argument / flag | What it does |
|---|---|
--experiment <id-name-or-url> | Scope to one experiment by id, exact display name, or details-page URL. Omit to list runs across the org. |
--group-by <dimension> | Group rows by status (default), experiment, variant, agent, product, environment, or prompt. |
--agent / --model / --variant / --product / --environment / --prompt | Dimension filters (repeatable, comma-separated). |
--status <status> | Filter by queued, waiting-for-ci, running, completed, errored, canceled. failed is a deprecated alias for errored. Default shows everything except canceled. |
--triggered-by <user-id> | Runs triggered by these users (platform runs only). |
--agent-test-aware | Only runs classified as "test aware": the agent noticed it was being tested, or changed its behavior because of it (platform runs only). --test-aware is a deprecated alias. |
--since <RFC3339> | Status last updated at or after this time. |
--experiment-version <v> | latest (default) or a version fingerprint (platform runs only). |
--scope <scope> | local (.axp/runs/, no network), remote, or both (default). |
--limit <n> | Max remote runs (default 50, platform cap 500). |
--output-root <dir> | Where to read local runs from (default .). |
That classification lands minutes after a run finishes, so --agent-test-aware
never matches a run that is still unclassified, and there is no filter for the
opposite verdict. Seeing a classification does not require the flag: whenever any
listed run carries one, the table gains a trailing AGENT TEST AWARENESS
column, and piped output gains three matching columns. --json rows always
carry test_aware, behavior_changed, and awareness_cause, each null until
the run is classified.
Examples:
# Errored runs for one experiment
ax run list --experiment my-experiment --status errored
# Local runs only, grouped by variant
ax run list --scope local --group-by variantrun view
View a run request roll-up, or one run in detail with its tests or conversation.
ax run view <RUN_REQUEST_ID | RUN_ID>| Argument / flag | What it does |
|---|---|
<RUN_REQUEST_ID | RUN_ID> | Required. Request id for the roll-up, or composite run id for one run. |
--tests | Per-test summary table (name, status, exit, duration). Not with --test. |
--test <name> | Stdout/stderr detail for one test. Composite run ids only. |
--transcript | The run conversation. Composite run ids only. Not with --tests or --test. |
--include reasoning | Add reasoning to a transcript. Dialogue is shown by default. Only with --transcript. |
--limit <n> | Maximum transcript turns (default 500, maximum 10,000). Only with --transcript. |
--scope <scope> | Resolve composite ids from local, remote, or both (default). |
--web | Open the run page in a browser. |
--output-root <dir> | Where to read local runs from (default .). |
Transcript output adapts to stdout. An interactive terminal shows the run
summary and a table. A pipe receives headerless TSV in seq, ts, role,
kind, text order, with the run summary and completeness notices on stderr.
Pass --json for one structured run document with transcript.turns.
Transcripts currently require a platform run.
Run status and test outcome are separate signals: a run whose agent finished
but whose tests failed is still completed (with a non-green tests line);
only infrastructure failures are errored. For request roll-ups the command
exits non-zero when any run ended unsuccessfully (setup failure, timeout,
cancel); test failures alone do not change the exit code.
When a platform run contains evidence that the agent is aware it is being
tested, it is classified as "test aware" and an agent test awareness row
appears with the cause and evidence excerpts. Unclassified runs, and runs
where the agent did not notice, look unchanged.
Examples:
# Roll-up for a whole submission
ax run view 01KSDQ...
# One run's failing test, with output
ax run view 'claude::p0::abc123' --test report-exists
# One run's dialogue as structured output
ax run view 'claude::p0::abc123' --transcript --json
# Dialogue and reasoning with a raised turn limit
ax run view 'claude::p0::abc123' --transcript --include reasoning --limit 1000run watch
Poll a run request until it reaches a terminal status.
ax run watch <RUN_REQUEST_ID>run query
Run read-only SQL scoped to one run: against uploaded platform data, or a
local run with --local. Platform test scripts use the separate test-only
ax-run-query command; authentication is supplied automatically and limited
to that run.
ax run query <RUN_ID> sql "SELECT * FROM test_analysis"| Argument / flag | What it does |
|---|---|
<RUN_ID> | Required. Composite run id, as printed by ax run list. |
sql <SQL|@file|-> | SQL to run: inline text, @file.sql, or - for stdin. |
--tables [<NAME>] | List queryable tables, or describe one table. --table is an alias. |
--local | Query the local run with DuckDB instead of platform ClickHouse. |
--limit <n> | Max rows for platform queries (default 100, server cap 10000). |
--format json|table | Force NDJSON or an ASCII table. Defaults to table on a TTY and json when piped. |
--dry-run | Validate without executing. |
--save <heading> | Save the query as an insight. --slug, --description, --label, and --force only apply together with it. |
--output-root <dir> | Where to read local runs from. Only with --local. |
Examples:
# What is queryable
ax run query 'claude::p0::abc123' sql --tables
# Failing tests for one run
ax run query 'claude::p0::abc123' sql \
"SELECT test_name, exit_code FROM test_analysis WHERE exit_code != 0" --format table
# Conversational evidence inside a platform test (stdout and/or transcript)
ax-run-query "$AX_RUN_ID" sql \
"SELECT count() AS matches FROM events WHERE source IN ('stdout', 'transcript')"
# Same run, offline
ax run query 'claude::p0::abc123' sql "SELECT * FROM runs" --localSee Tests for a complete transcript assertion.
run rerun
Rerun failed or canceled variants from a prior platform request, or re-run
tests only against completed runs with --tests (see Tests).
ax run rerun <RUN_REQUEST_ID>
ax run rerun <RUN_ID> --tests --experiment experiment.yaml| Argument / flag | What it does |
|---|---|
<RUN_OR_REQUEST_ID>... | Required. Request id(s) to rerun, or run id(s) to retest with --tests. |
--tests | Re-run tests only; the agent is not driven again. |
--experiment <path> | Use this edited YAML instead of the latest registered version. |
--variant <id> | Limit which variants are retested (exact ids, repeatable). |
--flag <name> | Override inherited run-request flags. Not with --tests. |
--local | Retest a local .axp/runs group. Only with --tests. |
--run <id> / --env <NAME[=VALUE]> / --env-file <path> / --output-root <dir> | Local retest targeting and secrets. Only with --tests --local. |
Remote test-only reruns execute against each run's captured sandbox snapshot;
if the snapshot was garbage-collected or never captured, the submit is
rejected and you should create new runs instead. Tests that query transcript
events with $AX_RUN_ID receive the same run-scoped access during a retest.
User-triggered test-only reruns do not run the environment's teardown again.
run cancel
Cancel all jobs in a run request, or selected runs within it.
ax run cancel <RUN_REQUEST_ID>| Argument / flag | What it does |
|---|---|
<RUN_REQUEST_ID> | Required. Request id, as printed by ax experiment run. |
--run <id,...> | Cancel only these run ids; default cancels all. |
run cleanup
Run a previously skipped or failed declared teardown for one platform run.
ax run cleanup <RUN_ID>| Argument / flag | What it does |
|---|---|
<RUN_ID> | Required. Composite run id, as printed by ax run list. |
Create a retained platform run with ax experiment run <EXPERIMENT> --no-teardown.
When that run finishes, the CLI prints its exact cleanup command. Cleanup runs
the declared teardown only. It does not repeat setup, agent execution, or tests,
and exits non-zero if teardown fails.
run cleanup operates on the retained platform snapshot for one run. It is
different from run clean, which finds orphaned local sandbox
containers.
run upload
Upload a local run to the platform so it appears alongside platform runs.
ax run upload [RUN_ID]| Argument / flag | What it does |
|---|---|
[RUN_ID] | Run id. Omit to use the most recent completed local group. |
--force | Retry an existing upload of the same run. |
run clean
List orphaned local sandbox containers, and remove them with --yes.
ax run clean [--yes]| Argument / flag | What it does |
|---|---|
--yes | Actually remove. Without it, only lists the orphans. |