ax agent

These commands require authentication and an active organization. All accept --org <ORG_ID> to target another organization for one command. agent list, agent test, and agent version accept --json for machine-readable output. Arguments marked required must be supplied.

The ax agent validate command was removed. Run ax experiment validate <path> instead; it applies the same agent-version and compatibility checks as submit.

Compatibility status

Compatibility is tracked for an exact agent version and canonical model combination. The status field has three values:

A combination is supported only when its status is verified and the model is on the agent's model menu, the catalog models that agent offers. Listing, validation, and submission all apply this one rule: agent list and ax model list list only supported combinations, ax experiment validate and submission run only supported combinations and skip every other one with a warning, and a definition with nothing left to run is rejected. An untested combination, unknown or with no compatibility entry yet, can be tested with agent test. A model absent from the canonical catalog has no compatibility entry and is never supported.

Only unknown changes automatically. Its first conclusive outcome, from agent test or the platform's own compatibility checks, sets the combination to verified or unsupported; later runs do not replace that status.

In agent version JSON output, each compatibility entry includes the canonical model id, status, and an optional reason, for every model the registry tracks for that version. agent list includes only the supported models, so every status is verified, and omits the reason:

{
  "compatibility": [
    {
      "canonical_model_id": "anthropic/claude-opus-4-8",
      "status": "verified",
      "reason": null
    }
  ]
}

agent list

List agents and their latest supported versions.

ax agent list [AGENT] [--all-versions] [--model <MODEL>]

A version supports a model only when the combination is supported: verified and on the agent's model menu. This is the rule validation and submission use, and the same rule as ax model list. The latest supported version is the newest stable supporting version, which is the version an experiment without a version or saved pin runs for that model alone. An agent with no supporting version is left out.

A pre-release, such as Codex 0.161.0-alpha.9, is never the latest version. It runs only when an experiment pins that exact version, and a saved pin to one is kept. --all-versions lists pre-releases with (pre-release).

$ ax agent list --model anthropic/claude-haiku-4-5 --all-versions

Supported agents and versions for anthropic/claude-haiku-4-5:

AGENT   VERSION
claude  2.1.283 (latest)
cursor  2026.09.26-dd393fe (latest)
cursor  2026.09.18

--json lists the same rows as the table. Each version's compatibility lists only its supported models, without reasons; agent version shows every status. To get every registered version, as --json returned before --all-versions existed, add --all-versions.

agent test

Test whether an exact agent version works with a model.

ax agent test <AGENT>@<VERSION> --model <MODEL>

The command runs a fixed check: one short turn in which the agent is asked to reply to the model. You choose only the agent version and model. The check runs at Fiveonefour's expense and is not billed to your organization. The agent itself answers, so the check also runs for a model that experiment validation does not offer for that agent.

The answer is saved for everyone and sets the pair's compatibility status: verified when the model replied, unsupported when the agent rejected the model. Later ax model list, ax agent list, experiment validation, and submission use it: a verified pair on the agent's model menu is then listed and runs. A pair that already has a saved result answers at once without a new check, and a pair already being checked joins that check.

$ ax agent test cursor@2026.09.26-dd393fe --model anthropic/claude-haiku-4-5
Running a compatibility test...

MODEL                       AGENTS SUPPORTED
anthropic/claude-haiku-4-5  Cursor (latest)

Support result saved for this agent version and model.

When the agent rejects the model, the row reads None. Other failures save nothing and leave support unknown. The detail explains the failure and whether retrying could help. Provider HTTP 500, model-specific 4xx, and unavailable models do not suggest retrying. Authentication failures need credentials fixed first. Network errors, timeouts, and HTTP 408/429/502/503/504/529 can suggest retrying later. Unknown failures retain retry guidance. The human output prints Retry: only when retrying could help; an older server without retryability information retains that hint.

For an undetermined JSON result, retryable tells you whether repeating the check could help, and failure_class uses these full names:

The command exits 0 when the pair works, 3 when it does not, and 4 when the check could not determine either; other errors exit 1. Starting checks is rate limited. Repeated provider errors do not turn the pair into saved unsupported support.

agent version

Inspect one exact agent version, its artifact digests, and model compatibility.

ax agent version <AGENT> <VERSION>