Models

models defines which models run, as an independent axis cross-multiplied with agents. Listing agents: [claude] with models: [opus, sonnet] produces both variants without duplicating the agent.

Accepted shapes:

# String form: one model.
models: claude-opus-4-8

# List of strings: multiple models.
models:
  - claude-opus-4-8
  - claude-sonnet-4-6

# Object form: a model with controls.
models:
  - name: gpt-5.3-codex
    effort: high

models is optional. If omitted, each agent runs its provider-default model.

Find supported models

Which models an agent can run depends on its exact version. Check support with the live listing before you write models, then validate the file:

# Models each agent's latest version supports
ax model list

# Models Cursor's latest version supports, then one exact version
ax model list --agent cursor
ax model list --agent cursor@<VERSION>

# The latest agent versions that support specific models
ax model list --model claude-opus-4-8 --model gpt-5.3-codex

# Every claude version that supports a model
ax model list --agent claude --all-versions --model claude-opus-4-8

# Every version of each agent that supports one model
ax agent list --model claude-opus-4-8 --all-versions

# Whether one exact agent version works with a model
ax agent test cursor@<VERSION> --model claude-opus-4-8

ax agent test needs an exact registered version and accepts a model id or catalog alias. It exits 0 when the pairing works and 3 when it does not, and saves that answer for everyone, so ax model list, ax agent list, and validation use it from then on. A pairing that already has a saved answer returns it at once. When the test cannot tell, for example because of a network or infrastructure error, it exits 4 and saves nothing.

A pairing is supported only when the platform has verified it (verified) and the model is on the agent's model menu. Listing, validation, and submission all use this rule: a listed pairing validates and runs, and a pairing that is not listed is skipped. Pairings that have not been tested yet are not listed and do not run until ax agent test verifies them. An agent listed without a version runs its saved pin when the experiment has one; otherwise it runs the stable version that supports the most of the models paired with it (the newest on a tie), and its other models are skipped. A pre-release runs only when pinned. To run a specific version, pin it on the agent, such as agents: [{ name: claude, version: <VERSION> }].

If no single version supports every model, AX uses the newest stable version that supports the most of them and skips the rest, with a hint naming the version that supports each skipped model. ax experiment validate lists the pairings submit will skip, grouped by agent and exact version, with an ax agent test <AGENT>@<VERSION> --model <MODEL> command for each untested pairing, that hint for each skipped model another version supports, and an ax agent list --model <MODEL> --all-versions command for each skipped model. Skipped pairings are a warning, not a failure; validation fails only when no pairing is left to run.

Pruning incompatible pairs

The agents × models cross product keeps only the compatible pairs. A model the catalog knows for a different agent's provider — for example claude × an openai model — is pruned (dropped), not an error, so one agents + models pair of lists expresses every valid combination:

agents: [claude, codex]
models: [claude-opus-4-8, gpt-5.3-codex]
# 2 × 2 = 4 pairs → pruned to the 2 valid ones:
#   ✓ claude + claude-opus-4-8     ✓ codex + gpt-5.3-codex
#   ✗ claude + gpt-5.3-codex       ✗ codex + claude-opus-4-8   (dropped)

Pruning is visible — dropped pairs show up as a smaller resolved matrix in ax experiment variants. If a listed agent or model has no pair left (every pair it appears in is pruned, because the agent does not offer the model), its pairings are skipped like any other pairing that cannot run: ax experiment validate lists them in the skipped-pairings warning, names the agents that offer each model, and still exits zero. Validation fails only when no pairing in the experiment is left to run.

Model id format

Prefer the lab-official models.dev id with no owner prefix, such as claude-opus-4-8, gpt-5.3-codex, or composer-2.5. Fiveonefour infers the runtime provider from the selected agent and stores a canonical provider/lab/model id behind the scenes.

If a bare id is ambiguous, validation asks for lab; use the model object form, for example { name: claude-opus-4-8, lab: anthropic }. Most model ids do not need this.

Older owner/provider-prefixed and gateway-style ids, such as anthropic/claude-opus-4-8 or anthropic/claude-opus-4.8, are accepted for compatibility and normalize to the same canonical id.

Model object

A model object configures one model and its controls. Optional controls are passed through when the selected agent supports them. To compare two control settings of the same model, list two models entries.

agents: [claude, codex]
models:
  - name: claude-opus-4-8
    effort: high
    context_window_size: 1M
    thinking: true
  - gpt-5

Reasoning summaries on Codex

Codex never exposes its raw reasoning; the model returns it encrypted. The thinking you see in a Codex trace comes from reasoning summaries, and OpenAI returns those on a best-effort basis for each reasoning step. Brief reasoning often gets no summary, and minimal effort returns none.

If your experiment needs thinking evidence from Codex, set the Codex model's effort to medium or higher and reasoning_summary to detailed:

agents: [codex]
models:
  - name: gpt-5.3-codex
    effort: high
    reasoning_summary: detailed

Summaries are requested only on runs with managed model access. A run that uses your own OpenAI key requests none, even when a model sets reasoning_summary, and ax experiment run warns that Codex thinking will not be captured.

Supported models

Common catalog model IDs that at least one shipped agent can use (claude → Anthropic, codex → OpenAI, cursor → Cursor catalog). Media-only and non-tool entries are omitted; duplicate provider rows collapse to one lab + bare model. Unknown or agent-native ids can still work; this list is a convenience reference, not a gate. It does not track agent versions: run ax model list for the models each agent version supports.