Troubleshooting

This page covers failures that the current CLI can produce. For command syntax, compare against ax --help and the command-specific --help output.

axp: command not found after install

The installer writes the wrapper to ~/.fiveonefour/bin/ax by default. If that directory is not already on PATH, the installer prints the export to add and, for writable zsh or bash config directories, appends an installer-managed entry.

Open a new shell, reload your shell config, or add the path manually:

export PATH="$HOME/.fiveonefour/bin:$PATH"
ax --help

If you installed with FIVEONEFOUR_HOME=<path>, use <path>/bin instead.

Install download fails

The installer requires curl and downloads the version manifest, binary, checksum, and optional signature from https://download.514.ax with curl -fsSL. If the manifest download fails, the script prints the failed manifest URL and the underlying curl error.

Check that curl is installed and that your network, proxy, or firewall can reach the download host:

curl -fsSL https://download.514.ax/stable/index.json >/dev/null

Then rerun the installer. If the binary or checksum download fails after the manifest succeeds, retry the install and keep the printed download URL for debugging.

Unsupported OS or architecture

The installer has prebuilt targets for:

• macOS arm64 and x86_64
• Linux arm64/aarch64 and x86_64/amd64

On any other uname -s and uname -m pair, it exits with unsupported platform: <os>/<arch>. Use one of the supported platforms for the hosted installer.

Uninstall leaves a PATH entry

By default, uninstall removes the AXP wrapper and cached versions but leaves shell startup files untouched. To remove exact installer-managed PATH entries from ~/.zshrc, ~/.bashrc, ~/.bash_profile, and ~/.profile, pass --remove-path:

bash <(curl -fsSL https://download.514.ax/uninstall.sh) --yes --remove-path

The script only removes lines that exactly match the installer-managed ~/.fiveonefour/bin export, plus the adjacent # AXP CLI comment when present. Remove custom PATH edits manually.

Uninstall skips the ax binary

The uninstall script only removes ~/.fiveonefour/bin/ax automatically when it looks like the hosted installer wrapper. If that file is a custom binary, symlink, or a Cargo-installed ax, uninstall prints that it was skipped.

Pass --force only when you intentionally want to remove that file anyway:

bash <(curl -fsSL https://download.514.ax/uninstall.sh) --yes --force

Cargo-installed binaries outside ~/.fiveonefour/bin are not removed by the hosted uninstall script.

Non-interactive uninstall fails

Interactive uninstall prompts before removing files. In a non-interactive shell, the script refuses to continue unless you pass --yes:

bash <(curl -fsSL https://download.514.ax/uninstall.sh) --yes

Combine it with other flags when needed, for example --yes --remove-path.

Preview uninstall changes

Use --dry-run to print what uninstall would remove or update without changing files. Dry runs do not require confirmation and print Dry run: no files will be changed.

bash <(curl -fsSL https://download.514.ax/uninstall.sh) --dry-run --remove-path

No command was provided

ax requires a subcommand. Running it with no command exits non-zero and prints usage.

ax --help

Experiment YAML does not validate

ax experiment validate <EXPERIMENT> prints Validated <EXPERIMENT> and a suggested ax experiment run <EXPERIMENT> command on success. On schema failure, it prints one error line and exits non-zero. It may also print non-fatal warning[<rule>] lint messages to stderr while still exiting zero.

Common schema issues include:

• schema_version is not 1.
• id, variant IDs, or test names are not kebab-case.
• name, flow.prompt, model, variant tags, or test scripts are empty.
• matrix.variant has no entries.
• tests has no entries.
• limits.max_turns, limits.max_time_seconds, or limits.max_cost_usd are zero or invalid.
• The YAML contains an unknown field.

max_cost_usd must be a positive number. AXP aborts the agent when reported cumulative cost reaches that value; if the agent does not report cost during the run, AXP cannot stop on cost. max_time_seconds is enforced by AXP, and max_turns is passed to the agent where supported.

Run validation before starting containers:

ax experiment validate experiment.yaml

If you see warning[polls-for-file-in-setup], check any variant overrides.setup loop that waits for a /workspace/... path. Variants are isolated, so each variant must create or fetch its own setup files.

A run used the wrong variant set

ax experiment run --variant <ID> only accepts IDs from matrix.variant. If a requested variant does not exist, the CLI prints the known variant IDs and exits with code 2.

Use a local dry run to inspect the planned variants without creating containers or making API calls:

ax experiment run experiment.yaml --local --resolve-variants

A run failed with setup_failed

A variant rolls up as setup_failed when its setup script exits non-zero — setup-checks, the agent, and tests never run. This is distinct from driver_error, which now means an actual harness/driver fault rather than a mundane setup bug.

The script's stdout + stderr (tail) is included in the ax experiment run failure output, and the full output is saved to the variant's setup.log artifact (fetch it with ax run download). A common cause is a missing command — e.g. installing docker.io without docker-cli, so a docker --version line exits 127.

Iterate on the setup script with a local run, which writes the same setup.log:

ax experiment run experiment.yaml --local

--repeat 0 fails

--repeat must be at least 1.

ax experiment run experiment.yaml --repeat 3

A referenced secret can't be resolved

An axp://secrets/<slug> reference resolves from your org's secret store on both run paths. If the slug doesn't exist in the org (and, for local runs, no local source supplies the bound name), the run fails fast at preflight before any container starts.

Store the value, then re-run:

ax secret set prod-gh        # prompts for the value
ax secret list               # confirm the slug exists (values are never shown)

For a local run you can also override the bound name from a local source instead of the store (lowest → highest precedence): auto-loaded ./.env, --env-file, host env, then --env:

echo "GITHUB_TOKEN=..." > .env                          # auto-loaded
ax experiment run experiment.yaml --local --env-file .env.local      # explicit file
GITHUB_TOKEN=... ax experiment run experiment.yaml --local           # host env
ax experiment run experiment.yaml --local --env GITHUB_TOKEN=...     # highest precedence

A name must match ^[A-Z_][A-Z0-9_]*$. Harness-managed names cannot be used, including ANTHROPIC_API_KEY, ANTHROPIC_BASE_URL, OPENAI_API_KEY, OPENAI_BASE_URL, CURSOR_API_KEY, MODEL, MAX_TURNS, IS_SANDBOX, TRACEPARENT, and names starting with AXP_, CLAUDE_CODE_, CODEX_, CURSOR_, or OTEL_.

Anthropic API key is missing

ax experiment run uses platform-supplied model access, so it needs no host model key. Local runs do: ax experiment run --local requires a model credential in the host environment for the selected agent — ANTHROPIC_API_KEY for Claude or OPENAI_API_KEY for Codex. These are harness-managed, so do not list them under environment_variables.

export ANTHROPIC_API_KEY=...
ax experiment run experiment.yaml --local

export OPENAI_API_KEY=...
ax experiment run codex-experiment.yaml --local

If you are using AXP-provided model access, make sure the CLI is logged in with ax auth login or has AX_API_KEY set. You do not need to set Anthropic or OpenAI API keys for that run.

No runs are listed

ax run list shows local runs from ./.axp/runs/ and, when signed in, remote platform runs. If no local runs exist in the current directory and no remote runs match, it prints an empty-list message.

Run a local experiment from the project directory that should own the results:

ax experiment run experiment.yaml --local
ax run list

ax run list shows no runs

Use ax run list --scope local to check only ./.axp/runs/. If it is empty, run a local experiment first. Inspect a run's artifacts directly under .axp/runs/<run-id>/, or query them with ax results query "<SQL>" --local <run-id>.

To inspect platform results locally, download the run first:

ax run download my-experiment-01HX...

Docker image pull fails

ax experiment run --local pulls axp-base from Docker Hub (514labs/axp-base) on first use. If the pull fails, check Docker is running and Docker Hub is reachable; you can pre-pull manually:

docker pull 514labs/axp-base:latest

Use ax experiment run experiment.yaml --local --resolve-variants when you only want experiment validation without touching containers. (Platform runs with ax experiment run need no local Docker.)

ax experiment run says you're not signed in

ax experiment run submits to the platform, so it needs saved credentials. If you see an authentication error, sign in (or set AXP_API_KEY in CI):

ax auth login
ax auth status

To run without the platform, use ax experiment run <path> --local (Docker plus your own model key).

ax experiment run and secrets

Platform runs support secrets: reference an org secret from environment_variables as axp://secrets/<slug> and dispatch resolves it server-side from your org's secret store. (Earlier releases rejected secret-bearing experiments at submit — that restriction is gone.) Store the value first with ax secret set <slug>; a reference to a slug that doesn't exist in the org fails at submit. The deprecated secrets: field remains local-only.

See Runs and Environment and Auth.

Cancel a platform run

ax experiment run prints a run request id. Cancel queued or in-flight jobs with it:

ax run cancel <run-request-id>

Cancel only specific jobs with --run <run-id> (comma-separated); the default cancels every job in the request.

Not logged in for platform upload

ax run upload requires credentials saved by ax auth login. If credentials are missing, run:

ax auth login
ax auth status

ax auth status round-trips through the platform API and prints the resolved user and active org. Use ax auth orgs to list available orgs, and ax auth orgs --switch org_... to change the active CLI org.

The credentials file lives at ~/.fiveonefour/axp/credentials.toml. AX_API_KEY can override the saved API key.

Upload target is wrong

Auth, upload, and download requests go to the AXP platform by default. If the AXP team gives you a different platform URL, set AX_PLATFORM_URL:

AXP_PLATFORM_URL=https://platform.example.com ax run upload my-experiment-01HX...

Active org is wrong

Platform commands use the active CLI org by default. Check it with:

ax auth status
ax auth orgs

Switch the active org with:

ax auth orgs --switch org_...

For a one-off upload or download without changing the active org, pass --org:

ax run upload my-experiment-01HX... --org org_...
ax run download my-experiment-01HX... --org org_...

If a run exists in another org that your API key can access, the platform error includes this hint. If the run is not in an accessible org, the error does not reveal where it exists.

Run was already uploaded

If the server reports that a run already exists, pass --force to overwrite that upload:

ax run upload my-experiment-01HX... --force

Download destination already exists

ax run download writes to .axp/downloads/<run-id>/ and refuses to replace an existing directory unless you pass --force:

ax run download my-experiment-01HX... --force

Local query derivation fails

Use ax results query --local to derive query files from a raw local run, or to read downloaded platform Parquet, and surface the conversion/query error:

ax results query "SELECT * FROM runs LIMIT 1" --local <run-id>

If the run is not local, use ax run list --scope local to find local runs or run ax run download <run-id> before querying locally.