Experiments

HTTP API routes for experiments.

RouteUseCLI equivalent
GET /experimentsList registered experiments.ax experiment list --json
POST /experimentsRegister YAML without running it.ax experiment push
GET /experiments/{id}Inspect one experiment.ax experiment view --json
GET /experiments/{id}/versionsList stored versions.ax experiment versions --json
GET /experiments/{id}/yamlDownload registered YAML.ax experiment pull
POST /experiments/{id}/runsSubmit a run.ax experiment run <id>
POST /experiments/{id}/queryQuery results in experiment scope.ax experiment query

Every route takes org_id as a query parameter (required only when the key can see several orgs; see Organization scope); it is not repeated in the tables below. Parameters are optional unless marked required.

GET /api/v1/experiments

List registered experiments in the selected org, newest first.

curl -H "Authorization: Bearer $AX_API_KEY" \
  "https://app.514.ax/api/v1/experiments?search=install&limit=20"
ParameterWhat it does
searchCase-insensitive match on experiment name or id.
ownerOwner email or id filter.
agent / model / product / environmentDimension filters (repeatable or comma-separated). Same axis ORs; different axes AND.
sinceRFC 3339 updated-at floor (updated_since is an alias).
limitMax experiments to return.

POST /api/v1/experiments

Register experiment YAML without running it. The body is the raw YAML, not JSON; each push stores a content-addressed version.

curl -X POST \
  -H "Authorization: Bearer $AX_API_KEY" \
  -H "Content-Type: text/yaml" \
  --data-binary @experiment.yaml \
  "https://app.514.ax/api/v1/experiments"

Responds with the pushed experiment version metadata. Empty bodies return 400 invalid_argument.

GET /api/v1/experiments/{id}

Inspect one registered experiment: metadata and resolved axes. URL-encode {id} if it contains reserved characters.

curl -H "Authorization: Bearer $AX_API_KEY" \
  "https://app.514.ax/api/v1/experiments/my-first-experiment"

GET /api/v1/experiments/{id}/versions

List stored versions, last-pushed first. Each version carries fingerprint, timestamps, creator, runCount, and a first-created ordinal (the vN the CLI shows; re-pushing an older definition keeps its ordinal).

curl -H "Authorization: Bearer $AX_API_KEY" \
  "https://app.514.ax/api/v1/experiments/my-first-experiment/versions"

GET /api/v1/experiments/{id}/yaml

Download the registered YAML.

curl -H "Authorization: Bearer $AX_API_KEY" \
  "https://app.514.ax/api/v1/experiments/my-first-experiment/yaml"
ParameterWhat it does
versionVersion fingerprint to pull. Omit for the latest.

POST /api/v1/experiments/{id}/runs

Submit a run for a registered experiment. Returns 202 Accepted with the submit receipt (runRequestId, pinned version fingerprint and ordinal, variantCount, selected variantIds); run failures surface asynchronously on the request. Poll GET /api/v1/runs/{id} to watch it; there is no watch stream.

curl -X POST \
  -H "Authorization: Bearer $AX_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"repeat":3,"variantIds":["claude::baseline"]}' \
  "https://app.514.ax/api/v1/experiments/my-first-experiment/runs"

JSON body (optional):

Body fieldWhat it does
versionVersion fingerprint to run. Omit for the latest.
repeatRepeat count per variant (0 means 1).
variantIdsExact variant ids to run.
promptIds / agents / models / productIds / environmentIdsAxis selectors. Selectors are a union: a variant runs when it matches any of them.
flagsRegistered run flags.
mockBoolean; deterministic mock driver, no model spend.

Snake_case spellings (variant_ids, prompt_ids, ...) are also accepted. Unknown selectors, unknown versions, or a selection matching no variants return 400 invalid_argument. Experiments that declare files: staging cannot be staged over HTTP; use the CLI with --file.

POST /api/v1/experiments/{id}/query

Query results in experiment scope: SQL, or the experimental flag-based modeled query. The HTTP mirror of ax experiment query.

curl -X POST \
  -H "Authorization: Bearer $AX_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"sql":"SELECT test_name, exit_code FROM tests WHERE exit_code != 0"}' \
  "https://app.514.ax/api/v1/experiments/my-first-experiment/query"

JSON body:

Body fieldWhat it does
sqlRaw ClickHouse SELECT; the server applies the experiment scope. Cannot be combined with the modeled fields below.
limitRow limit, up to 1,000 (0 uses the default of 100).
dryRunValidate and return normalized SQL plus an estimate without executing. Not with saveInsight.
saveInsightSave the query as an insight: {heading, slug?, description?, labels?, force?}.

Experimental modeled fields (no sql; the surface may change):

Body field (experimental)What it does
metricstestPassRate, testsPassed, testsFailed, cost, wallClockTime, tokens, toolCalls, toolFailures. Default testPassRate.
groupByOrdered array of variant, agent, model, product, environment, prompt, test. Default variant.
statAggregate statistic. Only mean is supported today (the default).
agents / products / environments / promptsDimension filters. Same axis ORs; different axes AND.
experimentVersionlatest (default) or a version fingerprint.
discoverDimension"" lists all dimensions and metrics; a dimension name lists its observed values. Not with dryRun or saveInsight.

The CLI-style aliases version, dimensions, and save are also accepted. Responds with query rows, projection-order columns, compiled SQL, and (when requested) the dry-run estimate, discovered dimensions, or saved-insight receipt.