Experiments
HTTP API routes for experiments.
| Route | Use | CLI equivalent |
|---|---|---|
GET /experiments | List registered experiments. | ax experiment list --json |
POST /experiments | Register YAML without running it. | ax experiment push |
GET /experiments/{id} | Inspect one experiment. | ax experiment view --json |
GET /experiments/{id}/versions | List stored versions. | ax experiment versions --json |
GET /experiments/{id}/yaml | Download registered YAML. | ax experiment pull |
POST /experiments/{id}/runs | Submit a run. | ax experiment run <id> |
POST /experiments/{id}/query | Query results in experiment scope. | ax experiment query |
Every route takes org_id as a query parameter (required only when the key
can see several orgs; see Organization scope); it is
not repeated in the tables below. Parameters are optional unless marked
required.
GET /api/v1/experiments
List registered experiments in the selected org, newest first.
curl -H "Authorization: Bearer $AX_API_KEY" \
"https://app.514.ax/api/v1/experiments?search=install&limit=20"| Parameter | What it does |
|---|---|
search | Case-insensitive match on experiment name or id. |
owner | Owner email or id filter. |
agent / model / product / environment | Dimension filters (repeatable or comma-separated). Same axis ORs; different axes AND. |
since | RFC 3339 updated-at floor (updated_since is an alias). |
limit | Max experiments to return. |
POST /api/v1/experiments
Register experiment YAML without running it. The body is the raw YAML, not JSON; each push stores a content-addressed version.
curl -X POST \
-H "Authorization: Bearer $AX_API_KEY" \
-H "Content-Type: text/yaml" \
--data-binary @experiment.yaml \
"https://app.514.ax/api/v1/experiments"Responds with the pushed experiment version metadata. Empty bodies return
400 invalid_argument.
GET /api/v1/experiments/{id}
Inspect one registered experiment: metadata and resolved axes. URL-encode
{id} if it contains reserved characters.
curl -H "Authorization: Bearer $AX_API_KEY" \
"https://app.514.ax/api/v1/experiments/my-first-experiment"GET /api/v1/experiments/{id}/versions
List stored versions, last-pushed first. Each version carries fingerprint,
timestamps, creator, runCount, and a first-created ordinal (the vN the
CLI shows; re-pushing an older definition keeps its ordinal).
curl -H "Authorization: Bearer $AX_API_KEY" \
"https://app.514.ax/api/v1/experiments/my-first-experiment/versions"GET /api/v1/experiments/{id}/yaml
Download the registered YAML.
curl -H "Authorization: Bearer $AX_API_KEY" \
"https://app.514.ax/api/v1/experiments/my-first-experiment/yaml"| Parameter | What it does |
|---|---|
version | Version fingerprint to pull. Omit for the latest. |
POST /api/v1/experiments/{id}/runs
Submit a run for a registered experiment. Returns 202 Accepted with the
submit receipt (runRequestId, pinned version fingerprint and ordinal,
variantCount, selected variantIds); run failures surface asynchronously on
the request. Poll GET /api/v1/runs/{id} to
watch it; there is no watch stream.
curl -X POST \
-H "Authorization: Bearer $AX_API_KEY" \
-H "Content-Type: application/json" \
-d '{"repeat":3,"variantIds":["claude::baseline"]}' \
"https://app.514.ax/api/v1/experiments/my-first-experiment/runs"JSON body (optional):
| Body field | What it does |
|---|---|
version | Version fingerprint to run. Omit for the latest. |
repeat | Repeat count per variant (0 means 1). |
variantIds | Exact variant ids to run. |
promptIds / agents / models / productIds / environmentIds | Axis selectors. Selectors are a union: a variant runs when it matches any of them. |
flags | Registered run flags. |
mock | Boolean; deterministic mock driver, no model spend. |
Snake_case spellings (variant_ids, prompt_ids, ...) are also accepted.
Unknown selectors, unknown versions, or a selection matching no variants
return 400 invalid_argument. Experiments that declare files: staging
cannot be staged over HTTP; use the CLI with --file.
POST /api/v1/experiments/{id}/query
Query results in experiment scope: SQL, or the experimental flag-based modeled
query. The HTTP mirror of
ax experiment query.
curl -X POST \
-H "Authorization: Bearer $AX_API_KEY" \
-H "Content-Type: application/json" \
-d '{"sql":"SELECT test_name, exit_code FROM tests WHERE exit_code != 0"}' \
"https://app.514.ax/api/v1/experiments/my-first-experiment/query"JSON body:
| Body field | What it does |
|---|---|
sql | Raw ClickHouse SELECT; the server applies the experiment scope. Cannot be combined with the modeled fields below. |
limit | Row limit, up to 1,000 (0 uses the default of 100). |
dryRun | Validate and return normalized SQL plus an estimate without executing. Not with saveInsight. |
saveInsight | Save the query as an insight: {heading, slug?, description?, labels?, force?}. |
Experimental modeled fields (no sql; the surface may change):
| Body field (experimental) | What it does |
|---|---|
metrics | testPassRate, testsPassed, testsFailed, cost, wallClockTime, tokens, toolCalls, toolFailures. Default testPassRate. |
groupBy | Ordered array of variant, agent, model, product, environment, prompt, test. Default variant. |
stat | Aggregate statistic. Only mean is supported today (the default). |
agents / products / environments / prompts | Dimension filters. Same axis ORs; different axes AND. |
experimentVersion | latest (default) or a version fingerprint. |
discoverDimension | "" lists all dimensions and metrics; a dimension name lists its observed values. Not with dryRun or saveInsight. |
The CLI-style aliases version, dimensions, and save are also accepted.
Responds with query rows, projection-order columns, compiled SQL, and (when
requested) the dry-run estimate, discovered dimensions, or saved-insight
receipt.