Data, Results and Insights

Individual runs tell you what one agent did once. To answer a product question you need them added up, so Fiveonefour rolls a set of runs into a table: that table is a result.

Three entry points cover almost everything: ax experiment query for rollups across an experiment, ax run query for one run, and ax insight for a query you want to keep and share.

Two ways to query

Both ax experiment query and ax run query accept either style:

• SQL is the primary path: write it yourself, scoped to one experiment or one run.
• Flags are an experimental shorthand for common rollups. You choose metrics, filters, and grouping, and Fiveonefour compiles the SQL.

With SQL

Add the sql subcommand. Start with --tables to see what is queryable, then write the query:

# What tables and columns exist?
ax run query '<RUN_ID>' sql --tables
ax run query '<RUN_ID>' sql --tables events

# Failing tests in one run
ax run query '<RUN_ID>' sql "SELECT test_name, exit_code FROM test_output WHERE exit_code != 0" --format table

# Across an experiment, from a file, saved as an insight
ax experiment query <EXPERIMENT_ID> sql @query.sql --save "Failing tests"

Available tables

Run dimensions such as experiment, variant, agent, model, prompt, product, environment, and status are available on these tables, so there is no separate metadata table.

events is an observations table, not a deduplicated timeline. The same activity can appear in several source records. Filter its source column (setup, acp, stdout, hooks, otel, transcript, harness, other). Its ts column is nullable because some records have no trustworthy time, and payload keeps the source-specific body as JSON-encoded text.

Query limits

Remote SQL runs with server-enforced limits; SETTINGS clauses in your SQL are rejected. Authenticated queries have no scan limits: they can read as much as the analytical store can serve within a 13 minute execution ceiling, and each query returns at most 10,000 rows and 16 MiB. Publicly shared insight pages run with much smaller scan and time limits.

Result rows are capped but reads are not, so keep aggregation in SQL instead of paginating raw rows out for local processing: one aggregate query is faster, cheaper, and returns the numbers you actually want. The pattern that scales to full experiments is one statement that flags each run in a subquery, then joins every run back in for the denominator:

-- Share of runs whose transcript mentions a term, by agent and mode
SELECT m.agent, m.agent_mode,
  count() AS run_count,
  countIf(t.mentioned = 1) AS matched_count,
  round(matched_count / run_count, 3) AS rate
FROM measurements AS m
LEFT JOIN (
  SELECT run_id, max(match(payload, '(?i)mongodb|mongoose')) AS mentioned
  FROM events
  WHERE kind IN ('message', 'reasoning', 'tool_call')
  GROUP BY run_id
) AS t ON t.run_id = m.run_id
GROUP BY m.agent, m.agent_mode

Run it as one ax experiment query <EXPERIMENT_ID> sql @query.sql call: the experiment scope is applied server-side, runs with no matching events stay in the denominator, and only the grouped rates come back.

With flags (experimental)

Flag-based modeled queries are experimental, so expect the surface to move. Three choices decide what comes back:

• Filters pick which runs count: --filter dim=value, the axis flags (--agent, --prompt, --product, --environment), and --experiment-version.
• Grouping splits those runs, one row per group: --group-by variant|agent|model|product|environment|prompt|test.
• Metrics decide what is measured for each group, one column each: test outcomes (testPassRate, testsPassed, testsFailed), spend (cost, tokens), time (wallClockTime), and tool behavior (toolCalls, toolFailures). See experiment query for the defaults and the full set.

# What can I group and measure on this experiment?
ax experiment query <EXPERIMENT_ID> --dimensions

# Pass rate per product
ax experiment query <EXPERIMENT_ID> --metric testPassRate --group-by product

# Cost and tokens per prompt, claude only
ax experiment query <EXPERIMENT_ID> --metric cost,tokens --group-by prompt --filter agent=claude

# Keep it as an insight
ax experiment query <EXPERIMENT_ID> --metric testPassRate --save "Pass rate by product"

You can group and filter along any dimension the experiment defined (variant, agent, model, prompt, product, environment, experiment version) plus execution-derived fields such as run status and test success.

Each row averages or totals every run in that group, so check RUNS before trusting a comparison: a row backed by 50 runs means more than a row backed by one. --dry-run prints the compiled SQL without running it, and --json gives machine-readable output.

Legacy ax results view, ax results query, and ax results scatter spellings still work (legacy commands); prefer ax experiment query and ax run query.

Saved insights

An insight is org-scoped saved SQL with a heading and an immutable slug. Create one from a query with --save, or directly:

ax insight create "Cross-experiment cost" --sql @query.sql
ax insight list
ax insight view <slug>
ax insight share <slug>
ax insight edit <slug> --heading "Updated title"
ax insight export <slug> out.csv --format csv
ax insight delete <slug>

Insights re-execute live against your org's uploaded data on view and export. Listing returns metadata only and does not rerun SQL.

ax insight share prints a public capability URL. The first share publishes a frozen copy of the heading, description, query, data family, and execution scope. That page reruns the frozen query against live data, so rows can change while later edits to the source insight do not. Repeated shares return the same URL, and deleting the insight is the only way to revoke it in this release.

Local data

Use local data when you run an experiment on your machine or need to analyze results from AX Cloud offline.

Query local runs

ax experiment run --local always writes the run data to .axp/runs/<RUN_REQUEST_ID>/. Query that data directly from your machine, starting with SHOW TABLES to discover the available local tables.

# Discover the available local tables
ax run query '<RUN_ID>' sql "SHOW TABLES" --local --format table

# Query the local runs table
ax run query '<RUN_ID>' sql "SELECT * FROM runs LIMIT 10" --local --format table

See Local runs for the artifact layout.

Upload a local run

Local run data can be uploaded to your org at any time. When authenticated, AX attempts this automatically after the run finishes. Publication is best effort, so the local record remains available when an automatic upload is skipped or fails. If you weren't signed in when the run finished, you can sign in and upload it anytime:

ax auth login
ax run upload <RUN_REQUEST_ID>

Analyze AX Cloud data offline

Download a run from AX Cloud when you need to query it without a network connection. The downloaded tables live under .axp/downloads/<RUN_ID>/ and can be queried with --local.

# Download the run data
ax run download <RUN_ID>

# Discover the downloaded tables
ax run query '<RUN_ID>' sql "SHOW TABLES" --local --format table

Local and downloaded tables can differ from the tables available through remote SQL, so start with SHOW TABLES.