MCP tools

The MCP server mirrors the CLI nouns: same ids, same vocabulary. Org-scoped tools use the current MCP org (see mcp_org tools) unless you pass an org_id override. Arguments marked required must be supplied; everything else is optional.

Experiment tools

experiment_list filters with search, owner, agents, models, products, environments, updated_since (RFC 3339), and limit. Same-axis values OR; different axes AND.

experiment_view includes a resources array (id, title, url, createdAt, updatedAt). Empty is []. Resources hang off the experiment, not a version, and are not in YAML. Tool arguments are snake_case (experiment_id, resource_id); structured results use the same lowerCamelCase JSON as the HTTP API.

experiment_resource_add takes required experiment_id plus either one title and url, or a resources array of {title, url}. Do not send both shapes. Title and URL are both required on each item. http:// or https:// only; same title or URL twice is allowed.

experiment_resource_edit takes required experiment_id and resource_id (UUID from experiment_view) plus at least one of title or url. experiment_resource_delete takes required experiment_id and resource_id. A missing resource is an error, not a silent success.

experiment_push takes required experiment_yaml. An experiment that declares required dependencies (files:, setup.files, setup.skills, or tests[*].files entries with a source) needs its complete package to become a runnable version — ax experiment push resolves and uploads those files for you. A YAML-only call is only accepted when it matches the YAML of an existing stored version (an idempotent re-push); otherwise it fails rather than registering an incomplete version. A caller that has already staged each dependency's content-addressed blob for this org can supply the package directly with replace_version_files: true and version_files (dest, tar_sha256, size_bytes, source, and optionally variant_id, test_names, name) — the same fields the API's JSON push body accepts.

experiment_fork takes required experiment_id plus optional new_experiment_id (kebab-case; derived as <experiment_id>-v2 when omitted, with a counter appended when that id is taken) and new_name (derived as the source name followed by (V2)). It registers the source's latest stored definition and dependencies as a new experiment on the current results pipeline (session_data_v1) and creates no runs. The source experiment is only read. Use it when an experiment's results are held in the retired data pipeline and you want to run the same definition again, then pass the returned experimentId to experiment_run.

experiment_run accepts version, repeat, exact variant_ids, the axis selectors (prompt_ids, agents, models, product_ids, environment_ids), registered flags, and mock. Selectors are a union; no selector runs every variant. The response pins the version and returns its fingerprint, ordinal, selected variant ids, and the run request id.

experiment_query takes required experiment_id plus either raw sql (scoped to the experiment) or the experimental modeled fields (metrics, ordered group_by, stat (only mean today), dimension filters, version, and dimensions discovery; the surface may change). Both modes support limit (default 100, cap 1,000), dry_run, and save ({heading, slug?, description?, labels?, force?}); dry_run, save, and dimension discovery are mutually exclusive. Metrics are testPassRate, testsPassed, testsFailed, cost, wallClockTime, tokens, toolCalls, toolFailures. Both modes leave out runs classified as "test aware" (the agent noticed it was being tested) unless include_test_aware is set; those runs may not mimic real-world agent behavior. The modeled result reports how many were held back as excludedTestAwareRuns.

Run tools

run_list accepts experiment_id, version_scope, the dimension filters, exact variant_ids, statuses, created_by_ids, status_updated_since (when the run entered its current status, not creation), test_aware (only runs classified as "test aware", where the agent noticed it was being tested or changed its behavior because of it), run_request_id (only the runs of one submission), and limit.

run_view takes a request id or composite run id. Set tests: true for every captured test, or test_name for one test's exit code, duration, and output tails (composite run ids only; not both). For a transcript, select one composite run with transcript: true. Pass include: ["reasoning"] to add reasoning and limit to change the 500-turn default (minimum 1, maximum 10,000). Transcript and test payloads cannot be combined.

The transcript object matches CLI JSON. Its ordered turns contain seq, nullable ts, role, kind, and text; include_reasoning, limit, truncated, shown, total, and skipped_malformed describe completeness.

run_query takes required composite run_id and sql, plus limit, dry_run, and save with the same semantics as experiment_query.

Insight tools

insight_create takes required heading and sql, plus slug, description, labels, and force (overwrite mutable content on slug collision; slugs are immutable). insight_view executes live unless sql_only: true. insight_edit is presence-aware: an empty description or labels array clears the field (clear_labels also works).

Secret tools

Org context and CLI auth

Legacy tools

Kept for compatibility; prefer experiment_query, run_query, and insight_* for new agent workflows.

Not in MCP

Local authoring (ax experiment create / validate / schema / variants) and resolving local files into a package stay CLI-side — experiment_push accepts an already-staged package by reference (see above) but never reads from or uploads a caller's local disk. experiment_run can run registered experiments with files: after ax experiment push has stored their dependencies. There is no watch tool (poll run_view with the request id), and insight export is a client-side CLI operation; agents can consume the structured rows the query and insight tools return.