MCP tools

The MCP server mirrors the CLI nouns: same ids, same vocabulary. The available catalog can vary with organization features and access; call tools/list for the complete current catalog and input schemas. Org-scoped tools use the current MCP org (see mcp_org tools) unless you pass an org_id override. Arguments marked required must be supplied; everything else is optional.

Experiment tools

experiment_list filters with search, owner, agents, models, products, environments, updated_since (RFC 3339), and limit. Same-axis values OR; different axes AND.

experiment_view includes a resources array (id, title, url, createdAt, updatedAt). Empty is []. Resources hang off the experiment, not a version, and are not in YAML. Tool arguments are snake_case (experiment_id, resource_id); structured results use the same lowerCamelCase JSON as the HTTP API.

experiment_resource_add takes required experiment_id plus either one title and url, or a resources array of {title, url}. Do not send both shapes. Title and URL are both required on each item. http:// or https:// only; same title or URL twice is allowed.

experiment_resource_edit takes required experiment_id and resource_id (UUID from experiment_view) plus at least one of title or url. experiment_resource_delete takes required experiment_id and resource_id. A missing resource is an error, not a silent success.

experiment_push takes required experiment_yaml. An experiment that declares required dependencies (files:, setup.files, setup.skills, or tests[*].files entries with a source) needs its complete package to become a runnable version — ax experiment push resolves and uploads those files for you. A YAML-only call is only accepted when it matches the YAML of an existing stored version (an idempotent re-push); otherwise it fails rather than registering an incomplete version. A caller that has already staged each dependency's content-addressed blob for this org can supply the package directly with replace_version_files: true and version_files (dest, tar_sha256, size_bytes, source, and optionally variant_id, test_names, name) — the same fields the API's JSON push body accepts.

experiment_fork takes required experiment_id plus optional new_experiment_id (kebab-case; derived as <experiment_id>-v2 when omitted, with a counter appended when that id is taken) and new_name (derived as the source name followed by (V2)). It registers the source's latest stored definition and dependencies as a new experiment on the current results pipeline (session_data_v1) and creates no runs. The source experiment is only read. Use it when an experiment's results are held in the retired data pipeline and you want to run the same definition again, then pass the returned experimentId to experiment_run.

experiment_run accepts version, repeat, exact variant_ids, the axis selectors (prompt_ids, agents, models, product_ids, environment_ids), registered flags, mock, and setup_only. A setup_only submit runs one sandbox per distinct setup unless all_variants is set, the same as ax experiment run --setup-only. Setup-only runs have no model spend, but each run incurs the per-run platform fee (sandbox charge). Selectors are a union; no selector runs every variant. The response pins the version and returns its fingerprint, ordinal, selected variant ids, and the run request id.

experiment_query takes required experiment_id plus either raw sql (scoped to the experiment) or the experimental modeled fields (metrics, ordered group_by, stat (only mean today), dimension filters, version, and dimensions discovery; the surface may change). Both modes support limit (default 100, cap 10,000), dry_run, and save ({heading, slug?, description?, labels?, force?}); dry_run, save, and dimension discovery are mutually exclusive. Metrics are testPassRate, testsPassed, testsFailed, cost, wallClockTime, tokens, toolCalls, toolFailures. Both modes leave out runs classified as "test aware" (the agent noticed it was being tested) unless include_test_aware is set; those runs may not mimic real-world agent behavior. Both modes report how many were held back as excludedTestAwareRuns, and the result text repeats it when any were.

Run tools

run_list accepts experiment_id, version_scope, the dimension filters, exact variant_ids, statuses, created_by_ids, status_updated_since (when the run entered its current status, not creation), test_aware (only runs classified as "test aware", where the agent noticed it was being tested or changed its behavior because of it), run_request_id (only the runs of one submission), and limit.

run_view takes a request id or composite run id. Set tests: true for every captured test, or test_name for one test's exit code, duration, and output tails (composite run ids only; not both). For a transcript, select one composite run with transcript: true. Pass include: ["reasoning"] to add reasoning and limit to change the 500-turn default (minimum 1, maximum 10,000). Transcript and test payloads cannot be combined.

Set setup_log: true with one composite run id for the run's setup.log (the tail when it exceeds the size cap) and its setup-check results, the same as ax run view --setup-log. The structured result carries found, content, truncated, totalSizeBytes, maxBytes, and setupChecks. setup_log cannot be combined with tests, test_name, or transcript.

The transcript object matches CLI JSON. Its ordered turns contain seq, nullable ts, role, nullable origin (human or harness on user turns), kind, and text; include_reasoning, limit, truncated, shown, total, and skipped_malformed describe completeness.

A composite run's run_view result also lists its failed tool calls, with or without a transcript. The run's failedToolCalls array (up to 50 calls, with failedToolCallsTruncated set when there were more) gives each call's seq, position, tool_name, error_text (the joined error content with likely secrets redacted, shown as [redacted], cut to at most 2,000 Unicode code points; error_truncated says when the redacted text was cut, and error_chars is the length in Unicode code points of the redacted error text, before the cut), and the summary text lists them. With a transcript, each listed failed call also appears as a tool_error turn carrying the same tool_call object. When failedToolCallsTruncated is set, query every failed call with run_query (pass limit: 10000, since the default is 100): SELECT seq, tool_name, payload FROM tool_interactions WHERE is_error ORDER BY seq.

When more than half of a run's MCP calls failed (returned an error, or returned a result naming an HTTP 4xx or 5xx status), or otherwise more than half of all its tool calls, the run also carries toolFailureFlag (failedCalls, totalCalls, mcp, errorStatuses, and summary, for example 4/4 MCP calls failed: 404), and the summary text opens its failed-call section with Most tool calls failed: <summary>. A flagged run can pass its tests without its agent ever reading real data.

run_query takes required composite run_id plus sql or tables, and limit, dry_run, and save with the same semantics as experiment_query.

Pass tables to run_query or experiment_query to discover what raw sql can read, the same as ax run query <RUN_ID> sql --tables and ax experiment query <EXPERIMENT_ID> --table on the CLI: "" lists the tables, and a table name returns its columns, types, join keys, and allowed values. run_query also reports each table's row count. The structured result carries the same fields as the CLI's JSON output, in the tools' lowerCamelCase JSON. tables cannot be combined with sql, limit, dry_run, save, or the modeled fields.

Insight tools

insight_create takes required heading and sql, plus slug, description, labels, and force (overwrite mutable content on slug collision; slugs are immutable). insight_view executes live unless sql_only: true. insight_edit is presence-aware: an empty description or labels array clears the field (clear_labels also works).

Set experiment on insight_list to discover its three built-in Results charts.

Report tools

report_get_metadata and report_get_content each take exactly one of report_id or report_name. Clarify ambiguous names before reading. When combining their results, use the same report ID and matching updatedAt values; reread if the report changed between calls. report_archive takes exactly one of report_id or report_name. If a name matches multiple reports, it returns candidates and does not archive anything. Name selectors match active reports only; an archived report is reached by report_id. report_create takes required title plus description and body_markdown. report_add_block takes exactly one of report_id or report_name, plus required block, position and expected_updated_at. To append several blocks, insert each at position: { at: "end" } in order, using the completed write’s updatedAt for the next insertion. Clarify ambiguous report names before writing. report_update_metadata, report_move_block, report_remove_block, and block_update also require the report's latest expected_updated_at value. block_get can return all saved properties or a selected subset with fields. block_update is typed by block kind, preserves omitted fields, and replaces an array when that array is included in the patch. The catalog exposes separate metadata and content reads, plus focused edit tools. report_update_metadata changes the title or description. report_move_block and report_remove_block manipulate existing blocks; report_add_block accepts a typed block payload to insert one without replacing the full body. The former report_edit tool is retired. For product-version Comparison references, call block_list_data_sources with required report_id or report_name (exactly one), block_kind set to version-selection-rate-evidence, and filters containing category: "product-version" and experimentId. You can also pass experimentFingerprint to pin a registered version or productId to narrow the returned products. Omitting the fingerprint reads all experiment versions. If the report name is ambiguous, choose a returned report before requesting sources. An unknown experiment or fingerprint is an error, not an empty list.

The result contains report identity, experiment-version identities, product IDs and versions, and exact selection-test names. Copy those references without choosing a version pair from labels alone. Preserve a nonempty experimentFingerprint as the block's scope.experimentFingerprint; omit that scope field when the returned value is empty. Discovery returns no report revision, measurements, or selection counts. Read the report through the appropriate content or block read before a later write and use that read's concurrency token. block_list_data_sources takes required block_kind and filters, plus exactly one of report_id or report_name. For block_kind: "selection-rate", filters requires experimentId and accepts experimentFingerprint (default: the current version) and exact testName. Resolve ambiguous report names before reading sources. Make a separate call for each baseline experiment/version.

Read candidate identities from tests and versions, and scope values from availableFilters. availableFiltersScope: "experiment" means dimension options span the experiment; registered tests follow the returned experimentFingerprint. A test-name filter can return zero or multiple candidates. Select a returned identity explicitly; duplicate names do not establish identity. The response contains references, not rates, run counts or a revision for writing the report. Keep the revision from your report read when using the guarded report writer. Selection sources return an empty nextPageToken. For paginated source kinds, pass a nonempty token unchanged as page_token with the same report, kind and filters, even after an empty page. For CLI selection discovery, use ax report selection-rate-sources. CLI and REST source discovery keep their existing experiment-only inputs and result shape.

block_list_data_sources requires exactly one authorized report_id or report_name; an ambiguous name returns candidates for clarification. For A/B Comparison test results, set block_kind: "version-selection-rate-evidence" and filters: {"category": "test-result"}. Add filters.search for a literal, case-insensitive test-name match, or filters.experimentId and filters.experimentFingerprint for an exact version (a fingerprint requires its experiment). Copy each row's kind, experimentId, experimentFingerprint and testId unchanged. Results contain references only, never pass counts or rates. Read-only credentials cannot discover comparison test results.

When nextPageToken is not empty, pass it unchanged as page_token with the same report, block kind and filters, even after an empty page. Start without a page token when changing tools; tokens from earlier discovery tools are not interchangeable. Discovery does not return a report revision. Use the updatedAt from your saved report read as expected_updated_at for the next write, and read again if that revision is stale.

report_frontmatter_group_sources takes report_id, experiment_id (with an optional experiment_fingerprint) or both. Each product reads as available, archived or missing, each experiment as available or missing, and the metric as available, missing_experiment, missing_version or missing_test.

Read report_get_content before adding or changing a front-matter group. Use report_add_block to insert a block with kind: "frontmatter-group" at an explicit position. Use block_update to change an existing group, with its full block_id from that read and a patch with kind: "frontmatter-group". Both writers take exactly one of report_id or report_name and require that read's updatedAt as expected_updated_at. Clarify ambiguous report names or destination headings before writing.

Follow the tool's input schema for the group's fields. Omitted patch fields stay unchanged; metric: null clears the metric. If you replace a rows collection, include the rows you want to keep. A newly selected source must resolve in the org; an unavailable source already in an untouched row stays. Changing the metric's experiment or version while carrying over its old test or pinned version clears the metric to null. Select the new source's version and test together to configure it again.

Retry an insertion with its original request and expected_updated_at. A new insertion at a fresh revision creates another group even when the values match. A stale revision refuses the write; read the current content before planning a new change.

Use block_update for individual row or text choices so omitted fields retain the member's edits.

The report_add_block tool’s block parameter states the document format and carries the JSON Schema for every ax-block kind, so an agent can build a valid fence without another lookup. Results use the same lowerCamelCase JSON as the HTTP API.

Use block_list_data_sources with exactly one of report_id or report_name, block_kind: "trace-quote", and filters: { experimentId, runId, limit? } to discover candidate messages. limit defaults to 50 and is at most 100. If a report name is ambiguous, choose the intended report before continuing. Copy each candidate's canonical source unchanged. report_trace_quote_source takes required experiment_id, run_id and message_entity_id to read one candidate's exact text and recorded identity. The quote's authored text is separate from that recorded evidence. To modify a saved quote, use block_update with its saved block_id, a typed trace-quote patch and the last-read expected_updated_at. Omitted fields stay unchanged. Recorded identity comes from the chosen source and cannot be authored in the block.

Pass nextPageToken unchanged as page_token, keeping the report, block type and filters unchanged, even after an empty page. An empty token ends paging. Message previews are exact prefixes, not full verified quotes. Read a candidate with report_trace_quote_source before quoting it. A message that is missing, unavailable or empty comes back without text or a block. For chart insights, insight_view with sql_only: true returns the parameter schema, response schema, and available filter values as parameterSchema, responseSchema, and availableValues inside insight.

Execute with a parameters object, for example {"ref":"<INSIGHT_REF>","parameters":{"group_by":"model","metric":"wallSeconds"}}. Chart results are in chartData; saved SQL results remain in rows. To retrieve points or underlying runs, set result_view to points or runs, then pass chartData.nextCursor as cursor until it is null. Keep the same parameters and view for every page. Use group_value and test_name for an exact heatmap cell. Omit limit for chart insights; summaries always cover all matching runs.

See CLI insight usage for filter and pagination examples.

Secret tools

Org context and CLI auth

auth_whoami takes no arguments and returns user_id, email, org_id, org_name, and org_slug. It uses your MCP connection's identity and organization selection. Organization fields are null when no organization is selected; email is null for service identities or users without an email address. It never returns credentials.

Context tools

context_list_types, context_list, context_get, and context_list_documents accept scope: "org" (default) or "global". Object discovery is the source of truth for object types, allowed filters, ordering, and the relationships context_get embeds one level deep. Relationship tables are not object types and cannot be listed; their rows appear only as embedded arrays. Document hosts are companies, products, use-cases, personas, agents, persona-agents, interfaces, capabilities, user-flows, markets, product-packages, product-versions, product-competitors, agent-versions, and models. The companies type holds Company Context objects (companies, customers, vendors, and competitors), shown as Companies in the platform.

context_get and context_list_documents accept exactly one target selector: id, name, or url. Supply kind with IDs, names, or company/product website URLs. AX page URLs identify their kind and scope. Ambiguous names or websites return candidates to choose from; an incomplete search requires an ID or AX page URL. A URL does not grant access to another organization's data. Persona-agent attachment targets require an association ID.

context_get accepts optional fields, such as ["description", "website_url"], and relationships, such as [{"key": "personas", "limit": 20}]. Selecting fields loads no unrelated relationships. Each selected field contains value, source: "saved", and its stored revision when available. Null values remain null; unselected fields are omitted. The response also includes canonical kind, id, scope, and revision. Live editor values are unavailable.

Each relationship returns key, kind, items, and complete. A known total counts all items only when the underlying read is complete. These bounded reads provide no continuation cursor: after is rejected. Use context_list filters or context_list_documents pagination to continue discovery. truncatedRelationships names incomplete relationships, and the top-level complete applies only to selected values. Do not use an incomplete relationship as a replacement value. Narrow oversized reads with fields or use a paginated list.

context_list_documents returns attachment metadata, not complete file content. Continue with page_token until next_page_token is empty to list every attachment.

context_options separates candidate discovery from type discovery. It accepts kind, optional search, supported filters, and page_token. Its opaque continuation token belongs to the same organization, kind, search, and filters. Discovery does not create an object or attach a candidate.

The five mutation tools above are organization-only. context_create takes kind and typed values; an optional id makes a create retry use the same object ID. Companies, products, personas, markets, and use cases are checked for existing identity matches before creation. A conflicting or pending identity stops creation.

For context_update, context_remove, and context_owners, identify the object with exactly one of id, name, or url. IDs and names also require kind. An AX page URL carries its kind; company and product websites require kind. Ambiguous or incomplete matches make no changes. Shared catalog objects cannot be mutated through these tools.

context_update also takes expected_updated_at and values. Omitted fields stay unchanged; null clears only nullable fields. A stale revision conflicts. For a user flow, send flow_steps instead of values, listing every current Step ID once in the desired order. Step content and links remain unchanged.

context_remove also takes the expected effect (archive or delete). It checks dependencies itself and reports blockers without removing anything. Live roots archive; permanently deleting an archived root requires an organization admin. Other object kinds require effect: "delete". An unexpected effect stops removal.

context_batch_mutate takes up to 100 operations, each a create or update with the corresponding fields above (flow ordering is a single-object update). Every operation must validate before any changes commit. A stale revision or duplicate create rejects the whole batch.

context_owners also takes an operation: list, members, assign, or unassign. Use members with optional search to find eligible members, then pass a user_id to assign or unassign one owner. Other owners stay assigned. Ownership grants no access or role. Use after with the returned nextPageToken to continue a list or member search.

context_edit is organization-only. Its actions are read, options, preview-delete, mutate, mutate-many, flow-steps, and document upload actions. models and agent-versions are read-only through this editor. Use mutate with kind user-flow-steps to create a Step or update its action and expected_result. Use attach and detach on user-flow-step-memberships, user-flow-step-interfaces, and user-flow-step-capabilities to manage its links. flow-steps takes {id,expectedUpdatedAt,steps:[{id}]}, with the Flow ID and existing Step IDs in the desired order. Omitting a Step detaches its membership and preserves the Step and its content.

On an archived root (context_list with archived: "only" finds them), an org admin's preview-delete and delete remove it permanently with its owned records; records that restrict it are deleted or detached first. context_copy accepts global companies, products, user flows, user flow steps, use cases, personas, and agents, and carries their published tags into the organization library (reusing tags matched by origin or name, creating the rest, and keeping local edits); a name collision with a different organization tag fails the whole copy with a message naming both tags. context_claim can check companies, products, agents, personas, use cases, or documents; ownership writes are limited to companies, products, and agents for organization admins.

context_tags takes one action: targets, list, get, create, update, delete, assign, unassign, usage, or assigned (the tags one object carries, by kind and target_id), with snake_case fields (tag_id, target_id, kind, name, color, patch, expected_updated_at, expected_usage_count, search, after, limit, include_usage, scope). Organization tags and their assignments are editable; scope: "global" reads the shared library and refuses writes, and shared objects take tag changes through the tags action of context_suggestions instead of assign. create returns the existing tag with created: false when the name is taken, and assign and unassign are idempotent, so retries are safe. update requires the tag's current updatedAt as expected_updated_at, and delete requires the count from usage as expected_usage_count once a tag is in use; when the count moved, the error message names the current count to confirm with. usage items carry parentKind and parentId for nested objects such as an interface under its product. context_list narrows taggable types to objects carrying any of the tags in filters.tags (comma-separated tag ids), alongside its column filters.

context_match takes candidates, up to 50 objects each with a type (companies, products, personas, markets, or use-cases) and a name, website, or aliases. It answers each candidate in order with the same result as POST /context/match: matches starts with the existing object its exact identity belongs to (the one research reuses), duplicates lists other objects with the same identity, and lookAlikes are different objects whose names only look similar, which are never matched. The optional different list contains live rows previously marked Different; ordinary reruns do not ask about those pairs again. To reconsider an answer, ask AX in web chat. MCP, CLI, and REST can read these results but cannot record identity answers.

context_follow and context_follow_state act on global companies, products, use cases, personas, and agents. They change or read only the authenticated person's own follow state and never list other followers; org_id only selects the organization whose feature rollout applies.

context_suggestions takes input, the same JSON action used by ax context suggest, and optional org_id. Actions are snapshot, submit, tags, list, and withdraw. tags proposes shared-tag changes for any taggable shared object: {"action":"tags","op":"snapshot","kind","targetId"} returns the published tags plus the shared library, and {"action":"tags","op":"submit","kind","targetId","add","remove","reason","idempotencyKey"} files the proposal. It never edits shared Context directly and cannot approve or deny a suggestion. list and withdraw only access the authenticated user's own suggestions; list takes requestKind (suggestion or tag_assignment) to return one kind. Reuse a submission's idempotencyKey only when retrying that same submission.

Notification tools

Both act only on the authenticated person's feed in the current MCP org. notification_feed takes page_size (at most 100), cursor (the previous result's next_cursor), and unread_only. Each entry is one changed object with a stable event_id. notification_feed_mark_read takes 1 to 100 event_ids and returns marked_count; already-read or unknown ids are ignored, so repeating it is safe. The same routes are /api/v1/notifications.

Workflow guidance

learn with no arguments lists the workflows you can follow with MCP tools; pass a workflow's slug for its steps, in order. A tool step names the tool to call and how to call it; other steps describe what to do between calls. The label-behaviors workflow appears only when behaviors are enabled for one of your organizations.

Errors

Errors come back in one of two forms, following the MCP specification:

tools/list advertises each tool's input schema as JSON Schema draft 2020-12 ($schema is https://json-schema.org/draft/2020-12/schema, shared definitions sit under $defs).

Legacy tools

Kept for compatibility; prefer experiment_query, run_query, and insight_* for new agent workflows.

The former analyze_* and persisted-findings tools are not in the current catalog. The only current findings-related tool is findings_resolve, which resolves run, tool-call, timeline, span, or prompt anchors into live trace evidence; it is listed under Other tools.

Not in MCP

Some CLI commands have no direct MCP equivalent:

ax experiment create writes a local file. Resolving local files into a package also stays CLI-side: experiment_push accepts a staged package by reference but never reads from or uploads a caller's local disk. experiment_run can run registered experiments with files: after ax experiment push has stored their dependencies. Insight export is a client-side CLI operation; agents can consume the structured rows returned by the query and insight tools.

Evals and datasets

See eval and dataset tools for definitions, results, immutable snapshot reads, explicit held-out reveals and dataset publication.

Agent version and model tools

Behavior tools

Behavior tools are experimental and may change.

Other tools