Trusted CI

Trusted CI lets a GitHub Actions job send eval results to your organization with a short-lived token. An org admin chooses which repository, ref, and environment can send results. The token grants only evals:ingest.

1. Add a trust rule

Enable agent evals for your organization (ax-agent-evals) before sending results. With Trusted CI enabled for your organization, open Integrations → GitHub → Trusted CI or Settings → Trusted CI and select Add rule. Both locations are available only to org admins. The ax-trusted-ci flag must be enabled for your organization both to show settings and to admit Trusted CI requests. Turning it off stops admission even when rules exist.

To enable Trusted CI for an org admin, target the ax-trusted-ci flag by organization, not by user: ingestion evaluates it with org context only.

Enter the repository name (owner/repo). Fiveonefour resolves its immutable GitHub repository ID using your organization’s GitHub App installation, or the public GitHub API when the installation does not cover it. For a private repository, grant your organization’s GitHub App access first. The saved rule displays the resolved ID read-only.

If the repository cannot be resolved, check the name and installation access, then try again.

Use the exact ref refs/heads/main and a protected GitHub environment, such as production. Configure environment protection rules in GitHub before trusting it.

Leaving the ref empty admits any branch and also admits jobs triggered by pull_request_target or workflow_run when the other constraints match. Leaving the environment empty removes that constraint. Review which workflows can request tokens before creating an unconstrained rule.

Both the immutable repository ID and the repository name must match. Names are compared case-insensitively; refs and environment names are exact matches. Recreating a repository under the same name does not inherit access.

2. Grant the workflow token permission

Set permissions: id-token: write on the job that sends results. The token audience must be the platform origin, such as https://app.514.ax. Set AX_ORG_ID to your organization's ID.

The following upload step expects your preceding eval step to write an IngestEvalBatch request to eval-batch.json. The file must contain agentId (an existing agent UUID) and batchJson (a JSON string containing the batch). The upload step sets the request's org ID explicitly. Replace the org ID and keep any ref or environment constraints consistent with your trust rule.

name: Send eval results
on:
  workflow_dispatch:

jobs:
  upload:
    runs-on: ubuntu-latest
    environment: production
    permissions:
      contents: read
      id-token: write
    env:
      AX_PLATFORM_URL: https://app.514.ax
      AX_ORG_ID: org_REPLACE_ME
    steps:
      - uses: actions/checkout@v4
      # Run your evals here and write eval-batch.json.
      - name: Upload eval results
        run: |
          node --input-type=module <<'JS'
          import { readFile } from 'node:fs/promises';
          const audience = new URL(process.env.AX_PLATFORM_URL).origin;
          const tokenUrl = new URL(process.env.ACTIONS_ID_TOKEN_REQUEST_URL);
          if (tokenUrl.protocol !== 'https:' || tokenUrl.port || tokenUrl.username ||
              tokenUrl.password || !tokenUrl.hostname.endsWith('.actions.githubusercontent.com')) {
            throw new Error('Invalid GitHub OIDC request URL');
          }
          tokenUrl.searchParams.set('audience', audience);
          const tokenResponse = await fetch(tokenUrl, {
            headers: {
              authorization: `Bearer ${process.env.ACTIONS_ID_TOKEN_REQUEST_TOKEN}`,
            },
            redirect: 'error',
          });
          if (!tokenResponse.ok) throw new Error('Could not get GitHub token');
          const { value: token } = await tokenResponse.json();
          const batch = JSON.parse(await readFile('eval-batch.json', 'utf8'));
          batch.orgId = process.env.AX_ORG_ID;
          const response = await fetch(`${audience}/rpc/axp.v1.AxpEvalsService/IngestEvalBatch`, {
            method: 'POST',
            headers: {
              authorization: `Bearer ${token}`,
              'content-type': 'application/json',
              'connect-protocol-version': '1',
            },
            body: JSON.stringify(batch),
            redirect: 'error',
          });
          if (!response.ok) throw new Error(`Upload refused: ${response.status}`);
          JS

The ax CLI does not yet have an eval ingestion command and does not request GitHub OIDC tokens. Use the upload step above for Trusted CI ingestion. Other CLI commands continue to require their existing authentication.

GitHub documents the token permission and environment variables in its OIDC reference.

3. Check access

After an accepted authentication, the rule's Last used value updates. Each use records the rule, workflow run ID, workflow name, commit SHA, ref, environment (when present), event name, and actor. An ingestion validation error can occur after authentication succeeds.

A different repository, repository ID, constrained ref, or constrained environment is refused. Expired tokens, tokens issued more than ten minutes ago, and tokens for another audience are refused (time checks allow 30 seconds of clock skew). Only the organization named in the ingestion request decides access. If that organization has no matching rule, the request is refused, even if another organization trusts the repository.

Several organizations can trust the same repository. Your workflow chooses the destination organization through orgId; each rule grants ingestion only into the organization that created it. Registering a repository in another organization cannot grant access to yours or block your workflows.

To stop access, remove the rule in Integrations → GitHub → Trusted CI or Settings → Trusted CI. Removing a rule keeps its audit history. Turning off the org's ax-trusted-ci flag also stops admission; it retains the rules and audit history.

Accepted run and result rows have source: "ci". Run rows receive the verified GitHub run ID and run URL; result rows have is_ci: true. The server stamps repository, commit SHA, and branch from the verified token wherever those columns exist, overriding submitted values.