Docker in Sandbox
Run Docker containers inside an experiment's sandbox.
Docker is installed in every sandbox and the daemon is running before setup starts, so an experiment uses Docker the way it would on a laptop:
docker run -d -p 8080:80 nginx
curl -sS http://localhost:8080
docker compose up -d
docker build -t my-image .There is nothing to install, no daemon to start, and no special flags to pass. docker runs as the sandbox user, so sudo is not needed either.
What the sandbox gives you
| Tooling | Docker Engine 29 with the buildx and compose plugins |
| Networking | Stock bridge networking: -p publishes to the sandbox's localhost, containers on a user-defined network resolve each other by name, and containers reach the internet |
| Lifetime | The daemon starts before setup and stays up through setup_checks, the agent, and tests |
| Daemon log | /var/log/dockerd.log |
Containers the agent leaves running are still running when tests execute, so a test can talk to a service the agent started:
tests:
- name: service-answers-on-published-port
script: curl -fsS http://localhost:8080/health
- name: container-still-up
script: docker ps --filter name=api --filter status=running --quiet | grep -q .Cost of images
Each sandbox starts with an empty image store, so the first docker run or docker build in a variant pays for the pull. Account for that in limits.max_time_seconds, and prefer small images (alpine, -slim) for the parts of the task that are not what you are measuring.
Validate a Docker-heavy setup before spending tokens: ax experiment run <experiment.yaml> --setup-only runs staging, setup, setup_checks, and teardown once per distinct setup, with no agent and no tests. (--mock runs the full lifecycle, tests included, with a no-op agent.) setup runs under set -euo pipefail, so the first failing command, unset variable, or failed pipeline stage aborts the variant.
Complete example
A runnable experiment where the agent containerizes a service and the tests check the running container:
schema_version: 2
id: docker-in-sandbox
name: "Docker in sandbox: agent runs a published service"
agents: [claude]
models: [claude-sonnet-4-6]
prompts:
- id: serve-in-container
prompt: |
Docker is installed and the daemon is running. Start the `nginx:alpine`
image as a container named `web`, publish it on port 8080 of this
machine, and confirm it answers with `curl http://localhost:8080`.
Write the response body to /workspace/published-port.txt.
environments:
- name: docker-host
setup:
- name: prefetch-image
script: docker pull nginx:alpine
tests:
- name: published-port-answered
script: grep -qi "welcome to nginx" /workspace/published-port.txt
- name: container-is-running
script: docker ps --filter name=web --filter status=running --quiet | grep -q .
limits:
max_time_seconds: 1200