The Agent Loop

How your coding agent creates, runs, reads, fixes, and reruns tests, and how coverage compounds with every pass.

Verification should be a loop your coding agent runs continuously, not a gate at the end. Each time the agent changes code, it verifies the behavior it touched and banks the result.

1. Not covered → create

   Describe the behavior as a plan file (frontend) or code (backend), then create and run it:

   testsprite test create \
     --project proj_8f0f6 --type frontend \
     --plan-from ./checkout-flow.plan.json \
     --run --wait --output json
   

   --run --wait chains create, trigger, and poll. Exit 0 means the test passed and is banked.

2. Already covered → rerun

   testsprite test rerun --all --project proj_8f0f6 --wait
   

   A frontend rerun has auto-heal on by default: if the app changed, the test is repaired and the repaired version saved. For a strict regression gate that replays the saved script exactly, add --no-auto-heal. A rerun is billed the same as a fresh run, so rerun the relevant suite after every significant change.

3. Failed → read the bundle, fix, rerun

   On exit 1, pull the failure bundle, fix the code, and rerun:

   mkdir -p .testsprite
   testsprite test failure get test_3a9f21c7 --out ./.testsprite/failure
   testsprite test rerun test_3a9f21c7 --wait
   

   A test that has never passed, or has only run with --local, has no saved code, so there is no bundle (exit 4, reason no_code). Use testsprite test steps <test-id> --run-id <run-id> and testsprite test failure summary <test-id> instead.

Why this design works

One self-consistent bundle

Every artifact from test failure get and test artifact get shares one snapshotId, and the CLI refuses to stitch data across runs or code versions. The agent never “fixes” the wrong thing from mixed context.

Coverage compounds

Every passing test joins a durable suite, so “already covered?” is answered by replayable tests rather than the agent’s memory, and regressions are caught on the next run.

Real browsers and APIs

Tests run against your live product, with nothing to install on your side. A run uses the project’s Default environment unless you pass --env <name>. To test an app on your machine, see Environments.

Machine-readable contract

--output json and stable exit codes let the agent branch without defensive parsing. See Output & Scripting.

Safe retries

project create, test create, test run, and test rerun carry an idempotency key (random per invocation), so retrying never duplicates a project, test, or run. Pin your own with --idempotency-key my-agent-step-42. Replacing backend code is guarded by a codeVersion token; see Editing & Deleting Tests.

Run-scoped vs latest

test failure get follows the latest failing run, which can shift if a Portal or scheduled run lands mid-loop; test artifact get <run-id> is pinned to one run. In agent loops and CI, capture the runId from --output json and use test artifact get. See Reading Results.

Where the CLI fits

The CLI is one of three surfaces over the same backend; see One Platform, Three Surfaces. It covers the test lifecycle, schedules, and test lists (subject to your plan). Billing, site discovery, and per-step regeneration stay in the Web Portal.

Where to Go Next

Key Terms

Core concepts defined

Quickstart

Your first test end to end

Running Tests

Triggering runs and handling exit codes

Agent Integration

Let your coding agent drive the loop