Output & Scripting

The stable JSON contract, piping into jq, and composing the verification loop in a script or CI job.

The CLI is built to be driven by scripts and agents, not just typed by hand. Two guarantees make that safe: a stable JSON output contract and stable exit codes. Together they let you compose the full loop without ever parsing human-readable text.

The JSON contract

--output json is the stable, machine-readable contract — the shape an agent or script can depend on across releases. Changes are additive: new fields may appear, but existing fields and exit codes won’t change within a major version. --output text is human-friendly and may change freely. When in doubt, script against JSON.

testsprite project list --output json

Every command also runs under --dry-run, which emits canned data whose shape matches the live response — perfect for writing and testing a script before you wire up auth.

testsprite --dry-run test run example_test --wait --output json

Piping into jq

JSON output pipes straight into jq or an LLM:

# Grab the runId of a freshly triggered run
RUN_ID=$(testsprite test run test_3a9f21c7 --output json | jq -r '.run.runId')

# Wait on it and branch on the exit code
testsprite test wait "$RUN_ID" --timeout 600 --output json || echo "run did not pass"

Pull just the analysis fields an agent needs to triage a failure:

testsprite test result test_3a9f21c7 --include-analysis --output json \
  | jq '{hypothesis: .analysis.rootCauseHypothesis, fix: .analysis.recommendedFixTarget}'

Branching on exit codes

The CLI’s exit codes are stable and meaningful, so scripts can branch without parsing output at all:

testsprite test rerun test_3a9f21c7 --wait --output json
case $? in
  0)  echo "passed ✅" ;;
  1)  echo "failed — pulling the bundle"; testsprite test failure get test_3a9f21c7 --out ./.testsprite/failure ;;
  7)  echo "timed out — resume with: testsprite test wait <run-id>" ;;
  12) echo "out of credits" ;;
  *)  echo "other error" ;;
esac

Timeouts still leave you something to parse

When a --wait poll hits its --timeout ceiling, the exit code is 7 — but the CLI still writes a partial result object to stdout before exiting, so a script never loses the runId on the unhappy path:

{ "runId": "run_5c1d9a2b", "status": "running" }

testsprite test run test_3a9f21c7 --wait --timeout 30 --output json > result.json
# exit 7 — result.json still has {"runId": "run_5c1d9a2b", "status": "running"}
testsprite test wait "$(jq -r .runId result.json)" --output json

A minimal CI gate

#!/usr/bin/env bash
set -euo pipefail
# Auth comes entirely from the environment — the credentials file is never touched.
export TESTSPRITE_API_KEY="$TESTSPRITE_API_KEY"

# Replay the whole suite against the deployed preview URL and fail the build on any non-pass.
testsprite test rerun --all --project "$PROJECT_ID" \
  --wait --timeout 900 --max-concurrency 4 --output json

CI/CD

A full GitHub Actions workflow — including failure-artifact upload and rate-limit handling

For agents, testsprite agent install writes skill files that already encode these patterns — exit codes, the failure-bundle layout, and the create → run → failure get → fix → rerun loop. See Agent Integration.

Debugging a script

• --verbose — human-readable retry / backoff / polling messages to stderr.
• --debug — HTTP method, path, request id, latency, and retry decisions to stderr. The API key is never included.

Both write to stderr, so they never pollute the JSON on stdout that you’re piping into jq. The same stdout-purity rule holds everywhere in the CLI: interactive prompts (the API key prompt in testsprite setup, the --target prompt in agent install) and any onboarding preamble also print to stderr, so stdout stays a single parseable payload even in an interactive terminal session.

Once every 24 hours, an interactive run also probes npm for a newer CLI version and, if one exists, prints a one-line notice to stderr. It’s automatically skipped in CI, when stderr isn’t a TTY, under --output json, and under --dry-run — disable it explicitly with TESTSPRITE_NO_UPDATE_NOTIFIER=1.

Where to Go Next

Exit Codes & Errors

The full branching contract for scripts and agents

Configuration

Profiles, environment variables, and global flags

CI/CD

A complete GitHub Actions pipeline

Command Reference

Every command, flag, and output shape