Connect your agent

A run hands your agent three things: the request, the apps' addresses and credentials, and a place to put its answer. This page covers how each one reaches it, from any language.

#The run lifecycle

1. Start. Your harness starts a run of a test. StateWeave copies the test's apps, starts them, and answers with the request and each app's connection details. This takes a second or two.
2. Work. Your agent calls the apps. Every call is recorded on the run.
3. Finish. Your harness hands in the answer (nothing, for an action test). StateWeave grades the run on the apps' end state and the answer, stops the apps and returns the result.

A run that is never finished ends as timed_out when its time limit passes, and one your harness gives up ends as cancelled. Neither is graded.

#With sw run

sw run does the start and finish for you around any command. The command can be written in any language; it only needs to read environment variables and, for an answer test, write a file.

sw run t-bde537 --agent my-agent@0.1 -- python my_agent.py

Everything after -- is your command, run as is (no shell, unless you call one). It gets these variables on top of your own environment:

The run is graded when your command exits, whatever its exit code: an agent that did the work and then crashed is still graded on what it did. Press Ctrl+C to give the run up instead. sw run itself exits with 0 when the run passed and 1 otherwise. A command that doesn't exist is refused before any run starts.

#A minimal agent

The skeleton below reads what a run hands over and answers. Replace the middle with your agent.

import json
import os

run = json.load(open(os.environ["STATEWEAVE_RUN_FILE"]))
prompt = os.environ["STATEWEAVE_PROMPT"]

for name, app in run["apps"].items():
    print(name, app["base_url"], list(app["credentials"]))

answer = ...  # your agent works here: call the apps, decide, act

if os.environ["STATEWEAVE_KIND"] == "answer":
    with open(os.environ["STATEWEAVE_ANSWER_FILE"], "w") as f:
        json.dump(answer, f)

#The run file

The run file holds the run as the API returns it. The parts your agent needs:

{
  "id": "r-de238618",
  "test": "t-bde537",
  "kind": "answer",
  "prompt": "Ethan Smith looks like he's been buying from us under two different email addresses. Can you work out how much he has actually paid us in total across both of his accounts? …",
  "answer_format": "{\"total_paid_usd\": 123.45, \"orders\": [\"NG-1234\", \"NG-1235\"]}",
  "persona": {"name": "Priya Castellano", "handle": "priya.castellano", "title": "Customer Operations Coordinator"},
  "expires_at": 1793670877.43,
  "apps": {
    "shopify": {
      "base_url": "https://t-q8m2xk.app.stateweave.net",
      "endpoints": {
        "admin_graphql": "https://t-q8m2xk.app.stateweave.net/admin/api/2026-10/graphql.json",
        "token_endpoint": "https://t-q8m2xk.app.stateweave.net/admin/oauth/access_token",
        "mcp": "https://t-q8m2xk.app.stateweave.net/mcp"
      },
      "credentials": {
        "app:Northstar Ops Coordinator": "shpat_…",
        "setup:priya.castellano": "setup_…"
      },
      "mcp": {
        "url": "https://t-q8m2xk.app.stateweave.net/mcp",
        "headers": {"Authorization": "Bearer setup_…"}
      }
    }
  }
}

#Credentials

Credential labels are kind:name. The kind says whose token it is and what it opens:

A run never hands over a colleague's mailbox, a client secret or a refresh token: your agent gets what a deployed agent acting for the persona would get. With the Python SDK, run.credential("slack", "user") returns the first credential of a kind.

#Answers

For an answer test, answer with a JSON value in the shape answer_format shows: an object as an object, numbers as numbers, the same field names. Plain text is accepted too and handed in as a string, so a test that asks for an object fails it.

For an action test there is nothing to hand in. The result is what your agent changed in the apps, and what it left alone: a test can fail because the agent changed something it shouldn't have.

#From Python

Inside a Python harness, use the SDK instead of sw run. The with block gives the run up if your agent raises or the block ends without finish(), so no run is left waiting for its time limit.

import stateweave

client = stateweave.Client()  # STATEWEAVE_URL and STATEWEAVE_API_KEY, or what sw login kept

with client.start("t-bde537", agent="my-agent@0.1", meta={"model": "my-model", "prompt": "v7"}) as run:
    token = run.credential("shopify", "app")
    graphql = run.apps["shopify"]["endpoints"]["admin_graphql"]
    answer = my_agent(run.prompt, graphql, token)
    result = run.finish(answer)

print(result.status, result.reasons, result.url)

See the Python SDK reference.

#From any language, over HTTP

The same flow is three calls with your API key. See the REST API reference.

# start: the response is the run file above
curl -s https://acme.app.stateweave.net/api/v1/runs \
  -H "Authorization: Bearer $STATEWEAVE_API_KEY" -H "Content-Type: application/json" \
  -d '{"test": "t-bde537", "agent": "my-agent@0.1"}'

# … your agent works …

# finish: hand in the answer and get the result
curl -s https://acme.app.stateweave.net/api/v1/runs/r-de238618/finish \
  -H "Authorization: Bearer $STATEWEAVE_API_KEY" -H "Content-Type: application/json" \
  -d '{"answer": {"total_paid_usd": 349.10, "orders": ["NG-4824", "NG-4872", "NG-4881"]}}'

#Time limits

A run has 30 minutes by default. Set another limit, from 60 seconds to 24 hours, with --timeout (in seconds) or timeout= in Python. A run still going at its limit ends as timed_out and its apps stop.

#Labels and metadata

--agent (agent= in Python) is the label your runs are compared by on the Runs board. Use one per version of your agent, such as support-bot@0.4, up to 80 characters. Add meta, a JSON object of up to 4,000 characters, to record anything else with the run: the model, the prompt version, a git commit. sw run records your command line in it.

#Many tests in a row

Run a scenario's tests one after another from a shell, or from Python:

for t in $(sw task ls --env p-041ba8 | cut -d' ' -f1); do
  sw run "$t" --agent my-agent@0.2 -- python my_agent.py
done

import stateweave

client = stateweave.Client()
for test in client.tests(project="p-041ba8"):
    with client.start(test["id"], agent="my-agent@0.2") as run:
        result = run.finish(my_agent(run.prompt, run.apps))
        print(test["id"], result.status)

Each run gets its own apps, so you can also run tests in parallel from separate processes, up to your workspace's limit on runs going at once. Past it, starting a run answers HTTP 429 until one finishes.