Core concepts

A few ideas cover all of StateWeave: a workspace holds scenarios, a scenario holds apps and tests, and every run of a test gets fresh copies of the apps.

#Workspace

Your workspace is your team's StateWeave, at its own address such as https://acme.app.stateweave.net. You sign in to it in a browser to build scenarios, write tests and look at runs. Harnesses reach it with API keys (sw_…), which you create and revoke under Connect an agent. A key works for everything in the workspace, and it is shown only once.

#Scenario

A scenario is a company that never existed, generated from a description you write: its people and departments, its customers and partners, a few weeks of history, and the data each of its apps holds. All the apps tell the same story. The customer who wrote to support in Zendesk has orders in Shopify, a deal in Salesforce and a thread in the sales channel in Slack.

The sw command calls scenarios environments (sw env), and the API calls them projects. Their ids start with p-. See Scenarios.

#Apps

Each app in a scenario is a working twin of the real service: Slack, Gmail, Google Calendar, Google Sheets, Google Drive, Salesforce, Shopify or Zendesk. A twin answers the vendor's real API at its own address, so your agent uses the same clients, tools and code it would use in production. Writes stick, searches find what was written, and errors come back the way the real service sends them. See Apps and credentials.

#Test

A test is one request your agent could be given in the scenario's company, as a person would put it, plus what a right outcome is. There are two kinds:

Every test is checked before it is kept: a reference solution must pass it, an agent that does nothing must fail it, and a near miss (a plausible but wrong solution) must fail it too. So a test that passes everything never gets in. The sw command calls tests tasks (sw task). Their ids start with t-. See Tests.

A test can also inject faults: an outage, a 401 for a valid token, a page cut off mid-response. Your agent then has to cope the way it would have to in production. See Faults.

#Persona

Every test is asked on behalf of someone in the company, the persona. The run's credentials are theirs: the agent reads their inbox, posts as them in Slack and owns their Salesforce records. Some apps also hand over the company's bot or app tokens, as a real deployment would have them.

#Run

A run is one attempt by your agent at one test. Starting a run copies the test's apps from the scenario, so every run starts from exactly the same state and nothing one run does reaches another. The apps stay up until the run is finished, given up or out of time (30 minutes unless you set another limit). Run ids start with r-.

#Agent label

Each run carries the label of the agent that ran it, such as support-bot@0.4. The Runs board shows labels side by side, test by test, so you see whether a new prompt, model or tool helped. You can also attach meta to a run (the model, the prompt version, a commit) and it is shown with it.

#Worker

A worker is your harness waiting for runs: sw worker keeps asking your workspace for runs someone queued with Run again on a run's page, and drives each one. It lets you rerun a failed test from the browser while the agent runs on your machine. See Run again.

#Names at a glance