Upload a task
Uploading is optional. An uploaded task is a test in your workspace: every run of it is graded and kept with every call, anyone in the workspace can run it, and the Runs board compares versions of your agent on it. It is also the only way to add a test.
Hosted workspaces. A hosted workspace does not run uploaded code yet: a task with
tests/verify.py or solution/solve.py is refused there. Judge it with
tests/checks.json; doing nothing failing is its proof.
Prove it, then import it
Point sw task import at the folder. --check runs the proofs and keeps nothing; without it, a task
that passes them is kept as a test.
sw task import ./refund-digest --check # prove it; nothing is kept
sw task import ./refund-digest # prove it and keep it as a testimport stateweave
client = stateweave.Client()
print(client.import_task("./refund-digest", check_only=True)["proofs"])
kept = client.import_task("./refund-digest")
print(kept["test"]["id"])curl -s -X POST "$STATEWEAVE_URL/api/v1/sandboxes/p-6a129c/tasks" \
-H "Authorization: Bearer $STATEWEAVE_API_KEY" -H "Content-Type: application/json" \
-d '{"files": {"instruction.md": "…", "task.toml": "…", "tests/checks.json": "…"}, "check_only": true}' ✓ Doing nothing fails
with nothing done, judged failed:
- failed: posted to #support: messages[*].text: no value contains '#1042' (found 'Refunds go out on Fridays')
- failed: posted once: the records: wanted 1, found 0; request.channel: nothing there to compare with 'C0SUPPORT'
Kept as t-4f2a9c: Post this week's refunds to #support (sw run t-4f2a9c --agent … -- …)The proofs
StateWeave runs these on fresh copies of the sandbox's latest version. A task is kept only when every one holds:
| Proof | What it shows |
|---|---|
| The answer format gives no answer away | For an answer task: the format shows the shape and none of the values the checks look for. |
| Doing nothing fails | The checks don't hold on the untouched sandbox. A check that already holds there proves nothing. |
A task with code, where that is accepted, must also load and its reference solution must pass: see Code in a task.
When it is refused
A task that cannot be read answers 400 with problems, every one found. A task that fails
a proof answers 400 with proofs, each saying what it found. Kept, it answers
201 with the test. Your coding agent can do all of this: sw setup gives it the
stateweave-use skill, which writes, proves and uploads tasks with sw task import (see
From your coding agent).
After it is kept
- Run it.
sw run t-4f2a9c --agent my-agent@0.1 -- python my_agent.py, from Python or over HTTP: see Test runs. - Change it. Edit the folder and import it again: that makes a new test. The page of an uploaded test shows its files and an Uploaded badge; it is not edited in the browser.
- A new version of the sandbox. When you switch the sandbox to test mode with a new version that changed one of the test's apps, the test needs a re-check, and runs of it are refused (HTTP 409) until one passes. Press Re-check on its page, or:
sw task recheck p-6a129cA re-check proves the task again on the new version. If it still holds, it runs as before. If not, it stays as it is with the proofs that failed: fix the folder and import it again.
List tests
sw task ls --env p-041ba8for test in client.tests(environment="p-041ba8"):
print(test["id"], test["kind"], test["apps"], test["title"])A test's listing includes its prompt, kind, apps,
answer_format, persona and how many faults it injects. It never includes the checks or the
reference solution.
Limits
A hosted workspace has a daily limit on tests. Each task kept counts one; --check counts none. Past the
limit, an import is refused with HTTP 429.
Older tests
A workspace from before tasks were only uploaded may still hold tests StateWeave wrote for it. They stay and run as before, with their faults. On such a test's page you can still edit its request or criteria, unless it has random or pinned faults: the test is rebuilt and checked again as its next version. When a new version changes their data, the re-check rewrites them for it, and each one rewritten counts toward the daily limit.