SPADE: Self-Play in Adaptive Synthetic Executable Environments¶
Core agent-mechanism mini-suite. Implements independent planning, memory, credit assignment, or topology paths and records process counts on deterministic tasks, not the full external environment.
Paper information¶
| Field | Details |
|---|---|
| Paper | arXiv 2608.19197 |
| Organization | University of Washington |
| First public release | 2026-08-19 (arXiv v1) |
| Authors' code | spade |
| Adapter / method | spade |
| Local reproduction code | src/auto_research/agent_research/historical_b10_b11.py |
| First author | Bo Liu |
Original paper¶
Motivation and method¶
The same LLM plays environment designer and reasoning agent. Regret between runs with and without privileged hints trains it to produce executable environments at the capability boundary.
flowchart LR
O[Task and observation] --> P[Planning and state management]
P --> M[SPADE mechanism]
M --> A[Actions and tools]
A --> V[Verification and feedback]
V --> PKey figure from the paper¶
Figure 2: Overview of the paper's method and key components. From the original paper; copyright remains with the authors.
Core equation¶
Paper results¶
At 30B, the mean gain over eight held-out tasks is +5.3; BFCL v4 gains +5.7 and ACEBench-Agent +13.9. These are paper results, not local mini-suite or online conclusions.
Local reproduction¶
On 120 planbench-mini episodes, spade has joint success 1.0000 and average cost 0.7400. The long-context control has 1.0000 and 64.5000. With saturated success, inspect traces, source, and method-specific counts rather than inventing improvement. See the method metrics and batch index.
Reproduction boundaries¶
The deterministic mini-suite does not access real browsers, AppWorld, SWE-bench, or external tool services. It checks mechanism branches, intermediate state, and reporting contracts. Reproducing full environments requires the paper's resources. Raw runs live in runs/agent-research/; the repository keeps only metrics. See experiments.