RTPO: Reverse-Turn Policy Optimization for Stabilizing Agentic RL Training¶
Core agent-mechanism mini-suite. Implements independent planning, memory, credit assignment, or topology paths and records process counts on deterministic tasks, not the full external environment.
Paper information¶
| Field | Details |
|---|---|
| Paper | arXiv 2608.18682 |
| Organization | Adelaide University |
| First public release | 2026-08-19 (arXiv v1) |
| Authors' code | No public author code was found as of 2026-08-24. |
| Adapter / method | rtpo |
| Local reproduction code | src/auto_research/agent_research/historical_b10_b11.py |
| First author | Yugu Li |
Original paper¶
Motivation and method¶
Organize multi-turn rollouts as sparse backward trees. Update turns in reverse temporal order so decisions remain on-policy with downstream continuations.
flowchart LR
O[Task and observation] --> P[Planning and state management]
P --> M[RTPO mechanism]
M --> A[Actions and tools]
A --> V[Verification and feedback]
V --> PKey figure from the paper¶
Figure 2: Core architecture, modules, and connections. From the original paper; copyright remains with the authors.
Core equation¶
Paper results¶
Improvements over trajectory-level and turn-level baselines are 21.50% and 10.76%, respectively. These are paper results, not local mini-suite or online conclusions.
Local reproduction¶
On 120 planbench-mini episodes, RTPO has joint success 1.0000 and mean cost 0.6800. The long-context control scores 1.0000 and costs 64.5000. With saturated success, inspect traces, source, and method-specific counts rather than inventing improvement. See the method metrics and batch index.
Reproduction boundaries¶
The deterministic mini-suite does not access real browsers, AppWorld, SWE-bench, or external tool services. It verifies mechanism branches, intermediate states, and reporting contracts. Reproducing full environments requires the paper's resources. Raw runs are in runs/agent-research/; the repository keeps only metrics. See experiments.