GiGPO: Group-in-group relative advantages for agents¶

Isolated RL credit mechanism on a deterministic agent mini-suite.

Paper information¶

Field	Details
Paper	GiGPO (arXiv 2505.10978)
Institution	Nanyang Technological University / Skywork AI.
First public release	2025-05-16
Author code	No official author code found.
Local adapter or method key	`gigpo`
Local reproduction code	`src/auto_research/agent_research/`

Paper method and evidence¶

Sparse final rewards and whole-trajectory relative advantages cannot distinguish useful environment steps. Compute a macro advantage across full trajectories and a micro advantage across steps sharing an anchor state.

flowchart LR
  R[Full rollout groups] --> M[Macro advantage]
  R --> S[Group steps by anchor]
  S --> I[Micro advantage]
  M --> A[Combine credit]
  I --> A

Original paper figure¶

The original figure contrasts trajectory and step credit. From the original paper; copyright remains with the authors.

Core equation¶

\[ A^{\rm GiGPO}_{t}=A^{\rm macro}(\tau)+A^{\rm micro}(s_t,a_t),\quad A^{\rm micro}=r(s_t,a_t)-\operatorname{mean}_{a\in\mathcal G(s_t)}r(s_t,a). \]

Reported results¶

The paper reports gains over GRPO on ALFWorld, WebShop, and search-augmented QA, including over 12% on ALFWorld and 9% on WebShop. No online A/B.

Local reproduction¶

Explicitly construct full trajectory groups and step groups in PlanBench mini and record inter-/intra-group advantages, rollout, and turn credit.

auto-research agent-eval --method gigpo --benchmark planbench-mini --episodes 120 --seed 42

Fixed-seed summary. Deterministic tasks have no real cross-trajectory environment-state convergence; shared plan steps serve as auditable anchors.