RRC: Ranking-Based Reward Construction for Generative Reward Models¶
Core reward operator mini-suite, not full paper reproduction.
Paper information¶
| Field | Details |
|---|---|
| Paper | Paper or original source |
| Institution | Northeastern University. |
| First public release | 2026-08-06 (arXiv v1) |
| Author code | wangclnlp/RRC. |
| Local adapter or method key | rrc |
| Local reproduction code | src/auto_research/post_training/latest_20260809.py |
Paper method and evidence¶
Generative reward models compare outputs better than they assign independent scalars. RRC constructs centered reward with group self-competition and a small anchor ranking.
flowchart LR
A[Group candidates] --> B[Rank by self-competition and anchors]
B[Rank by self-competition and anchors] --> C[Compute centered reward]Original paper figure¶
The original figure shows ranking-based reward construction. From the original paper; copyright remains with the authors.
Core equation¶
\(r_i=\frac{\operatorname{rank}(y_i)-\bar r}{K-1}+\lambda r_i^{anchor}\).
Reported results¶
AlpacaEval2 rises from 35.8% to 41.3%; ArenaHardV2 from 8.0% to 11.2% in the paper.
Local reproduction¶
Compare with the same public mini-suite without RRC. Arithmetic metrics; no original model, proprietary environment or expensive judge. Evolve candidate discovery includes RRC; combinations remain bounded by public evaluations. No raw runs or checkpoints committed.