STARE: Surprisal-Guided Token-Level Advantage Reweighting¶
Core mechanism evidence. Paper and local evaluations are separate.
Paper information¶
| Field | Value |
|---|---|
| Paper | STARE: Surprisal-Guided Token-Level Advantage Reweighting (arXiv 2606.19236) |
| Affiliations | Shenzhen International Graduate School, Tsinghua University / Tencent Hunyuan |
| First public release | 2026-06-17 (arXiv v1) |
| Authors' code | Author code. |
| Adapter / method | stare |
| Local code | src/auto_research/post_training/ |
Original paper¶
Background and contribution¶
Identify entropy-critical tokens by batch surprisal quantiles, reweight their advantages, and use target entropy in a feedback gate.
flowchart LR
S[Token surprisal] --> Q[Batch quantiles] --> W[Advantage reweighting]
H[Entropy error] --> WKey figure from the paper¶
Original Figure 2: STARE. From the paper; copyright belongs to the authors. Not a local result.
Core mechanism¶
\[
\Delta H_t\approx A(\tau)\,g_\theta(x_t),\quad \tilde A_t=w(s_t,H-H^*)A(\tau).
\]
Paper results¶
The paper maintains entropy from 1.5B to 32B across short/long CoT and multi-turn tools; average AIME24/25 accuracy exceeds DAPO by 4%–8%. No live A/B.
Local reproduction¶
Arithmetic candidate suite, 120 steps, seed 42: 0.2344→0.5312 (+126.67%). Metrics.
auto-research post-train --algorithm stare --dataset arithmetic-smoke --maximum-examples 256 --steps 120 --seed 42
auto-research evolve --model post-training --dataset arithmetic-smoke --direction "Combine stare with installed paper operators" --generations 2 --population 4
Reproduction limits¶
Mechanism diagnostic, not the large model, multi-card, private environment, or full benchmark.