RIDE: Extrapolate along the RL teacher's representation residual¶
L1 representation target: layerwise base-to-RL extrapolation and masked regression.
Paper information¶
| Field | Details |
|---|---|
| Paper | arXiv v1 |
| Institution | No affiliation listed in the paper. |
| First public release | 2026-09-29 (arXiv v1) |
| Author code | xixixixixxxx/RIDE. |
| Local adapter or method key | ride-opd |
| Local reproduction code | src/auto_research/post_training/latest_20260930_followup.py |
Paper method and evidence¶
Instead of amplifying the teacher–student output probability ratio, take the hidden-state residual from each token/layer of the RL teacher relative to its pre-RL checkpoint. The student regresses to a point beyond the teacher; detach teacher and base so the target branch cannot be updated.
Original paper figure¶
The first-page overview contrasts output-space and representation-space extrapolation. From the original paper; copyright remains with the authors.
Core equation¶
\(r=h_T-h_B\) and target \(h_B+\alpha r\); \(\alpha>1\) extends beyond the teacher.
Local reproduction¶
ride_target and ride_loss test the \(α=1\) special case, \(α>1\) extrapolation, the token mask, and teacher stop-gradient. See the three-seed mechanism results. No four real base/RL checkpoint pairs were loaded, so this does not reproduce the paper's outcomes.