GRIN: Continual knowledge uptake through mixed-policy RL¶
Core candidate mini-suite, not paper-scale continual training.
Paper information¶
| Field | Details |
|---|---|
| Paper | Paper or original source |
| Institution | University of California, Merced (first author). |
| First public release | 2026-08-26 (arXiv v1) |
| Author code | No public author code found as of 2026-08-28. |
| Local adapter or method key | grin |
| Local reproduction code | src/auto_research/post_training/latest_20260827.py |
Paper method and evidence¶
When on-policy sampling rarely finds new correct answers, inject golden responses into failing groups and train with mixed-policy importance correction. Return to on-policy exploration when performance improves.
flowchart LR
A[Failed on-policy group] --> B[Inject golden response]
B[Inject golden response] --> C[Importance-correct mixed update]Original paper figure¶
Figure shows gold injection and mixed-policy updates. From the original paper; copyright remains with the authors.
Core equation¶
\[
A(y)=\rho(y)\,[R(y)-b],\qquad \rho(y)=\frac{\pi_\theta(y|x)}{q_{mix}(y|x)}.
\]
Reported results¶
Llama3.2-3B COUNTER mean accuracy 43.65%, fail@k 24.05%, outperforming SFT and mixed-policy baselines. No industrial A/B.
Local reproduction¶
auto-research post-train --algorithm grin --dataset arithmetic-smoke --steps 100 --seed 42 --offline
Arithmetic-smoke 100 steps 0.1250→0.3438 (+175.00%); gold is a local candidate oracle, not a capability claim. Metrics, batch index. No paper checkpoint, private data, service or benchmark.