ERILS: Beyond On-Policy Exploration with External Rollouts for Diffusion LMs¶
Core candidate-policy objective, not original diffusion LM training.
Paper information¶
| Field | Details |
|---|---|
| Paper | Paper or original source |
| Institution | Seoul National University; first author Wonseok Lee. |
| First public release | 2026-08-03 (arXiv v1) |
| Author code | No public author code found as of 2026-08-24. |
| Local adapter or method key | erils |
| Local reproduction code | src/auto_research/post_training/historical_b08_b09.py |
Paper method and evidence¶
Cap external-policy rollout length and process rewards separately by on-policy/external source to avoid collapse from joint normalization.
flowchart LR
A[Cap external rollout length] --> B[Track rollout source]
B[Track rollout source] --> C[Normalize rewards separately]Original paper figure¶
The original figure shows combining rollout sources. From the original paper; copyright remains with the authors.
Core equation¶
\[
A_i=\operatorname{Norm}_{source(i)}(R_i),\quad |y_i|\le L
\]
Reported results¶
Sudoku best-of-4 98.4% versus strong baseline 40.3%; deterministic accuracy around 90% across generation lengths. Paper evidence only.
Local reproduction¶
120 updates on arithmetic-smoke: 0.1953→0.5703 (+0.3750). Metrics, batch index. Before/after on one policy is not a matched comparison. No checkpoint, full data/compute, multi-seed or benchmark. Only metrics are committed; raw outputs stay in runs/post-training/.