Contrastive Reinforced Policy Optimization via Privileged Self-Distillation¶
Candidate-policy objective, not paper-scale training.
Paper information¶
| Field | Details |
|---|---|
| Paper | Paper or original source |
| Institution | No affiliation disclosed; first author Xingjian Wu. |
| First public release | 2026-07-30 (arXiv v1) |
| Author code | No public author code found as of 2026-08-24. |
| Local adapter or method key | crpo |
| Local reproduction code | src/auto_research/post_training/historical_b08_b09.py |
Paper method and evidence¶
Use predictive entropy to distinguish positive reflection/exploration positions from negative exposure-bias positions, then contrast privileged self-distillation signals within groups.
flowchart LR
A[Predictive entropy] --> B[Separate positive and negative positions]
B[Separate positive and negative positions] --> C[Contrast privileged signals]Original paper figure¶
Figure shows the contrastive teacher signal. From the original paper; copyright remains with the authors.
Core equation¶
Reported results¶
The paper reports consistent gains on 13 reasoning and deep-search benchmarks versus RL and self-distillation baselines.
Local reproduction¶
120 updates: accuracy 0.1953→0.6484 (+0.4531). Metrics, Batch index. Before/after arithmetic-smoke results on one candidate policy are not matched comparisons across methods. No full checkpoint, data, compute, multi-seed or benchmark. Raw outputs stay in runs/post-training/; experiment artifacts contain metrics only.