SERPO: Self-Evolving Rubric Policy Optimization for Open-Ended Test-Time Reinforcement Learning¶
Candidate-policy objective, not paper-scale training.
Paper information¶
| Field | Details |
|---|---|
| Paper | Paper or original source |
| Institution | No affiliation disclosed; first author Jianze Wang. |
| First public release | 2026-07-29 (arXiv v1) |
| Author code | No public author code found as of 2026-08-24. |
| Local adapter or method key | serpo |
| Local reproduction code | src/auto_research/post_training/historical_b08_b09.py |
Paper method and evidence¶
A Good–Normal–Bad response archive, query-specific rubrics and actor policy co-evolve in a three-way feedback loop.
flowchart LR
A[Archive graded responses] --> B[Update query-specific rubric]
B[Update query-specific rubric] --> C[Update actor policy]Original paper figure¶
Figure shows rubric and policy co-evolution. From the original paper; copyright remains with the authors.
Core equation¶
\[
\theta,rubric,archive\leftarrow\operatorname{Evolve}(\theta,rubric,archive)
\]
Reported results¶
The paper reports up to +20.63/+20.31 points on HealthBench/ResearchQA and +8.06 macro across six benchmarks.
Local reproduction¶
120 updates: accuracy 0.1953→0.3281 (+0.1328). Metrics, Batch index. Before/after arithmetic-smoke results on one candidate policy are not matched comparisons across methods. No full checkpoint, data, compute, multi-seed or benchmark. Raw outputs stay in runs/post-training/; experiment artifacts contain metrics only.