GRAFT: Exchange all-fail rollout groups across models

简体中文

L1 objective mechanism: peer compatibility gating, source-advantage preservation, and token ratio clipping.

Paper information

Field	Details
Paper	arXiv v1
Institution	KAIST (first author).
First public release	2026-09-29 (arXiv v1)
Author code	No author repository found as of 2026-09-30.
Local adapter or method key	graft
Local reproduction code	src/auto_research/post_training/latest_20260930_followup.py

Paper method and evidence

When the receiver's group is entirely wrong and a peer has both successful and failed trajectories on the same prompt, replace the uninformative group with the complete peer group and retain its internal advantages. Sequence-mean log-likelihood compatibility downweights cross-tokenizer policy mismatch; token-level importance ratio clipping bounds updates.

Original paper figure

Figure 1 shows heterogeneous models' complementary successes on all-fail prompts. From the original paper; copyright remains with the authors.

Local reproduction

Tests verify lower weight for incompatible peers, clipping of out-of-range ratios, and no gradient into source advantages. See the three-seed mechanism results. No two-model GRPO training or five-benchmark math evaluation; the paper's mean +2.1-point gain was not reproduced.