Even stronger teachers can teach the wrong thing: evidence, compatibility, and preserving success¶
简体中文 · English
Updated 2026-10-27 · 47 topic entries / 6 representative comparisons
Teacher value is more than aggregate score. OPSD changes training visibility, MOPD and token routing organize multiple teachers, CSD aligns probability events, SPOD protects successful student actions, and VAD isolates corrections attributable to visual evidence. Verify these constraints independently before imitation.
Mechanisms · Trade-offs · Cross-paper questions · Updates
How to read this card
The table compares representative methods. It is not a shared-benchmark leaderboard. Teaching examples are not measured results.
Mechanism comparison¶
Axes: Teacher source: a privileged same-weight view or independent domain experts · Combination granularity: domains, token routing, and successful action protection · Comparable interfaces: shared vocabulary coordinates or common text events · Attribution: privileged evidence, language priors, and existing student skills
| Paper and source | Mechanism | Benefits and trade-offs |
|---|---|---|
| OPSD · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 复现边界 |
Uses the same weights under two conditions: a teacher reads a verified answer, while a student generates its own answer-free prefix. | Extra evidence is training-teacher-only. The local dual-view candidate path does not reproduce Qwen parameter training. |
| MOPD · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 复现边界 |
Trains domain RL teachers independently, then combines domain distributions over the same student rollout. | Reuses expertise but can still cause inter-domain interference in the shared student. A deterministic local mini-suite is not large-model capability fusion. |
| Compatibility Matters: A Framework for Token-Level Teacher Selection in On-Policy Distillation · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 结果、代码映射与边界 |
Selects teachers at each student prefix by confidence, Hellinger distance, or aligned feature distance. Soft routing mixes probabilities arithmetically. | Confidence is not suitability, and features in different dimensions cannot be subtracted directly. Full training was not reproduced; a random-policy L1 component exists. |
| Beyond the Teacher: Understanding and Preserving Distillation Overshoot in Agentic On-Policy Distillation · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 结果、代码映射与边界 |
Protects decisions within successful episodes where teacher raw complete-action probability is lower, canceling distillation there and applying a soft floor. | Other decisions still learn. A soft floor cannot guarantee no regression under shared parameters. Full agent runs were too costly and not reproduced. |
| CSD: Common-Support Distillation for Cross-Tokenizer Knowledge Transfer · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 结果、代码映射与边界 |
Builds common chunk events from character spans and decoded text, using chained probabilities and REST for unmatched mass. | Cross-tokenizer comparison no longer assumes identical IDs mean the same text, but event coverage is limited. Local work verifies small-vocabulary alignment and gradients. |
| VAD · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 复现边界 |
Contrasts evidence-present and evidence-removed views of the same teacher, one-sided projects corrections onto the visual direction, then rebuilds a student-anchored target. | Operational contrasts do not prove all corrections causally visual. Local process-evidence features replace real crops; the original visual model was not run. |
Trade-off analysis¶
Check whether teacher signals describe the same event before mixing¶
MOPD and token routing can mix teachers but differ in domain-level and per-position selection. With different tokenizers, CSD first aligns decoded text events rather than mixing coordinates directly. OPSD can use the same model with extra conditions instead of adding independent experts.
Sources: MOPD · Compatibility Matters: A Framework for Token-Level Teacher Selection in On-Policy Distillation · CSD: Common-Support Distillation for Cross-Tokenizer Knowledge Transfer · OPSD
Teacher corrections should be examined rather than accepted wholesale¶
SPOD protects specific successful actions the teacher disfavors, not whole successful episodes. VAD asks whether a privileged correction comes from visual evidence or shared language bias. Student success and evidence attribution are separate checks even when the teacher scores higher overall.
Sources: Beyond the Teacher: Understanding and Preserving Distillation Overshoot in Agentic On-Policy Distillation · VAD · OPSD
Cross-paper questions¶
Try answering before expanding the explanation. Each question relates at least two papers.
Why isn't same-weight OPSD just letting the student copy verified answers?¶
Show answer and example
Answer: Conditions differ. The student only reads the question and generates its own prefix; the training teacher reads a verified answer before scoring that prefix. Supervision targets an unprivileged student. MOPD instead uses independent domain teachers. Neither can feed training answers into formal student prediction.
Illustrative example: Teaching example: A teacher can know the solution while evaluating the student's next step, but giving it to the student changes the task.
Can token teacher routing average same-ID logits from different tokenizers?¶
Show answer and example
Answer: No. Soft routing mixes probabilities over identical events, not generally logits or log-probabilities. Different tokenizers can assign the same ID to different text. CSD aligns textual events and preserves REST mass first. Fine-grained routing cannot fix incompatible event identity.
Illustrative example: Teaching example: Token 17 could be a whole word for one teacher and a fragment for the student; shared numbering does not make them the same event.
Sources: Compatibility Matters: A Framework for Token-Level Teacher Selection in On-Policy Distillation · CSD: Common-Support Distillation for Cross-Tokenizer Knowledge Transfer · MOPD
Does SPOD freeze a successful episode when the teacher disfavors one action?¶
Show answer and example
Answer: No. Success and raw complete-action probabilities identify qualifying decisions; only there does SPOD remove distillation and add a soft floor. Other decisions keep learning. MOPD and OPSD do not automatically protect this behavior.
Illustrative example: Teaching example: If step two works despite being unusual, protecting it need not discard useful corrections to steps one and three.
Sources: Beyond the Teacher: Understanding and Preserving Distillation Overshoot in Agentic On-Policy Distillation · MOPD · OPSD
Does giving an OPSD teacher more visual evidence prove its corrections are visual?¶
Show answer and example
Answer: No. Language priors, style, and prior prefix mistakes can also change. VAD contrasts evidence presence and removal with the same teacher and student prefix, then projects the original correction. Privileged inputs alone do not supply attribution, and the local proxy is not genuine visual attribution.
Illustrative example: Teaching example: A crop might cause shorter phrasing even if brevity is not supported by the image; compare views to distinguish shared changes.
Update log¶
Updates record new approaches, changed trade-offs, and follow-up evidence.
2026-10-27 · Initial edition: compares privileged self-teachers, multidomain and token routing, CSD text-event alignment, SPOD success protection, and VAD visual attribution; 47 abstracts and six close reads.
Covers 47 topic entries; see the full list below.
All 47 topic entries
The full catalog follows. The main comparison covers the representative papers listed above.
- When to Follow the Teacher: Advantage-Based On-Policy Distillation
- Edit, Don't Imitate: Minimal-Edit On-Policy Distillation for Multimodal Reasoning
- CSD: Common-Support Distillation for Cross-Tokenizer Knowledge Transfer
- Reinforcement Learning with Visually Attributable On-Policy Self-Distillation
- Correctness-Dependent Asymmetric Target Distributions for Self-Distillation Policy Optimization
- Beyond the Teacher: Understanding and Preserving Distillation Overshoot in Agentic On-Policy Distillation
- Tree-Aware Posterior Distillation for Long-Horizon Agentic Reinforcement Learning
- MultiAgent-OPD: End-to-End Training of Hierarchical Multi-Agent Systems with On-Policy Distillation
- Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillation
- Compatibility Matters: A Framework for Token-Level Teacher Selection in On-Policy Distillation
- Training-Free On-Policy Distillation
- Learning from Unverified Peers via Shared Residual Distillation
- OPRD: On-Policy Reminder Distillation for Long-Horizon Coding Agents
- EliteReplay: Preserving Diverse Solutions in RL Post-Training via Quality-Diversity Search
- Condition-Contrastive Extrapolation for On-Policy Distillation of Language Models
- Failure Is the Mother of Success: Recovering Supervision with Grounding Hindsight
- LoOPSD: On-Policy Self-Distillation with Looped Teachers
- Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillation
- CAST: Credit Alignment for Cross-Tokenizer On-Policy Distillation
- Which Skill to Distill? SGUID: Selecting a Compact Skill Bank for Model–Skill Co-Evolution
- MetaOPD: Meta-Learned Token Weighting for On-Policy Distillation
- Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenes
- The Weakest Link: Distilling LLM Reasoning with Worst-Case Constrained Reinforcement Learning
- PR-OPD: Privileged Representation On-policy Self-Distillation for Agentic Reinforcement Learning
- From Dissonance to Orchestration: Teacher Intervention in On-Policy Distillation
- LEGO-OPD: Factorized Teacher Composition for Multimodal On-Policy Distillation
- AdviSD: Learning to Advise Frontier LLMs via Targeted Multi-Turn Self-Distillation
- Learning from Teacher Continuations at Student States
- MAS-OPD: On-Policy Distillation for Multi-Agent Systems
- OPD-Aha: From Linguistic Momentum to Visual Reflection in Multimodal On-Policy Distillation
- Negative Self-Distillation: Learning to Reason by Avoiding Flaws
- CompassOPD: Cross-Family On-Policy Distillation via Within-Family Likelihood Shifts
- Eliciting Weak-to-Strong Generalization with On-Policy Reverse Distillation
- Boosting LLM Exploration via Weak-Model Guidance in RLVR
- Video-OPSD: Exploiting Privileged Visual Evidence for On-Policy Self-Distillation in Video Large Language Models
- Preserving General Capabilities during Domain Specialization with Uncertainty-Calibrated MOPD
- Where to Look Matters: On-Policy Self-Distillation for Long-Video Understanding
- SMOPD: Multi-Reward Reinforcement Learning via Specialize-and-Merge Online Policy Distillation
- Distill Where You Fail: Recovering Learning Signals of Negative RL-Groups from Adaptive Teacher Guidance
- VAD
- Contrastive Reinforced Policy Optimization via Privileged Self-Distillation
- Distilled RL
- ARMOR
- MOPD
- POLARIS: Guiding Small Models to Write Long Stories
- OPCD
- OPSD