LLM post-training by institution, company, or university

简体中文 · English

Grouped by the first listed affiliation of the first author, with newest first within each group. Each paper includes the first author's name and a short method description. Joint work is not repeated under every collaborator.

AMAP, Alibaba Group

• 2026-08-24 · First author: Xianlei Zhou · Beyond the Stability-Exploration Dilemma: Environmental Regularization for LLM Policy Optimization (erpo): ERPO observes that the induced query distribution also drifts during RL, so it puts regularization on the input side. Frozen-reference query likelihood gives static query weights; Query-KL controls environment-distribution drift without directly constraining the response score function.

Affiliation not listed in the paper

• 2026-08-27 · First author: Ivan Kruzhilov · Disentangling Optimization Scale from Preference Scale in DPO (normalized-dpo): Standard DPO beta changes preference noise and gradient size simultaneously, implicitly scaling learning rate. Divide a centered softplus by beta to preserve the minimizer and keep nonvanishing gradient in the beta-to-zero limit.

Alibaba / Qwen

• 2026-08-04 · First author: Wen Wang · SMOPD: Multi-Reward Reinforcement Learning via Specialize-and-Merge Online Policy Distillation (smopd): Train specialist teachers with reward priorities, then merge their capabilities on the student's own trajectories.

Alibaba Cloud Computing

• 2026-10-15 · First author: Bingxi Hou · Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillation (waypoint-opd): Student top-k chooses the candidates. Early and late shifts are centered by the student weights on those candidates; the positive cosine then scales the endpoint credit. Bounded component only; full experiments are not reproduced.

Alibaba DAMO Academy

• 2023-04-11 · First author: Zheng Yuan · RRHF (rrhf): PPO-RLHF coordinates policy, old policy, reward and value models, making training and tuning complex. RRHF samples responses from several models or humans, uses reward to rank them all, aligns the model's average log-likelihood order with reward order, and applies SFT to the highest-quality answer.

Alibaba Group

• 2026-10-16 · First author: Yuhan Sun · MultiAgent-OPD: End-to-End Training of Hierarchical Multi-Agent Systems with On-Policy Distillation (multiagent-opd): The Leader learns from team outcomes; the Answerer learns from its own candidate's correctness. See Section 3. Not reproduced due to cost (review complete); research material only, no executable entry.
• 2026-10-09 · First author: Zhentao Zhang · State-OPD: State-Grounded On-Policy Distillation for Multi-Turn Agents (state-opd): Full-vocabulary entropy selects states for extra complete reasoning-and-action samples, whose actions are not executed. The implementation keeps Appendix D.1's actual sampled-K1 dual clipping, not a scoring heuristic.
• 2026-10-08 · First author: Fei Ding · When KL Regularization Misfires in Group Policy Optimization (zcpo): Independent KL penalties may update policy despite tied rewards or canceling reward gradients. Center response-average conditional KL within the group and put it in the frozen reward coefficient, not a standalone loss.
• 2026-10-01 · First author: Yihua Zhu · My FAULT: Self-Diagnosis as Credit Assignment in Self-Evolving Agentic Reinforcement Learning (my-fault): Only claims verified by evidence enter online error pricing. Learned relative costs redistribute terminal credit to steps while conserving its total.
• 2026-09-15 · First author: Zishuo Zhao · Turn-level Multiscale Density Ratio Estimation for LLM Agents (tlm-dre): Allocate weights at multiple scales by conversation turn, and use asymmetric token updates for positive and negative density ratios.
• 2025-08-15 · First author: Wenhao Zhang · CHORD (chord): Separate SFT and RL stages can overfit expert data or forget it too early. Add expert SFT as an annealed auxiliary term during on-policy RL; token-level uncertainty weights smooth the transition from imitation to exploration.
• 2025-08-11 · First author: Zhenpeng Su · GPPO (gppo): This can suppress exploration and learning from negative samples. GPPO keeps PPO's forward clipped objective while restoring backward signals in those regions with a stop-gradient boundary weight.

Alibaba Qwen Team

• 2025-12-01 · First author: Chujie Zheng · Stabilizing RL with LLMs (minirl): Separate train–inference mismatch from policy staleness. Use on-policy importance correction and combine off-policy clipping with MoE Routing Replay.
• 2025-07-24 · First author: Chujie Zheng · GSPO (gspo): GSPO averages the log-ratios for a response, then exponentiates the result to get one sequence ratio. The entire sequence shares the clipping weight.

AllSpark Team

• 2026-09-22 · First author: Jiayan Fu · PACT: From Credit Assignment to Critic Alignment (pact): Update the actor first so the critic targets the updated policy. Using a current-token ratio rather than the product over the remaining sequence is a first-order approximation, not exact continuation importance sampling.

AllSpark Team (team byline; no institution listed)

• 2026-09-28 · First author: AllSpark Team · ROSS: Relearning from Self-Generated Rollouts through Selective Supervision (ross): Keep the full context of historical self-rollouts but compute loss only on selected model continuations, avoiding imitation of erroneous or redundant steps.

Amazon / Duke University

• 2026-09-03 · First author: Zhishuai Liu · Extremely Sparse Supervision Incentivizes Reasoning Ability (sparse-opd): Instead of supervising every generated token, choose one or two key positions (about 0.05%). Largest positive differences work particularly well, suggesting targeted correction rather than word-for-word matching.

Anhui Province Key Laboratory of Digital Security

• 2026-10-13 · First author: Wei Fan · LoOPSD: On-Policy Self-Distillation with Looped Teachers (loopsd): On-policy self-distillation needs a more useful teacher than the student, but that does not have to be a bigger external model or additional answer context. LoOPSD searches for contiguous blocks of layers to repeat without changing the weights, uses that route as the teacher for student-generated prefixes, and searches for a new teacher after the student update. Not reproduced for now due to cost; validation pending, no executable entry.

Ant Group

• 2025-12-16 · First author: Jian Hu · Online IcePop (online-icepop): Online IcePop allows only one update per batch, so the stale-policy ratio is 1 and PPO ratio/clipping are removed. Keep the IcePop two-sided mask and original in-range engine-mismatch ratio.
• 2025-10-21 · First author: Ling Team · IcePop (icepop): One-sided TIS may retain very small mismatched ratios. Mask ratios outside a fixed two-sided interval and retain original corrections inside.

Anthropic

• 2022-12-15 · First author: Yuntao Bai · Constitutional AI (constitutional-ai): In stage one, the model critiques and rewrites its responses according to the principles, then learns by SFT on the revisions. In stage two, AI compares responses, a preference model is trained, and its reward drives RLAIF.

Apple

• 2025-06-30 · First author: Bo Liu · SPIRAL (spiral): The same model plays problem setter and solver. Automatically scored zero-sum multiround games create a progressively harder curriculum.

Arizona State University / University of Virginia / Stevens Institute of Technology

• 2026-09-15 · First author: Chenhao Qiu · OPD-Aha: From Linguistic Momentum to Visual Reflection in Multimodal On-Policy Distillation (opd-aha): Reconstruct a visually preferred target from the difference between the teacher's distributions with actual and empty visual input. This counters momentum from an incorrect linguistic prefix.

Authors did not disclose affiliation

• 2026-08-19 · First author: Chen Yang · Beyond Imitation: Filtering On-Policy Distillation by Reasoning Progress (r2-opd): Rank reasoning spans by teacher reward and an independent progress reward. Mask distillation where those rankings conflict.
• 2026-08-04 · First author: Jian Zhang · CausalOPD: First-Wrong-Step Supervision for Distilling Causal Chain Reasoning (causal-opd): The teacher identifies the first incorrect step in the on-policy causal chain, repairs it with short-horizon RL, and advances the curriculum from evidence through mechanism to conclusion.
• 2026-07-30 · First author: Xingjian Wu · Contrastive Reinforced Policy Optimization via Privileged Self-Distillation (crpo): Use predictive entropy to distinguish positive reflection/exploration positions from negative exposure-bias positions, then contrast privileged self-distillation signals within groups.
• 2026-07-29 · First author: Jianze Wang · SERPO: Self-Evolving Rubric Policy Optimization for Open-Ended Test-Time Reinforcement Learning (serpo): A Good–Normal–Bad response archive, query-specific rubrics and actor policy co-evolve in a three-way feedback loop.

Baidu Inc.

• 2026-10-19 · First author: Shiao Xie · Reinforcement Learning with Visually Attributable On-Policy Self-Distillation (visually-attributable-opd): It keeps corrections aligned with the visual contrast and uses a bounded term to modulate the verifier's group-normalized advantage. A text-only teacher or fixed-logit gate would not implement this multimodal path. Not reproduced due to cost (review complete); research material only, no executable entry.

Beihang University

• 2026-10-15 · First author: Xiuhui Zhang · Progress Reward Model for Reinforcement Learning via Large Language Models (progress-reward): It uses the subtask's semantics as a state prior and the difference in progress between successive states as dense reward for low-level RL. The plan is generated in advance; the LLM does not run at every environment step. Bounded component only; full experiments are not reproduced.

Beijing Institute of Technology

• 2026-05-13 · First author: Feng Zhang · ConSPO (conspo): Express preferences among sequences in the same group as length-normalized InfoNCE, avoiding length and within-group scale biases caused by summing token scores.

ByteDance Seed

• 2025-04-07 · First author: Yu Yue · VAPO (vapo): Critic bias, heterogeneous lengths and sparse rewards destabilize value-based PPO on long traces. VAPO pretrains a value model and adapts GAE and actor updates to response length.
• 2025-03-18 · First author: Qiying Yu · DAPO (dapo): Standard symmetric clipping can suppress exploration in online RL for long chains of thought, and groups with all correct or all wrong answers offer no learning signal. DAPO's Clip-Higher gives more room to increase probabilities; Dynamic Sampling discards zero-variance groups; token-level loss handles responses of different lengths fairly; and overlong responses receive a piecewise penalty.

ByteDance internship

• 2026-07-28 · First author: Bo-Wen Zhang · CoRT (cort): Replay the same response with a rubric and with criteria removed. Token likelihood differences redistribute GRPO's response-level advantage.

Carnegie Mellon University

• 2026-10-27 · First author: Jixuan Leng · CONCUR: Concordant Auxiliary Rewards for Multi-Task, Multi-Reward Reinforcement Learning (concur): CONCUR-T always ties auxiliary scores at the primary-reward ceiling. CONCUR-A relaxes that tie once cumulative agreement meets a threshold. Bounded component only; full experiments are not reproduced.
• 2026-04-17 · First author: Jixuan Leng · GroupDPO: Memory-Efficient Group-Wise Direct Preference Optimization (group-dpo): Section 4, Algorithm 1 and Table 1 in v1/v2 were reviewed; no material difference was found for these objectives. The implementation follows pinned v2, including accumulation for responses shared across groups. Bounded component only; full experiments are not reproduced.

Chongqing University

• 2026-09-10 · First author: Bincheng Gu · Beyond Confidence: Stability-Aware Test-Time Adaptation for LLM Reasoning (tasco): Freeze the base model and optimize a lightweight prefix. Penalize instability under neighborhood perturbations as well as low confidence, to avoid confident but incorrect trajectories.

City University of Hong Kong

• 2026-10-22 · First author: Xuyang Zhong · MOER: Mix-Policy Optimization under Expert Prefix Guidance with Selective Rollback (moer): The whole guided trajectory, including student-generated suffix, is off-policy. Rollback avoids an additional generation, not the original generation/verifier cost. Bounded component only; full experiments are not reproduced.

Cohere For AI

• 2024-02-22 · First author: Arash Ahmadian · RLOO (rloo): RLOO treats a full response as one action. Sample several responses for the same prompt and use the other responses' mean reward as the current one’s baseline.

Contextual AI

• 2024-02-02 · First author: Kawin Ethayarajh · KTO (kto): It uses the expected KL between policy and reference as a reference point. Examples of each class can be collected independently and need not be balanced.

Cornell University

• 2026-10-16 · First author: Wen-Zhi Li · Preserving Reasoning Diversity in Reinforcement Learning with Verifiable Rewards via Adaptive Replay (tar-adaptive-replay): This avoids continuously imitating a limited bank at the expense of adaptation. See Sections 3–4. Bounded component only; full experiments are not reproduced.
• 2026-10-15 · First author: Zhengxin Zhang · Does On-Policy Distillation Need to Complete Every Rollout? (stopd): Every retained token receives teacher supervision weighted by its exact inclusion probability; natural EOS is an ordinary valid token. Dividing by the current batch's random token count would invalidate the gradient-sum argument, so training uses the previous batch's weighted token count. Bounded component only; full experiments are not reproduced.

Dalian Maritime University

• 2026-10-09 · First author: Ronghan Li · A Step Harder, Not a Step Too Far: The Benefits and Limits of Curriculum Difficulty Progression in GRPO (grpo-difficulty-curriculum): Advancing to B5 can hurt results. This implementation reestimates difficulty from actual rollouts at every step, retains B5 as a control, and does not substitute problem length or stale scores.

Dalian University of Technology / Kuaishou

• 2026-09-29 · First author: Youling Huang · Guide, Then Let Go: Gap-Adaptive Teacher Scheduling for Sparse-Reward Agentic RL (gats): OPD weight decreases linearly as the gap closes. Once the student reaches the teacher reference, turn off teacher supervision permanently and continue with GRPO alone.

DeepSeek-AI

• 2024-02-05 · First author: Zhihong Shao · DeepSeekMath / GRPO (grpo): GRPO samples a group of responses to each question and computes advantages from the group's reward mean and standard deviation, removing the critic. The policy still uses an old-policy ratio, clipping, and reference KL.

East China Normal University

• 2026-10-08 · First author: Zipeng Wang · MetaOPD: Meta-Learned Token Weighting for On-Policy Distillation (meta-opd): A lightweight network learns token supervision weights from student and teacher predictions. It receives a meta-gradient from reference-solution loss after one differentiable virtual student update, then the new weights are used for the real student update.

Eastern Institute of Technology, Ningbo / The Hong Kong Polytechnic University

• 2026-10-08 · First author: Anhao Zhao · DIAL-OPD: Learning More from Fewer Tokens in On-Policy Distillation (dial-opd): A logarithmic probability mean reduces supervision priority for tokens that both teacher and student consider unlikely. Rank positions separately per response; unselected positions remain in the context.

FAIR at Meta / Inria

• 2026-07-28 · First author: Pierre Chambon · Reinforcement Learning for Code Optimization (code-optimization-rl): Offline replay selects rewards and environments; online GRPO addresses sparse rewards and stale batches. See Sections 3–5 and Appendices C–F. Not reproduced due to cost (review complete); research material only, no executable entry.

Fudan University

• 2026-08-27 · First author: Siye Wu · Consolidating RLVR Capabilities Across Domains: A Deep Dive into Fusion Paradigms (rlvr-fusion): The paper compares three cross-domain capability fusion paradigms that reuse different artifacts: Merge combines expert task vectors, Mix RL combines training data, and MOPD reuses both experts and data. Average differences are at most 1.4 points but can reach 8.6 points on individual tasks, so the choice depends on expert availability, data, and cost.

Georgia Institute of Technology

• 2026-09-30 · First author: Xinchen Du · SHARPO: Segment-Level Credit Assignment for Agentic Reinforcement Learning (sharpo): Use a successful trajectory from the same group as self-distillation context. Measure teacher–student log-probability differences within each environment-interaction segment and rescale its GRPO advantage with a bounded multiplier.
• 2026-09-29 · First author: Zhiqi Li · FlowMap-OPD: Rollout-Kernel Separation for On-Policy Distillation of Few-Step Flow-Map Generators (flowmap-opd): Rollout supplies states with the correct marginal distribution. On frozen states, the optimization kernel compares flow maps, induced velocities, or instantaneous velocities.

Google

• 2026-10-23 · First author: Leon Sick · On-Policy Self-Training: Data-Free Self-Improvement in Large Language Models (opst): Discard a too-easy item only if all \(G\) student rollouts match the proposed answer; agreement on some other answer is not sufficient. Cross-round semantic deduplication is external. Bounded component only; full experiments are not reproduced.
• 2026-02-24 · First author: Sanket Badhe · Prompt-Level Distillation: A Non-Parametric Alternative to Model Fine-Tuning for Efficient Reasoning (prompt-level-distillation): It abstracts teacher reasoning into reviewable system instructions: extract micro-rules from labeled training examples, denoise them using DBSCAN on semantic vectors, synthesize rules, then revise conflicts using training errors. The student uses the resulting instructions without reading test answers. Not reproduced due to cost (review complete); research material only, no executable entry.
• 2025-09-30 · First author: Zhuohang Li · Judging with Confidence: Calibrating Autoraters to Preference Distributions (probabilistic-autorater): The strict local tag parser is an explicit local choice, not the unpublished author parser. A numerical gradient diagnostic does not establish backpropagation through text-to-float parsing or GRPO training. Bounded component only; full experiments are not reproduced.
• 2025-09-30 · First author: Wenda Xu · When LLMs Benchmark Themselves: Deconstructing Self-Bias in Automated Evaluation (benchmark-self-bias): Average ranks for ties are an explicit local choice. Peer consensus is not absolute truth, and synthetic tables do not establish real-model bias. Bounded component only; full experiments are not reproduced.

Google DeepMind

• 2023-10-18 · First author: Mohammad Gheshlaghi Azar · IPO (ipo): The paper proposes \(\Psi\)PO, which optimizes preference probabilities directly. The identity mapping yields IPO: fit a finite target margin instead of indefinitely widening the chosen/rejected gap with logistic loss on separable training data.
• 2023-06-23 · First author: Rishabh Agarwal · GKD (gkd): GKD lets the student generate rollouts from its current policy and queries the teacher for full distributions at the states the student actually visits. The student data fraction interpolates fixed and on-policy data, and the divergence may be forward KL, reverse KL, or generalized JSD.
• 2023-05-17 · First author: Yao Zhao · SLiC-HF (slic-hf): Instead of a multi-model PPO-RLHF stack, enforce a margin between preferred and rejected sequence log-likelihoods while regularizing with SFT. The objective can use off-policy preferences collected for other models.

Google Research

• 2026-03-17 · First author: Lucas Bandarkar · Large Reasoning Models Struggle to Transfer Parametric Knowledge Across Scripts (cross-script-knowledge-transfer): The work does not propose a new closed-form RL loss. This batch records the paper only, without inventing an operator. Not reproduced due to cost (review complete); research material only, no executable entry.
• 2023-09-01 · First author: Harrison Lee · RLAIF (rlaif): An AI labeler judges candidates in both presentation orders. Average away positional bias and train the preference policy with the paired labels.

Google Research / Courant Institute of Mathematical Sciences

• 2026-04-30 · First author: Mehryar Mohri · Mind the Gap: Structure-Aware Consistency in Preference Learning (sa-dpo): SA-DPO sets the margin as a function of response distance while preserving implicit reward differences relative to a reference policy. It studies consistency among the margin, model capacity, and surrogate loss tail rather than merely tuning DPO's learning rate. Not reproduced due to cost (review complete); research material only, no executable entry.

HKUST

• 2023-04-13 · First author: Hanze Dong · RAFT (raft): RAFT samples multiple responses from the current model each round, ranks them with a reward model, discards lower-quality samples and maximum-likelihood fine-tunes on the selected responses. The updated policy starts the next round.

Harbin Engineering University / Tencent / Harbin Institute of Technology

• 2026-10-08 · First author: Xiaobing Chen · Residual Advantage: Student-Relative Teacher Guidance for RL with Verifiable Rewards (residual-advantage): These are centered per response and added to the verifier advantage. Co-RA updates teacher LoRA on the same scored student batch and uses the new teacher only in the next round.

Harbin Institute of Technology (Shenzhen)

• 2026-10-08 · First author: Hexuan Deng · GRPODropout: Less is More for Online Reinforcement Learning Rollouts (grpo-dropout): GRPODropout removes some high-probability, positive-advantage responses from a same-prompt GRPO group, then recenters retained advantages using old-policy probabilities. This is not random dropout and does not change the generation budget.

Huawei Noah’s Ark Lab

• 2026-09-29 · First author: Matthieu Zimmer · The Weakest Link: Distilling LLM Reasoning with Worst-Case Constrained Reinforcement Learning (weakest-link): A violation enters an absorbing penalty state, so easy tokens cannot undo bad reasoning generated earlier. The objective also differentiates the exact full-vocabulary expectation of immediate reward to reduce variance from sampling only the current action.

ILLC, University of Amsterdam

• 2026-07-27 · First author: Ivo Verhoeven · What do Reward Models Memorize? (rm-memorization-protocol): User-level grouping prevents pairs from one person crossing train/test. The local component aggregates real scores/manifests supplied by its caller and checks user consistency; it neither generates 25 reward models or the optimized data splits nor implements the underspecified co-occurrence search. Bounded component only; full experiments are not reproduced.

Independent Researcher

• 2026-08-10 · First author: Zhuo Sun · SR-OPSD: Self-Referenced On-Policy Self-Distillation (sr-opsd): Geometrically interpolate self-teacher and reference to obtain an effective target. Rényi divergence controls projection geometry and density-ratio sensitivity.

Independent researchers

• 2025-01-04 · First author: Jian Hu · REINFORCE++ (reinforce-plus): Prompt-local standard deviations in GRPO/RLOO can randomly reweight groups of different difficulty. REINFORCE++ retains within-group centering but normalizes by a global advantage scale across batches to reduce variance and local bias without a critic.

Institute of Automation, Chinese Academy of Sciences

• 2026-10-15 · First author: Xun Lu · Learning to Reason with Selective Internal Feedback (sif): It first votes over windows, then derives a binary reward from the fraction of eligible peers beaten. Ineligible responses and constant-reward groups are filtered; policy and KL terms both use the selection mask. Bounded component only; full experiments are not reproduced.

Institute of Automation, Chinese Academy of Sciences / Tencent

• 2026-09-12 · First author: Gengsheng Li · Data-free On-policy Distillation (df-opd): Have the teacher generate questions in its own policy space, reducing dependence on external post-training corpora.

Institute of Information Engineering, Chinese Academy of Sciences

• 2026-09-09 · First author: Naibin Gu · CompassOPD: Cross-Family On-Policy Distillation via Within-Family Likelihood Shifts (compass-opd): Absolute probability scales differ across model families. Compare the post-trained teacher against its reference, then center the change within its family to remove a global offset while retaining the direction.

Johns Hopkins University

• 2026-07-08 · First author: Xiuyi Lou · TACO (taco): A correct trajectory's uniform positive advantage can reward implausible low-probability tokens. Estimate contextual tail risk, smoothly reduce high-risk positive credit, and retain negative credit.

KAIST

• 2026-10-09 · First author: Jiwon Jeon · Inject a Turn, Not a Token: Learning Where to Redirect LLM Reasoning for Self-Correction (upturn): The following redraw explains the mechanism; it is not the original figure.
• 2026-09-29 · First author: Youngjun Jun · DriftOPD: Sequence-Level Reverse-KL Distillation for One-Step VLA Policies (drift-opd): Decompose sequence-level reverse KL into one-step reverse KL for the current action chunk and a future potential for long-horizon consequences. The critic learns from offline demonstrations.
• 2026-09-29 · First author: Doohyuk Jang · Learning Beyond What You Sample: Off-Policy-Aware Cross-Model Trajectory Exchange for RLVR (graft): When the receiver's group is entirely wrong and a peer has both successful and failed trajectories on the same prompt, replace the uninformative group with the complete peer group and retain its internal advantages. Sequence-mean log-likelihood compatibility downweights cross-tokenizer policy mismatch; token-level importance ratio clipping bounds updates.
• 2026-09-29 · First author: Jaeyun Shin · LEGO-OPD: Factorized Teacher Composition for Multimodal On-Policy Distillation (lego-opd): Use a language expert as the token prior and a grounding expert only as the visual likelihood. Their product of experts forms the OPD teacher without also distilling the VLM's language bias.
• 2024-03-12 · First author: Jiwoo Hong · ORPO (orpo): ORPO combines the chosen response's NLL and a chosen-versus-rejected odds-ratio penalty into one objective. Near probabilities of 0 or 1, odds provide a more sensitive contrastive signal than raw probability differences.

KAIST AI

• 2026-09-08 · First author: Youngrok Park · Eliciting Weak-to-Strong Generalization with On-Policy Reverse Distillation (oprd): Extract a weak model's before/after post-training logit direction and amplify the strong model's verifiable-reward gradient along it. Learning direction may transfer better than final capability.

KAIST AI / LG AI Research

• 2026-10-09 · First author: Woosung Koh · Continually Post-training Small Models via Synthetic Self-distillation (synthself): The evaluated student cannot see reference answers. Text filtering alone is not training.

Korea University

• 2026-10-19 · First author: Suwan Yoon · Correctness-Dependent Asymmetric Target Distributions for Self-Distillation Policy Optimization (apsd): Detaching the entire target loses the defining coupled gradient. See Section 4 and Appendix B. Bounded component only; full experiments are not reproduced.
• 2026-08-12 · First author: Byungoh Ko · Context Blindness in DPO: Mitigating Object Hallucination in MLLMs via Context-Calibrated Preference Optimization (c2-dpo): The paper defines CPG as the increase in the chosen-versus-rejected preference margin when context is added. C²-DPO increases that gain directly while retaining the original preference ordering.

LeapLab, Tsinghua University

• 2026-08-25 · First author: Wenze Lin · OPDVR: On-Policy Distillation with Verifiable Rewards (opdvr): Sampled-token teacher/student ratios can penalize correct trajectories or reward wrong ones. Rather than adding a separate RL loss, gate the existing implicit reward with verifier correctness: retain nonnegative signals for correct traces and nonpositive signals for incorrect traces.

Leiden University

• 2026-10-22 · First author: Jujia Zhao · UserArena: Benchmarking Interactive User Simulation for Long-Horizon Shopping Agents (userarena-optd): Exclude specific actions, item IDs, prices, outcomes, future pages and evidence indices from intent. KL alone cannot prove absence of leakage. Bounded component only; full experiments are not reproduced.

Ling / Ring Team

• 2026-06-13 · First author: Ang Li · KPop (kpop): A fixed ratio mask can reject legitimate exploration or keep harmful gradients. Collapse the sampled token and remaining vocabulary to a Bernoulli distribution; retain its update only if forward and reverse binary KL meet the threshold.

LinkedIn

• 2026-05-12 · First author: Hejian Sang · Beyond GRPO and On-Policy Distillation: An Empirical Sparse-to-Dense Reward Principle for Language-Model Post-Training (sparse-to-dense-reward-principle): Existing GRPO, OPD, and KL operators are not the full workflow. Generic KL wrappers or four state labels would not reproduce it. Not reproduced due to cost (review complete); research material only, no executable entry.

MIT HAN Lab

• 2026-04-14 · First author: Yecheng Wu · Lightning OPD (lightning-opd): The same teacher computes token distributions once and caches them. Training reads only the cache, preserving distillation on the student distribution without an online teacher during training.

Massachusetts Institute of Technology

• 2025-07-22 · First author: Mehul Damani · Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty (rlcr): RLCR predicts both an answer and its confidence and adds a bounded proper scoring rule so honest probabilities maximize expected reward. The local baseline minimizes multiclass Brier over the full candidate distribution.

McGill University

• 2025-05-26 · First author: Meng Cao · SCAR: Shapley Credit Assignment for More Efficient RLHF (scar-shapley-credit): The local scope is exact enumeration for N<=10 and reward scatter at gamma=1. It does not treat an unspecified parser, an Owen hierarchy, or a synthetic reward model as the original system. Bounded component only; full experiments are not reproduced.

Meituan

• 2026-02-05 · First author: Fanfan Liu · LUSPO (luspo): Decompose RLVR objectives to explain different response length trajectories. GSPO still carries length bias; normalize sequence log-probabilities to prevent collapse.

Meta AI

• 2026-07-21 · First author: Priyank Agrawal · Off-Context GRPO: Learning to Reason on Hard Problems using Privileged Information (off-context-grpo): The target remains the original unhinted policy, and an importance ratio corrects for the difference in sampling distributions. Inference does not need privileged context.
• 2024-01-18 · First author: Weizhe Yuan · Self-Rewarding LM (self-rewarding): The current model generates candidates and judges them each round. New preference pairs feed the next DPO step, forming a bootstrapping loop.

Meta AI / University of California, Riverside

• 2026-09-25 · First author: Shangjian Yin · Recursive Self-Improvement via On-Policy Distillation for Reasoning (recursive-opsd): 1. DCE (dynamic co-evolution): At round k, the current student samples answer yᵏ. The teacher uses the same current weights, stops gradients and also sees the reference solution to score the same answer prefixes, producing qᵏ.

Meta Superintelligence Labs

• 2026-10-21 · First author: Wen Ye · Edit, Don't Imitate: Minimal-Edit On-Policy Distillation for Multimodal Reasoning (minimal-edit-opd): The teacher sees a training-only diagram description, verified solution and the entire original response, then teacher-forces that response a second time instead of generating a new answer. Correct responses receive task GRPO only; none of the teacher, references or verifier enters deployment. Bounded component only; full experiments are not reproduced.
• 2026-09-12 · First author: Sourabh Kulkarni · MaD-RL: Matching Distributions for Calibrating LLMs with Reinforcement Learning (mad-rl): Measure distribution fit and accuracy separately. See Section 2. Not reproduced due to cost (review complete); research material only, no executable entry.

Meta Superintelligence Labs / University of Wisconsin–Madison

• 2026-10-01 · First author: Changdae Oh · Sharpening Tax in Post-Training (sharpening-tax): Compare coverage of the base and post-trained policies under the same sampling budget. Posterior-tempered group sampling adjusts temperature by estimated difficulty to balance pass@1 and coverage.

Microsoft Research

• 2026-02-12 · First author: Tianzhu Ye · OPCD (opcd): A teacher with experience or a system prompt scores that same rollout, using reverse KL to internalize context-dependent behavior. The method covers experience and system prompt distillation and supports different teacher sizes.

MiniMax

• 2025-06-16 · First author: MiniMax · CISPO / MiniMax-M1 (cispo): Sample with a fixed rollout policy and compute token-level importance ratios. Clipping only the ratios preserves the direction of advantages and effective gradients.

Moore Threads AI

• 2026-10-01 · First author: Yafei Zhang · CARM: Cancellation-Aware Response Masking for LLM Reinforcement Learning (carm): Take absolute values of per-token current/rollout log-ratios before averaging. Positive and negative drifts can otherwise cancel in a sequence-level geometric mean.

NAVER AI Lab

• 2026-08-06 · First author: Byeongho Heo · On-Policy Delta Distillation for Multilingual Math Reasoning (opd2): Distill the probability change of a post-trained teacher relative to its base model, reducing replication of pre-existing teacher biases.

NVIDIA

• 2026-10-20 · First author: Mingyu Yang · CSD: Common-Support Distillation for Cross-Tokenizer Knowledge Transfer (csd): For 1:1 use shared token events; for 1:N or M:1 hold observed prefixes fixed and enumerate equivalent final completions; M:N defaults to the observed sequence only. The REST event preserves unmatched mass. Bounded component only; full experiments are not reproduced.
• 2023-10-09 · First author: Yi Dong · SteerLM (steerlm): Traditional RLHF reduces multidimensional preferences to one reward, and users cannot change the target at inference. SteerLM uses an attribute predictor to label responses for helpfulness, quality, and other properties, adds attributes and values as conditions in ordinary SFT, and lets the user specify target attributes at inference.

Nanjing University

• 2026-10-20 · First author: Guangyu Liu · Adaptive Pullback: Reweighting Supervision at Penalty Jumps in On-Policy Distillation (ap-opd): Measure adjacent blocks of positive log-ratios from the same student prefixes, then downscale the entire following block without another teacher forward or sign changes. Read raw unweighted previous and next statistics, not recursively downweighted ones. Bounded component only; full experiments are not reproduced.
• 2026-10-15 · First author: Yucheng Li · OpCredit: Operation-Aware Policy Learning for Agentic Knowledge Graph Question Answering (opcredit): Both credits align to the same reasoning and executable-operation span; tool observations incur no policy loss. Section 3 and Appendix D Bounded component only; full experiments are not reproduced.
• 2026-09-30 · First author: Bo-Wen Zhang · T2SPO: Trajectory-to-Step Policy Optimization for Agentic Reinforcement Learning (t2spo): Convert distance changes between consecutive states into step advantages and optimize them jointly with trajectory rewards. Frozen TabPFN avoids training another critic; unsuccessful trajectories are not mislabeled as positive near-success examples.
• 2026-08-06 · First author: Zhiyan Hou · DASH (dash): DASH uses local teacher/student divergence to form a stop-gradient gate and recursively aggregates weights backward. It obtains an adaptive distillation horizon without extra teacher forward passes.
• 2026-08-06 · First author: Xinye Wang · RP-OPSD (rp-opsd): Surface wording and tokens that change the reasoning state should not receive equal weight in cross-language transfer. RP-OPSD compares matched teacher views with and without an English reference solution, finds pivots by distribution shift, and strengthens privileged distillation there while retaining a reference anchor.

Nanjing University / JD.com

• 2026-10-09 · First author: Shi-Yu Tian · Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillation (road-mopd): Road uses the non-routed teacher to obtain compatibility evidence on the same student trajectory in the target domain. It adjusts token credit, computes an actual auxiliary gradient, and projects the target gradient while retaining a lower bound on its norm.

Nankai University

• 2026-07-19 · First author: Chen Wang · Distilled RL (distilled-rl): RL provides sequence-level rewards, while OPD imitates a teacher unconditionally. Distilled RL uses the teacher-to-student probability ratio to reweight reward at each token, activates teacher weighting only for positive advantages, and uses a sequence geometric mean to remove length-scale bias.

Nanyang Technological University

• 2026-09-29 · First author: Yuhao Wang · From Dissonance to Orchestration: Teacher Intervention in On-Policy Distillation (maestro-opd): Compute per-token disagreement with teacher top-k coverage and Bhattacharyya similarity, emphasizing positions near the response start. Intervene where disagreement is high rather than paying for full teacher rollouts.
• 2026-09-08 · First author: Jiacheng Xu · Entropy-Regularized Rank-Masked Policy Optimization for Test-Time Reinforcement Learning in Code Generation (probe-erpo): Estimate question reliability via a probe consistency ratio (PCR) without using final answers. Reinforce high-consistency samples, penalize lower-ranked candidates at low consistency, and constrain exploration with entropy.
• 2026-08-27 · First author: Ziyue Wang · Video-OPSD: Exploiting Privileged Visual Evidence for On-Policy Self-Distillation in Video Large Language Models (video-opsd): Distillation is weighted by each token's dependence on that evidence. The paper reports performance close to GRPO with approximately 60% less training time.

National University of Defense Technology

• 2026-09-15 · First author: Qixiu Li · TIAO: Token Importance-Aware Policy Optimization for Text Summarization (tiao): Estimate dependency from changes in token probabilities before and after masking the source document, then reshape trajectory advantages and concentrate updates on important tokens.

National University of Science and Technology POLITEHNICA Bucharest

• 2026-10-01 · First author: Andreea Dutulescu · GAW-PO: Preference Optimization with Gradient-Aligned Token Weights (gaw-po): Weight rejected-response tokens by their gradient alignment with the preferred update direction. Tokens that better support preferred behavior receive less penalty.

National University of Singapore

• 2026-10-01 · First author: Shiu-Hong Kao · RLCPR: Rethinking Probability-Based Reinforcement Learning From Posterior Concentration (rlcpr): RLCPR applies relative length penalty only if all responses are long and their posteriors concentrated. Mean direct-answer token entropy stratifies data so retained examples favor low uncertainty.
• 2026-09-16 · First author: Yihao Ai · Trajectory Learnability for Offline On-Policy Distillation (trajectory-learnability): Train a reference model on successful trajectories. Changes in per-token log likelihood between that reference and the current policy estimate learnability, which reweights offline OPD trajectories.

National University of Singapore / Nanjing University

• 2026-10-09 · First author: Suorong Yang · Learning More by Training Less: Uncertainty-Guided Progressive Contraction for Efficient Reasoning Post-Training (progressive-contraction): The paper samples Monte Carlo continuations from the actual prefixes, contracts the teacher pool by branch uncertainty, runs a real student probing epoch, and then selects and reroutes top-M examples by current student NLL. Teacher profiling cost and student exposure must be accounted for separately.

New York University

• 2026-10-15 · First author: Manya Wadhwa · Training Reasoning Models to Generate Diverse Scientific Ideas (diverge-ideation): Different strings do not establish scientific novelty. See Sections 2–3. Evaluation protocol interface; not a model capability result.
• 2026-10-08 · First author: Yuhan Liu · Which Skill to Distill? SGUID: Selecting a Compact Skill Bank for Model–Skill Co-Evolution (sguid): After probing, restore the round's initial weights and retrain only on the selected compact bank. The updated model generates the next bank rather than reusing fixed handwritten skills.

North South University

• 2026-09-29 · First author: Md. Ismail Hossain · Overcoming Scaling Limits in On-Policy Self-Distillation for LLM Reasoning (oasis-opsd): Questions without a verified rollout produce no training signal. This requires only a final-answer verifier, not a gold solution.

Northeastern University

• 2026-08-06 · First author: Chenglong Wang · RRC: Unlocking Generative Reward Models in LLM Reinforcement Learning via Ranking-Based Reward Construction (rrc): Generative reward models compare outputs better than they assign independent scalars. RRC constructs centered reward with group self-competition and a small anchor ranking.

OpenAI

• 2023-05-31 · First author: Hunter Lightman · Let's Verify Step by Step (process-supervision): A step-level reward model judges each reasoning step and prioritizes uncertain steps for annotation. The local comparison uses the same candidates and budget as outcome-only reward.
• 2022-03-04 · First author: Long Ouyang · InstructGPT / PPO-RLHF (ppo-rlhf): Increasing language-model size alone does not guarantee alignment with user intent. InstructGPT establishes the three-stage pipeline: SFT on annotator demonstrations, reward-model training on pairwise rankings, and PPO policy optimization with a KL penalty to limit drift from the SFT/reference model.

PRIME-RL author team

• 2025-04-22 · First author: Yuxin Zuo · TTRL (ttrl): Sample several answers for the same test question, use the majority answer as a pseudo-label, and update immediately without gold labels.

Peking University

• 2026-10-14 · First author: Tiantian Peng · Divergence-Routed On-Policy Distillation for Language Models (divergence-routed-opd): The reverse-KL logit gradient contains the student probability factor, so correction weakens when the student assigns a teacher token almost no mass. The forward-KL gradient is the difference between student and teacher probabilities and can still push toward those teacher modes. Bounded component only; full experiments are not reproduced.
• 2026-08-27 · First author: Xingyu Shen · Boosting LLM Exploration via Weak-Model Guidance in RLVR (weak-guide-rlvr): A smaller, weaker model supplies part of the reasoning prefix, exposing the target model to unfamiliar trajectories. Entropy truncation and mixed training on native and guided samples preserve coverage.
• 2026-08-27 · First author: Ziyuan Liu · Preserving General Capabilities during Domain Specialization with Uncertainty-Calibrated MOPD (uc-mopd): Ordinary MOPD seldom samples tokens with strongly positive advantages and cannot tell whether update directions are reliable. This method broadens the sampling temperature range, selects trajectories by positive-advantage density, and uses entropy-calibrated CLL probabilities to gate token updates.
• 2026-08-26 · First author: Lam So · GRIP: Granular Reward-Guided Parameter Interpolation for Efficient Reasoning (grip): Replace a global blend between thinking and instruction weights with reward-driven layer/parameter interpolation to keep reasoning accuracy while shortening output.
• 2026-07-30 · First author: Yuran Wang · Flux-OPD (flux-opd): Flux-OPD holds the context-free teacher fixed as an anchor and injects only log-probability differences from multiple evolving contextual teachers. The geometric-mean normalization constant captures conflict; higher conflict weakens the correction.
• 2023-12-14 · First author: Peiyi Wang · Math-Shepherd (math-shepherd): Sample several continuations from an intermediate reasoning step. Their final answer correctness supplies automatic step labels for verifier training and reranking.

Ping An Technology / University of Science and Technology of China

• 2026-10-08 · First author: Heng Li · PIVOT: Perplexity-Informed KD-to-RL Transition Scheduling for Vertical-Domain Few-Shot Distillation (pivot): Switching every example from distillation to RL at once can remove teacher supervision from hard examples too early. PIVOT computes teacher perplexity on student-generated responses, moving low-perplexity questions to GRPO first and keeping high-perplexity questions on OPD.

Princeton University

• 2026-10-15 · First author: Stanley Wei · Compatibility Matters: A Framework for Token-Level Teacher Selection in On-Policy Distillation (token-teacher-selection): Soft routing mixes probabilities, not average logits or log-probabilities. Paper §3 Bounded component only; full experiments are not reproduced.
• 2024-05-23 · First author: Yu Meng · SimPO (simpo): DPO keeps a reference model and sequence probability tends to favor short responses. SimPO uses mean token log-probability as the implicit reward, drops the reference, and adds a fixed margin to Bradley–Terry preference training.

Qwen Business Unit of Alibaba / University of Science and Technology of China

• 2026-11-02 · First author: Tianyu Liu · SpecOPD: Branch-Parallel On-Policy Distillation for Speculative Decoding (specopd): This is not unrestricted on-policy response generation. Sparse indexers and recurrent targets additionally need branch-consistent grouping and state replay; a mask alone is insufficient. Bounded component only; full experiments are not reproduced.

Reichman University

• 2026-08-12 · First author: Lior Baruch · Preference Tree Optimization: Enhancing Goal-Oriented Dialogue with Look-Ahead Simulations (pto): An oracle scores both current responses and continuations; preference pairs train DPO iteratively. Deeper lookahead makes the long-term policy more stable.

Renmin University of China

• 2026-07-06 · First author: Yu Li · Turning Off-Policy Tokens On-Policy: A Plug-in Approach for Improving LLM Alignment (sis): The exact envelope is \(M_t=\max_v\pi_\theta(v)/\pi_{\mathrm{old}}(v)\), approximated on behavior top-K \(V_K\). The analysis bounds accumulated log deviation \(D=\sum_t|\log w_t|\).

Rensselaer Polytechnic Institute

• 2026-09-28 · First author: Jonathan Light · Shockingly Simple Self-retrospection Improves Agentic Models Without RL (roft): Instead of directly updating the action policy with environment rewards, ROFT generates a self-retrospection after the complete trajectory and feedback, then supervises retrospection tokens only. The resulting training signal can inform future behavior without treating environment text or prior actions as targets.

Rutgers University

• 2026-09-29 · First author: Zhenyu Wang · Dr. OPD: Learning What to Follow for Optimal On-Policy Distillation of Large Language Models (dr-opd): OPD uses bilevel optimization to choose token weights that maximize expected reward after the student update. Each round first updates weights in closed form, then takes a weighted OPD step.

S-Lab, Nanyang Technological University

• 2026-08-26 · First author: Shulin Tian · V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning (v-rubrics): Decompose a reference answer into atomic rubric criteria for visual fidelity, reasoning consistency and instruction following. Attribute evidence to the relevant prefix so a terminal scalar does not hide local hallucinations.

SAIL research team

• 2025-03-26 · First author: Zichen Liu · Dr. GRPO (dr-grpo): GRPO's response-length averaging and within-group standard deviation can introduce length and problem-difficulty biases. Remove both normalizations while keeping centered group-relative reward so trajectories retain their scale.

Scale AI

• 2026-08-12 · First author: Minglai Yang · Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL (rubric-dropout): Each rollout group shares a randomly thinned rubric, preventing the policy from persistently exploiting a fixed judge proxy.

Seoul National University

• 2026-08-03 · First author: Wonseok Lee · Beyond On-Policy Exploration: Integrating External Policy Rollouts for Reinforcement Learning in Diffusion Language Models (erils): Cap external-policy rollout length and process rewards separately by on-policy/external source to avoid collapse from joint normalization.
• 2026-07-29 · First author: Junoh Park · ReCo (reco-grpo): GRPO can resample high-probability responses and amplify saturated tokens, reducing reasoning diversity at large k. Weight responses inversely by expected occurrence in the rollout group, then use a Bernoulli variance ratio to focus updates on unsaturated decisions.

Shanghai AI Laboratory

• 2026-08-12 · First author: Kai Yang · GCPO: Diagnosing and Constraining Subspace Geometry in Rollout RL for LLMs (gcpo): GCPO fixes the top-k singular spaces on both sides and restricts low-rank updates to their input and output orthogonal complements. This is a hard feasible set, not a soft penalty with a tuned coefficient.

Shanghai Artificial Intelligence Laboratory

• 2026-10-09 · First author: Jiaming Su · BoundaryEvo: Data-Model Co-Evolution at the Capability Boundary for Long-Chain Reasoning Distillation (boundary-evo): Replay questions that the student has not yet mastered are retained, and new questions are added. Fresh teacher trajectories are generated and verified only after question selection.

Shanghai Jiao Tong University

• 2026-10-22 · First author: Shijie Li · GIFT: Gradient Informed Filtering for Distillation Augmented Reasoning RL (gift): RL and teacher gradients can conflict. Decompose teacher gradients in the original vocabulary-logit space, prefer nonconflicting tokens with a large orthogonal contribution, and gradually reduce the number distilled rather than diluting their coefficients. Bounded component only; full experiments are not reproduced.
• 2026-10-16 · First author: Yushu Wang · GR-OPD: Group-Relative Sequence Credit for On-Policy Distillation (gr-opd): It preserves part of the group mean pressure so centering does not erase overall teacher preference, then applies clipped PPO. See the paper method. Bounded component only; full experiments are not reproduced.
• 2026-10-15 · First author: Feiyang Chen · Learning from Unverified Peers via Shared Residual Distillation (shared-residual-distillation): It compares same-question peers' changes to token distributions against a cross-question control, accepting only conservative residuals supported by multiple peers. The student still reads only the question and its own prefix, and inference does not need peers. Bounded component only; full experiments are not reproduced.
• 2026-07-30 · First author: Kangning Zhang · VAD (vad): Project the original correction onto this direction with a one-sided constraint, then rebuild the target around the student's current distribution. The full privileged teacher remains only a weak regularizer.

Stanford University

• 2026-05-18 · First author: Muhammad Umer · GPRL (gprl): GPRL first computes a group-relative advantage separately for each preference dimension, then aggregates by context. A drift controller detects whether any one dimension dominates training and adjusts the weights.
• 2023-05-29 · First author: Rafael Rafailov · DPO (dpo): Conventional RLHF fits a reward model and then optimizes the policy with PPO, making training complex and unstable. DPO starts from the optimum of KL-regularized RLHF, writes the implicit reward as a policy-to-reference log ratio, and reduces training to binary classification on preference pairs.

Taobao & Tmall Group of Alibaba

• 2026-10-09 · First author: Jingru Yang · CAST: Credit Alignment for Cross-Tokenizer On-Policy Distillation (cast-opd): CAST aligns actual decoded text, projects the teacher span's total log probability uniformly onto the student span, then allocates credit by uncertainty and local smoothness. On unaligned positions, a KL anchor to the initial student limits drift.

Tencent

• 2026-09-08 · First author: Tianle Xia · Distillation as Probability Transport: Routed On-Policy Distillation (route-opd): RouteOPD identifies source tokens that the student overestimates and destination tokens it underestimates. Moving only the mass that needs correction makes supervision more targeted and interpretable.

Texas A&M University

• 2026-06-04 · First author: Xingyu Su · OPDLM (opd-lm): Changing an ARLM to bidirectional attention can cause forgetting and a shift from random-mask training to confidence-based decoding. OPDLM samples the bidirectional student's own inference trajectories and gets target logits from a frozen autoregressive teacher on the same trajectories.

The Chinese University of Hong Kong

• 2026-10-21 · First author: Wenhao Yu · CORD: Conflict-Aware Optimization for Joint RL and Distillation in Agents (cord): Conflict-only projects negative inner products, whereas always removes parallel components even for aligned gradients. These gates and the two protected-objective choices are distinct configurations. Bounded component only; full experiments are not reproduced.

The Chinese University of Hong Kong, Shenzhen

• 2026-10-15 · First author: Wenxuan Luo · Condition-Contrastive Extrapolation for On-Policy Distillation of Language Models (cce-opd): The student always generates from the full question. Positions that fail condition eligibility or the entropy gate still use the original sampled PPO-OPD objective. Bounded component only; full experiments are not reproduced.
• 2026-10-14 · First author: Chenxi Li · Information-Gain Calibrated On-Policy Distillation for Deep Search Agents (ig-opd): ITC reallocates turn budget; IQC uses signed gain to correct credit direction for action-critical tokens. References are only for a training-only frozen teacher probe and never enter student sampling context. §4 Bounded component only; full experiments are not reproduced.

The Hong Kong Polytechnic University

• 2026-10-20 · First author: Runyang You · On-Policy Transition Distillation (optd): Keep final OPD, add supervision on log-probability increments across selected shallow-to-deep readouts, and fix candidate identities using final student top-k after full-vocabulary log-softmax. No per-layer top-k or hidden-state MSE. Bounded component only; full experiments are not reproduced.

The Ohio State University

• 2026-10-15 · First author: Hanane Nour Moussa · EliteReplay: Preserving Diverse Solutions in RL Post-Training via Quality-Diversity Search (elite-replay): Cells are indexed by question, approach, and structural descriptor, retaining only the shortest verified solution per cell. It is not an ordinary IID replay buffer. Bounded component only; full experiments are not reproduced.
• 2026-10-01 · First author: Qijia He · Asynchronous LLM Post-Training: Group-Mass Capping and Convergence Analysis (gmc-grpo): Scale asynchronous-rollout importance ratios uniformly within a group to cap their total mass. Under a common second-moment guarantee, this reduces the persistent bias of trajectory-wise clipping.

The University of Hong Kong

• 2026-10-23 · First author: Zongyao Zhao · When to Stop Penalizing Actions? Martingale GRPO for VLA Reinforcement Learning (mart-grpo): Drop loss only after its upper endpoint, leaving the successful trajectories, original group advantages and original valid-token denominator unchanged. This is not early environment termination, reward editing or regrouping. Bounded component only; full experiments are not reproduced.

The University of Texas at Austin

• 2026-07-21 · First author: Hanqing Zhu · ISO: An RLVR-Native Optimization Stack (iso-rlvr): Fix pretrained singular values and optimize only input/output singular frames. The paper also provides a data-free specialist merger.

Tianjin University

• 2026-08-01 · First author: Zhuowen Han · Distill Where You Fail: Recovering Learning Signals of Negative RL-Groups from Adaptive Teacher Guidance (rstg): Enable the teacher only for negative zero-variance groups. Distill high-entropy or high teacher–student-gap tokens and add SFT on correct teacher trajectories.
• 2026-05-12 · First author: Zhong Guan · Missing Old Logits (missing-old-logits): Losing historical training-side logits conflates train/inference mismatch correction with staleness correction. Fixes include snapshots, an old-logit model, interrupted synchronization, and revised PPO-EWMA.

Together AI / University of Illinois Urbana-Champaign

• 2026-10-14 · First author: Yifan Yu · On-Policy Residual Learning for Diffusion Language Models (orl-diffusion): Training must see residual trajectories induced by the model itself. ORL combines online collection, state transfer, exact replay, and entropy contraction to narrow the residual-state distribution gap between training and inference. Not reproduced due to cost (review complete); research material only, no executable entry.

Trent AI Limited

• 2026-10-02 · First author: Joery A. de Vries · Follow the Winners: Conservative Policy Improvement with the Cross-Entropy Method for Critic-Free RFT (follow-the-winners): It stores real trajectories from recent policies in a FIFO pool, repeatedly samples minibatches, imitates only the \(K\) highest-return trajectories in each batch, and constrains updates with KL. Reselection counts naturally weight past trajectories, rather than cutting the entire pool once.

Tsinghua University

• 2026-10-16 · First author: Heming Zou · Tree-Aware Posterior Distillation for Long-Horizon Agentic Reinforcement Learning (treeopsd): As the student's change relative to the old policy reaches the target, the gradient vanishes; overshooting reverses it. See Sections 2–3. Bounded component only; full experiments are not reproduced.
• 2026-10-15 · First author: Yuechen Luo · OPRD: On-Policy Reminder Distillation for Long-Horizon Coding Agents (oprd-reminder): Inference uses neither the teacher nor any reminders. See Section 3. Bounded component only; full experiments are not reproduced.
• 2026-07-11 · First author: Zhicheng Cai · RIPO (ripo): Fixed PPO ratio bounds can be too conservative at low probabilities and too loose at high probabilities. Use Fisher–Rao policy distance to set clip radius from the old action probability for a more even local KL budget.
• 2026-06-17 · First author: Haipeng Luo · STARE (stare): Identify entropy-critical tokens by batch surprisal quantiles, reweight their advantages, and use target entropy in a feedback gate.
• 2025-05-06 · First author: Andrew Zhao · Absolute Zero (absolute-zero): A proposer generates verifiable tasks, a solver attempts them, and an executable verifier supplies rewards. Adapt task difficulty to the current capability boundary.
• 2023-06-14 · First author: Yuxian Gu · MiniLLM (minillm): Forward KL covers all teacher probability mass, so a small student may overestimate low-probability teacher regions. MiniLLM optimizes mode-seeking reverse KL on the student's own generation distribution and stabilizes policy gradients with teacher-mixed sampling, single-step decomposition, length normalization and a reward baseline.

Tsinghua University / DiDi Voyager Labs

• 2026-09-15 · First author: Shiqi Liu · Beyond Token-Local Imitation: Reward-Compatible Temporal Credit Assignment for On-Policy Distillation (gamma-opd): Approximate sequence-level reverse KL with discounted temporal credit and add verifiable outcomes through bounded reward advantages.

Tsinghua University / SenseTime Research

• 2026-10-09 · First author: Dongyang Chen · Branching On-Policy Distillation (branch-opd): It also uses protocol events to supervise termination. Branches are not teacher-forced and are not filtered by answer correctness.

Tsinghua University AIR

• 2026-10-09 · First author: Zhicheng Cai · Generalized Proximal Policy Optimization: Unifying LLM RL Algorithms from the Perspective of Divergence (gppo-framework): All combinations optimize tokens from real rollouts. This is not a registry alias for GRPO and does not replace the existing GPPO from arXiv 2508.07629.
• 2026-10-09 · First author: Zhicheng Cai · Trajectory-Aware Policy Optimization: Closing the State Distribution Gap in Long-Horizon LLM RL (tapo-trajectory): TAPO uses a geometric mean of prefix likelihood ratios, tightens clipping by length, and detaches the prefix so that earlier tokens receive no gradient from the current action. This adapter does not replace arXiv 2607.27973's TAPO.

UCLA / NVIDIA (internship noted in the paper)

• 2026-10-08 · First author: Siyan Zhao · When Do We Need On-Policy Distillation? Distilling on Offline Student Rollouts Is Often Better (semi-opd): Teacher scores are cached and reused unchanged. Each update recomputes current student probabilities and token advantages; caching old advantages would change the method.

University of British Columbia

• 2026-10-15 · First author: Yushu Li · Multi-Reward Hacking: From Aggregation-Induced Drift to Trust-Anchored Defense (tag-reward-gating): Auxiliary credit requires anchor success. Section 5 Bounded component only; full experiments are not reproduced.

University of California, Berkeley

• 2026-09-16 · First author: Peter Chen · A Zeroth-Order Paradigm for LLM Preference Alignment (compo): Compare outcomes at positive and negative parameter perturbations instead of differentiating the preference objective. The comparison returns one bit of direction, and coordinate-wise thresholding suppresses noise.
• 2025-05-26 · First author: Xuandong Zhao · INTUITOR (intuitor): The direction is uniform-to-policy, not negative policy entropy. Confidence does not imply correctness.

University of California, Los Angeles

• 2026-01-26 · First author: Siyan Zhao · OPSD (opsd): The student generates trajectories, and the teacher gives full-vocabulary supervision only at visited prefixes. Per-token pointwise divergence clipping keeps style tokens from dominating.
• 2024-01-02 · First author: Zixiang Chen · SPIN (spin): Starting from an SFT model, SPIN uses the previous model to answer training prompts, treats human demonstrations as positives and self-generated answers as negatives, and trains the next model with a self-play discriminative objective. This repeats without new human preference labels.

University of California, Merced

• 2026-08-26 · First author: Zhibo Hou · From Memorization to Absorption: Mixed-Policy RL for Continual Knowledge Injection (grin): When on-policy sampling rarely finds new correct answers, inject golden responses into failing groups and train with mixed-policy importance correction. Return to on-policy exploration when performance improves.

University of California, San Diego

• 2026-10-14 · First author: Sheldon Yu · Failure Is the Mother of Success: Recovering Supervision with Grounding Hindsight (rshd): Inference uses only the student without a note. Algorithms 1–2 Bounded component only; full experiments are not reproduced.
• 2026-08-06 · First author: Yijiang Li · U-OPSD (u-opsd): Sample repeatedly, majority-vote, and use the shortest agreeing solution as a privileged view to repair the longest high-confidence wrong trajectory. Self-distillation relies on internal consistency.
• 2025-08-05 · First author: Feng Yao · TIS (tis): TIS multiplies policy gradients by the training-to-rollout probability ratio. A one-sided upper cap controls heavy-tail variance while retaining small-weight samples.

University of Chinese Academy of Sciences

• 2026-08-25 · First author: Qinglin Ye · OPDSearch+: Search-Enhanced On-Policy Distillation with Reinforcement Learning (opd-search-plus): OPDSearch+ freezes a general instruct teacher and uses forward KL to distill query, reasoning and answer tokens on the student's online search trajectories. Verifier RL then starts from the better behavior distribution, aiming to move beyond the teacher and the local optima of RL alone.

University of Florida

• 2026-09-29 · First author: Muyang Li · PR-OPD: Privileged Representation On-policy Self-Distillation for Agentic Reinforcement Learning (pr-opd): PR-OPD aligns projected hidden states at each layer and response token, then combines alignment with the reward objective. Inference removes the skill and teacher branch.
• 2026-08-17 · First author: Yixuan Wang · Learn What's Left, Not What's Mastered: Saturation Aware Advantage Reweighting for Multi-Reward Policy Optimization (sa-mrpo): Normalize advantages per reward dimension and shift gradient budget toward unmastered objectives according to batch saturation.

University of Illinois Urbana-Champaign

• 2026-11-02 · First author: Ziyi Chen · RLiD: Reinforcement Learning in the Decision Plane for Language Agents and LLMs (rlid): Menus, afterstates, and memory change the task interface and must be matched in comparisons. Neither temporal-credit noise nor the benefit of reasoning computation disappears automatically. Bounded component only; full experiments are not reproduced.
• 2026-10-27 · First author: Sirui Chen · Proposing Better Rollouts for On-Policy Distillation (mpopd): The mixture is not a replacement for the teacher in the student loss. The student still minimizes reverse KL to the teacher. Bounded component only; full experiments are not reproduced.
• 2026-10-20 · First author: Jiashuo Sun · Ask to Distill: Learning to Synthesize Data for On-Policy Distillation (ask-to-distill): Reward both disagreement and teacher-favored tokens within student-reachable support. Filter and deduplicate the resulting bank and run OPD from the original student initialization, rather than tuning a student separately to value every question. Bounded component only; full experiments are not reproduced.
• 2026-10-15 · First author: Bangji Yang · Training-Free On-Policy Distillation (tf-opd): The key is not arbitrary covariance: use second moments of the inputs each teacher actually sees on its own generated responses after termination and loop filtering. Normalize by token count so longer answers do not dominate other domains. Bounded component only; full experiments are not reproduced.
• 2026-10-14 · First author: Xuanchang Zhang · Gibbs-Gap Policy Optimization for Off-Policy RL with Verifiable Rewards (ggpo): GGPO measures deviation by the difference between log-mean-exp and the arithmetic mean. It does not multiply updates by learner/behavior importance weights and periodically refreshes the reference to change subsequent targets. Bounded component only; full experiments are not reproduced.
• 2026-10-01 · First author: Ziyi Chen · Dependency-Aware Reward Shaping for Agentic Reinforcement Learning (dars): Verification, invalidation, and repair update it. The nearest broken dependency discounts prior credit by graph distance without penalizing independent branches.
• 2026-09-29 · First author: Zhenrui Yue · SIPO: Unifying Reinforcement Learning with On-Policy Self-Distillation (sipo): Their log-probability difference cancels shared bias. Environment reward sets the main direction; dense evidence redistributes credit across tokens.
• 2026-09-28 · First author: Haojin Wang · Learning from Teacher Continuations at Student States (olive): Have the current student generate a prefix, then ask the teacher to continue from that student state. Train on teacher continuation tokens only.

University of Luxembourg

• 2026-09-29 · First author: Hongyang Li · Unlocking the Critic: Reward-Free Policy Optimization for LLM Post-Training (rfpo): Freeze a calibrated critic and reuse prefix success probability as rollout reward, GAE baseline, and unfinished-trajectory forecast. Remove length bias before binarization to limit exploitation of critic bias.

University of Maryland

• 2026-06-02 · First author: Rishanth Rajendhran · POLARIS: Guiding Small Models to Write Long Stories (polaris-hri): The same external rubric grades the policy and reference. Choosing the exact clipped branch in log-ratio space avoids overflow leading to apparently finite forward values and NaN gradients without changing the objective. Bounded component only; full experiments are not reproduced.

University of Maryland, College Park

• 2026-08-26 · First author: Kaishen Wang · Where to Look Matters: On-Policy Self-Distillation for Long-Video Understanding (clue-opsd): During training a frozen teacher sees only the question-relevant clue interval. On-policy self-distillation uses the student's own rollouts; inference needs no clues, labels, or external teacher.
• 2026-07-30 · First author: Jiawei Xu · β-OPSD (beta-opsd): Vanilla OPSD is the β=1 case of KL-regularized policy optimization. Geometrically interpolate the reference policy with the privileged teacher, convert the high-variance RL solution to token-logit distillation, and use return-to-go for long-trace credit.

University of Massachusetts

• 2026-10-20 · First author: Shripad V. Deshmukh · Projected Distillation: On Learning to Solve Hard Problems from Close, Correct Targets (projected-distillation): Successful student answers separately receive SFT; the paper resamples each round. Minimal editing approximates, rather than exactly solves, finding the most likely correct student answer. Bounded component only; full experiments are not reproduced.

University of Minnesota

• 2026-10-15 · First author: Skylar Zhai · Each Reward to Its Relevant Tokens: Reward-Specific Token-Level Credit Assignment for Multi-Reward RL (mrtpo): The more that removal changes a token's distribution, the more of the reward's credit it receives, whether the advantage is positive or negative. Section 4 Bounded component only; full experiments are not reproduced.

University of North Carolina at Chapel Hill

• 2026-09-28 · First author: Shangzhe Li · An RL View of OPD: Least Square Policy Distillation for Sample-Efficient LLM Reasoning (lspd): Use teacher–student log-probability residuals, squared near zero and linear in the tails, with reuse of historical query–response trajectories.

University of Notre Dame / Amazon

• 2026-08-05 · First author: Zheyuan Zhang · Optimizing What Policies Learn From: Recoverability-Aware Rollout Intervention Learning (rail): Topic: rollout-budget allocation. Uniform rollouts waste budget, and static heuristics cannot track a changing policy. RAIL treats intervention timing and type as a contextual bandit and learns a recoverability controller from shadow-to-live trajectories.

University of Pennsylvania

• 2026-10-22 · First author: Mahdi Sabbaghi · When to Follow the Teacher: Advantage-Based On-Policy Distillation (aopd): Estimate its advantage from existing student rollouts, terminal rewards and teacher importance ratios, without counterfactual continuations. Keep the frozen rollout policy separate from the updated student. Bounded component only; full experiments are not reproduced.
• 2026-10-08 · First author: Lei Zhao · Why On-Policy Distillation Sometimes Fails: Vanishing Learning Signals (opd-learning-signals): Track loss, pre-clip gradients, two-fold occupancy, displacement and layerwise CKA. These correlations are not causal.
• 2026-10-02 · First author: Lyuxin David Zhang · LESSER: Post-Training Data Selection with Output-Layer Gradients (lesser): LESSER replaces features, not the selector: one forward yields hidden states and probabilities, from which output gradients are calculated analytically. SFT, RL question selection, and teacher selection rules can remain unchanged.

University of Pittsburgh

• 2026-08-27 · First author: Zhuochun Li · SPEAR: Distilling Domain-Adaptive Reasoning Skeletons via Sequential Symbolic Alignment in Reinforcement Learning (spear): Outcome rewards are sparse and neural process reward models are expensive. Project teacher reasoning into domain-specific symbolic milestones and reward student traces with order-sensitive longest-common-subsequence F1.

University of Science and Technology of China

• 2026-09-28 · First author: Qiyong Zhong · MAS-OPD: On-Policy Distillation for Multi-Agent Systems (mas-opd): Contrast teacher signals for target versus non-target roles to construct role advantage. Give conflict attributions only to the teacher, forming privileged coordination supervision.
• 2026-08-04 · First author: Ranxu Zhang · ADRS (adrs): ADRS normalizes teacher scores within each interaction step, gates them by correlation with realized returns, then injects the gated token signal into the native reward-to-advantage path. No skills are needed at inference.
• 2026-07-11 · First author: Kexin Huang · ARMOR (armor): ARMOR actively samples anchor trajectories from a frozen reference and mixes them with current-policy rollouts. Data, rather than an auxiliary KL term, stabilize long-horizon RL.

University of Science and Technology of China / Ant Group

• 2026-09-13 · First author: Zhiyu Gui · Know When to Stop, Where to Restart: Accelerating Multi-Turn Agentic On-Policy Distillation (stride-opd): Stop early when accumulated teacher confidence crosses the threshold, then restart at the cached prefix of the weakest correct turn.

University of Southern California

• 2026-10-18 · First author: Jike Zhong · Beyond the Teacher: Understanding and Preserving Distillation Overshoot in Agentic On-Policy Distillation (spod): All other decisions are distilled normally. See Sections 3–4. Bounded component only; full experiments are not reproduced.
• 2026-09-29 · First author: Rishabh Agrawal · AdviSD: Learning to Advise Frontier LLMs via Targeted Multi-Turn Self-Distillation (advisd): Reflection proposes a correction; score the same logged executor response with and without advice. Distill only decisions with sufficient effect under donor-advice calibration.
• 2026-01-19 · First author: Duygu Nur Yaldiz · Balancing Classification and Calibration Performance in Decision-Making LLMs via Calibration Aware Reinforcement Learning (calibration-aware-rl): The paper adjusts decision-token probabilities directly to reduce ECE without sacrificing accuracy. The repository reports accuracy alongside Brier score and ECE, fits temperature only on validation, and forbids test-set tuning.

University of Virginia

• 2026-09-10 · First author: Rongcan Pei · Negative Self-Distillation: Learning to Reason by Avoiding Flaws (nsd): Move the student away from its own flawed reasoning distribution. A dynamic gate limits updates to reasoning-relevant positions to avoid forgetting ordinary language.

University of Washington

• 2026-10-08 · First author: Ximing Lu · MagicBeanStalk: Escaping the RLVR Zero Advantage Trap with Recursive Data Synthesis (magicbeanstalk): MBS extracts reasoning blueprints from verified hard seeds, decomposes them top-down, measures whether solved children improve parent accuracy, synthesizes harder novel leaves, and composes them bottom-up according to the original dependencies. Independent validation must preserve the blueprint and meaningful novelty, then the target policy must have mixed successes and failures with at least one complete, correct solution within budget. Not reproduced for now due to cost; validation pending, no executable entry.
• 2026-09-10 · First author: Dongfang Zhao · LOCUS: Task-Aware Low-Rank Post-Training for Token-Efficient Language Generation (locus): Learn a task-related low-rank post-training subspace. Updating only compact directions shortens responses while retaining task capabilities.
• 2025-04-21 · First author: Jianhao Yan · LUFFY (luffy): Put high-quality offline reasoning and online rollouts on the same support. Regularize importance ratios to retain on-policy behavior.

VNU University of Engineering and Technology / Viettel AI

• 2026-08-05 · First author: Nhat Minh Pham · SpecRoll: Fast-Slow Verifier-Feedback Adaptation for Speculative Reinforcement Learning Rollouts (specroll): SpecRoll proposes with future-token heads, corrects hidden states through a no-backpropagation fast Reflex path, and activates slow head updates only under sustained degradation. An exact verifier preserves the sampling distribution.

Valeo.ai

• 2026-10-01 · First author: Sophia Sirko-Galouchenko · Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenes (where-opd): Give the teacher textual spatial privilege while the student sees only image and question. Distill teacher scores on the student's on-policy tokens.

Waseda University

• 2026-09-15 · First author: Takayuki Yamamoto · Style-Debiased DPO: Updating LLM Knowledge with Factuality-Aware Synthetic Preference Data (sd-dpo): Identify rejected responses that are factually correct but stylistically different, reverse their preference direction, and weight groups proportionally to cancel the style gradient.

WeChat / Tencent

• 2026-07-22 · First author: Beining Wang · Co-Evolving LLM Evaluators and Policies via DynamicRubric (dynamic-rubric): A discriminability objective targets criteria that distinguish current hard negatives, while an anchor objective limits evaluator drift. The evaluator and policy then co-evolve over multiple rounds.

WeChat AI, Tencent / Peking University

• 2026-09-15 · First author: Minghua He · GrowMTP: Can RL Grow Its Own Draft Head? (growmtp): During RL rollouts, train the multi-token prediction head by accepted draft depth and decouple its gradient from the backbone.

WeChat Vision, Tencent

• 2026-09-29 · First author: Youxu Shi · Interpolated Policy Distillation: A Controllable Continuum Between Off-Policy and On-Policy Distillation (ipd): Interpolate student and teacher at every token state as \(m_\gamma=(1-\gamma)\pi_S+\gamma\pi_T\), giving a continuum from pure on-policy student to off-policy teacher. The paper samples this target policy exactly using a verification-aware speculative sampler.

Wuhan University

• 2026-08-24 · First author: Jialong Liu · SRPO: Self-Reflective Policy Optimization for Long-Horizon Reasoning (srpo): SRPO asks the current model to write a short reflection patch from the complete trajectory and environment outcome, then adds that patch to the original prompt. The same model acts as a teacher with this privileged context and gives per-token scores to the student's on-policy rollout.

Xiaohongshu

• 2026-09-10 · First author: Suwan Wu · A Unified Per-Token Gating Family for On-Policy Distillation: FKL/RKL Mixing with Multi-Channel and Bias Coefficients (adaptive-opd-gate): Combine entropy, uncertainty, reference deviation, and teacher–student gap in a per-token gate, adaptively assigning weight between forward and reverse KL.

Xiaomi

• 2026-06-29 · First author: Wenhan Ma · MOPD (mopd): Joint RL couples domains; merging parameters or off-policy tuning can lose capabilities. Train specialists independently, then mix dense teacher signals only on student rollouts so domains can evolve in parallel.

Yonsei University

• 2026-10-01 · First author: Ryunyi Lee · Range-GRPO: Policy Optimization via Pairwise Relations among Reward Intervals (range-grpo): Use conformal reward intervals instead of point judge scores. Give a directional signal only for determinately ranked pairs; overlapping intervals do not create spurious preference.

Zhejiang University

• 2026-10-16 · First author: Kai Tang · SimOPD: Relational Knowledge Transfer for On-Policy Distillation (simopd): Teacher and student widths can differ without any cross-model projection. See Section 3 and Algorithm 1. Bounded component only; full experiments are not reproduced.
• 2026-10-15 · First author: Licheng Pan · CleanRM: Reward Modeling from Noisy Feedback with a Few Clean Feedback for RLHF (clean-rm): It performs optimal transport on the joint distribution of semantics and recovered preferences instead of matching only preference histograms. It then freezes the estimator and trains a reward model with a debiased surrogate. Bounded component only; full experiments are not reproduced.
• 2026-10-15 · First author: Siqi Chen · KnowGPO: Knowledge-Grouped Policy Optimization for Search Agents (knowgpo): Actions starting from the same knowledge state are compared by new knowledge and trained alongside final-answer rewards. See Section 3 and Appendix A. Bounded component only; full experiments are not reproduced.
• 2026-09-17 · First author: ZJU-REAL team · RetireOPD: Self-Retiring On-Policy Distillation for Agentic Reinforcement Learning (retire-opd): Assign separate teachers to skills. The student combines RL and on-policy distillation, automatically retiring a teacher once success approaches the teacher's and the distribution gap stops narrowing.
• 2026-08-27 · First author: Aozhe Wang · TTPO: Test-Time Policy Optimization (ttpo): Majority-vote pseudo-labels can be wrong, but rollouts disagreeing with the majority are usually still incorrect. TTPO applies OPSD to agreements and grouped RL to disagreements, separately filtering converged tokens and high-confidence errors.
• 2026-07-28 · First author: Haolei Xu · Relay-OPD (relay-opd): Detect a failing student prefix, let the teacher take over briefly, then return control to the student. A finite relay budget concentrates supervision on critical early positions.
• 2026-06-21 · First author: Pengxiang Cai · CoBA-RL (coba-rl): Standard RLVR may reweight trajectories already accessible to the base model, raising pass@1 without extending the capability boundary visible at high pass@k. CoBA-RL estimates that boundary with repeated sampling, introduces teacher reasoning near or beyond it, and reinforces it with RL.

Zhejiang University / Microsoft

• 2026-10-09 · First author: Shengze Huang · SOAR: Source-Aware Reinforcement Learning with Precise Credit Assignment for Search Agents (soar): Internal recall receives neither earlier reasoning nor external evidence. The policy learns from source agreement, correctness, and format rewards.

Zhongguancun Academy / Tencent / University of Science and Technology of China

• 2026-10-08 · First author: Houcheng Jiang · ReCal: Calibrating Structured Pruning for On-Policy Distillation Recovery (recal): ReCal measures teacher-trajectory damage with a pruned probe, then reweights calibration statistics from the original unpruned model and reselects channels. Keep width and recovery protocol fixed.

Affiliation not listed in the paper

• 2026-09-29 · First author: Hao Li · The Teacher Is a Direction, Not a Destination: Extrapolating RL-Induced Representation Residuals in On-Policy Distillation (ride-opd): Instead of amplifying the teacher–student output probability ratio, take the hidden-state residual from each token/layer of the RL teacher relative to its pre-RL checkpoint. The student regresses to a point beyond the teacher; detach teacher and base so the target branch cannot be updated.

First author's affiliation not listed on the first page

• 2026-10-01 · First author: Yifan Wang · Token-Level Video Reinforcement Learning (tvrl): A frozen VLM's video-input gradients identify which generated tokens most affect reward. Group advantage is then redistributed densely to tokens.

Unconfirmed (reviewed arXiv HTML lists no institution)

• 2026-10-14 · First author: Xinrui Lin · When Pairwise Preferences Cycle: A Reliability-Anchored Preference Walk for Video GRPO (rap-walk): A preference cycle does not necessarily mean low video quality. Different videos may excel at prompt adherence, visual quality, or motion. Bounded component only; full experiments are not reproduced.

Institution not stated in the paper

• 2026-08-03 · First author: Chunji Lv · PCSD (pcsd): PCSD exponentially accumulates teacher-favoring evidence in an adaptive window and decays falling trends. A continuous sigmoid gate is trained jointly with GRPO.
• 2026-07-22 · First author: Xubo Liu · TCR (tcr): TCR constructs a checklist per example and subtracts an exponential moving outcome baseline from process scores. This concentrates updates on reasoning gains unexplained by outcome reward.

Institution not stated in the paper; not inferred from email

• 2026-10-16 · First author: Jihao Huang · MGE-PO: Entropy-Aware Credit Assignment for Mathematical Reasoning (mge-po): It adjusts credit across four quadrants of success/failure and high/low entropy, schedules the curriculum with online problem entropy statistics, and constrains optimization using entropy-weighted reference KL. See Section 4 and Appendix B. Not reproduced due to cost (review complete); research material only, no executable entry.

The Chinese University of Hong Kong, Shenzhen / Shenzhen Research Institute of Big Data

• 2023-10-16 · First author: Ziniu Li · ReMax (remax): LLM rollouts have fast simulation, deterministic token transitions and typically terminal rewards. Drop PPO's value model and subtract the current policy's greedy reward as a prompt-dependent baseline for lower-variance REINFORCE.