跳转至

Read this page in English

LLM 后训练论文与资料索引

本页由统一论文规格生成的 docs/research-manifest.json 构建；元数据维护入口是 打包的 paper.yaml 和按领域保存的 catalog 规格，不直接编辑生成产物。 背景、架构、公式、原文效果和本地实验请进入独立详情页。

论文实现与研究资料

方向 方法 一作机构与日期 原作者代码 本地入口
策略优化 RLiD: Reinforcement Learning in the Decision Plane for Language Agents and LLMs University of Illinois Urbana-Champaign，2026-11-02 未发现官方代码 rlid：仅低成本组件，完整实验不复现
On-policy distillation SpecOPD: Branch-Parallel On-Policy Distillation for Speculative Decoding Qwen Business Unit of Alibaba / University of Science and Technology of China，2026-11-02 未发现官方代码 specopd：仅低成本组件，完整实验不复现
策略优化 CONCUR: Concordant Auxiliary Rewards for Multi-Task, Multi-Reward Reinforcement Learning Carnegie Mellon University，2026-10-27 未发现官方代码 concur：仅低成本组件，完整实验不复现
On-policy distillation Proposing Better Rollouts for On-Policy Distillation University of Illinois Urbana-Champaign，2026-10-27 原作者入口；作者公开仓库当前只有README和assets；这是论文/架构发布页，不是已发布可运行训练源码。固定revision见单篇信息块和来源核对。 mpopd：仅低成本组件，完整实验不复现
On-policy distillation On-Policy Self-Training: Data-Free Self-Improvement in Large Language Models Google，2026-10-23 未发现官方代码 opst：仅低成本组件，完整实验不复现
策略优化 When to Stop Penalizing Actions? Martingale GRPO for VLA Reinforcement Learning The University of Hong Kong，2026-10-23 未发现官方代码 mart-grpo：仅低成本组件，完整实验不复现
On-policy distillation GIFT: Gradient Informed Filtering for Distillation Augmented Reasoning RL Shanghai Jiao Tong University，2026-10-22 原作者入口；原作者源码可见；固定5e9a9dfdb371b7368949abbdc3a846d23a546557未找到第一方LICENSE/COPYING，GitHub license=null；未确认开放许可，不复制作者源码。本地根据论文独立实现 gift：仅低成本组件，完整实验不复现
后训练算法组件 MOER: Mix-Policy Optimization under Expert Prefix Guidance with Selective Rollback City University of Hong Kong，2026-10-22 原作者入口；原作者已发布可见源码；固定7dafbe33118e61597a4f0186d538c1630430c0be，无根LICENSE，仅有嵌套上游许可，作者新增代码许可未确认；本地据论文独立重写，不复制上游代码 moer：仅低成本组件，完整实验不复现
On-policy distillation UserArena: Benchmarking Interactive User Simulation for Long-Horizon Shopping Agents Leiden University，2026-10-22 原作者入口；原作者发布环境、评测及OPTD源码，固定d37e3c062d30950f2a3744c66a18fda9aa72c1f7；非商业研究且禁止未经授权再分发，本地只据论文独立实现损失，不复制上游代码或数据 userarena-optd：仅低成本组件，完整实验不复现
On-policy distillation When to Follow the Teacher: Advantage-Based On-Policy Distillation University of Pennsylvania，2026-10-22 原作者入口；原作者代码已发布，固定3f205368f225bfc4ed4ad007ccbae7879d1b38f2，Apache-2.0；本地据论文独立实现Eq3.8，不混入上游sigmoid residual与all-wrong KL附加项 aopd：仅低成本组件，完整实验不复现
Agentic RL CORD: Conflict-Aware Optimization for Joint RL and Distillation in Agents The Chinese University of Hong Kong，2026-10-21 原作者入口；原论文首页链接原作者开源仓库；固定快照的cord_utils实现保护第一目标的冲突投影，论文另定义交换保护方向与always门控 cord：仅低成本组件，完整实验不复现
On-policy distillation Edit, Don't Imitate: Minimal-Edit On-Policy Distillation for Multimodal Reasoning Meta Superintelligence Labs，2026-10-21 未发现官方代码 minimal-edit-opd：仅低成本组件，完整实验不复现
On-policy distillation Adaptive Pullback: Reweighting Supervision at Penalty Jumps in On-Policy Distillation Nanjing University，2026-10-20 未发现官方代码 ap-opd：仅低成本组件，完整实验不复现
On-policy distillation Ask to Distill: Learning to Synthesize Data for On-Policy Distillation University of Illinois Urbana-Champaign，2026-10-20 原作者入口；原作者公开代码、数据和模型；固定版本核验reward_worker与diversity接口 ask-to-distill：仅低成本组件，完整实验不复现
知识蒸馏 CSD: Common-Support Distillation for Cross-Tokenizer Knowledge Transfer NVIDIA，2026-10-20 原作者入口；正文脚注链接原作者公开PR4680；固定head可读取CSD loss与sub-token工具，核对时PR仍open，不将其标成已合并发布或完整GPU qualification csd：仅低成本组件，完整实验不复现
On-policy distillation On-Policy Transition Distillation The Hong Kong Polytechnic University，2026-10-20 原作者入口；原作者已发布源码；PDF首页链接已核对，仓库包含verl训练代码、配置与测试 optd：仅低成本组件，完整实验不复现
On-policy distillation Projected Distillation: On Learning to Solve Hard Problems from Close, Correct Targets University of Massachusetts，2026-10-20 未发现官方代码 projected-distillation：仅低成本组件，完整实验不复现
On-policy distillation Correctness-Dependent Asymmetric Target Distributions for Self-Distillation Policy Optimization Korea University，2026-10-19 未发现官方代码 apsd：仅低成本组件，完整实验不复现
On-policy distillation Reinforcement Learning with Visually Attributable On-Policy Self-Distillation Baidu Inc.，2026-10-19 未发现官方代码 计划键 visually-attributable-opd：成本过高，不复现（审查 DONE）；仅研究资料，无可执行入口。
On-policy distillation Beyond the Teacher: Understanding and Preserving Distillation Overshoot in Agentic On-Policy Distillation University of Southern California，2026-10-18 原作者入口；作者官方仓库已发布configs、spod训练与sgate方法源码；不是仅占位README spod：仅低成本组件，完整实验不复现
On-policy distillation GR-OPD: Group-Relative Sequence Credit for On-Policy Distillation Shanghai Jiao Tong University，2026-10-16 未发现官方代码 gr-opd：仅低成本组件，完整实验不复现
RLVR MGE-PO: Entropy-Aware Credit Assignment for Mathematical Reasoning 论文未列机构（不据邮箱域名推断），2026-10-16 未发现官方代码 计划键 mge-po：成本过高，不复现（审查 DONE）；仅研究资料，无可执行入口。
Agentic RL MultiAgent-OPD: End-to-End Training of Hierarchical Multi-Agent Systems with On-Policy Distillation Alibaba Group，2026-10-16 未发现官方代码 计划键 multiagent-opd：成本过高，不复现（审查 DONE）；仅研究资料，无可执行入口。
RLVR Preserving Reasoning Diversity in Reinforcement Learning with Verifiable Rewards via Adaptive Replay Cornell University，2026-10-16 原作者入口；作者官方仓库已发布基于verl的recipe/tar训练与memory-bank构建代码 tar-adaptive-replay：仅低成本组件，完整实验不复现
On-policy distillation SimOPD: Relational Knowledge Transfer for On-Policy Distillation Zhejiang University，2026-10-16 未发现官方代码 simopd：仅低成本组件，完整实验不复现
Agentic RL Tree-Aware Posterior Distillation for Long-Horizon Agentic Reinforcement Learning Tsinghua University，2026-10-16 未发现官方代码 treeopsd：仅低成本组件，完整实验不复现
二元反馈对齐 CleanRM: Reward Modeling from Noisy Feedback with a Few Clean Feedback for RLHF Zhejiang University，2026-10-15 原作者入口；论文提供匿名代码入口，但本轮访问失败，未核验其内容、许可证或可运行性；本地按公式独立实现 clean-rm：仅低成本组件，完整实验不复现
On-policy distillation Compatibility Matters: A Framework for Token-Level Teacher Selection in On-Policy Distillation Princeton University，2026-10-15 未发现官方代码 token-teacher-selection：仅低成本组件，完整实验不复现
On-policy distillation Condition-Contrastive Extrapolation for On-Policy Distillation of Language Models The Chinese University of Hong Kong, Shenzhen，2026-10-15 未发现官方代码 cce-opd：仅低成本组件，完整实验不复现
On-policy distillation Does On-Policy Distillation Need to Complete Every Rollout? Cornell University，2026-10-15 未发现官方代码 stopd：仅低成本组件，完整实验不复现
RLHF / RLVR Each Reward to Its Relevant Tokens: Reward-Specific Token-Level Credit Assignment for Multi-Reward RL University of Minnesota，2026-10-15 原作者入口；已找到并核对原作者 MRTPO 官方实现，包含三个任务训练及重放可视化 mrtpo：仅低成本组件，完整实验不复现
Reference anchor EliteReplay: Preserving Diverse Solutions in RL Post-Training via Quality-Diversity Search The Ohio State University，2026-10-15 原作者入口；原作者公开代码与实验资料；本地独立实现目标导向 MAP-Elites 与 replay 算法组件 elite-replay：仅低成本组件，完整实验不复现
Agentic RL KnowGPO: Knowledge-Grouped Policy Optimization for Search Agents Zhejiang University，2026-10-15 原作者入口；原作者官方占位仓库；仅 assets、LICENSE 和 README，方法代码待发布，不能标作训练实现已开源 knowgpo：仅低成本组件，完整实验不复现
On-policy self-distillation Learning from Unverified Peers via Shared Residual Distillation Shanghai Jiao Tong University，2026-10-15 未发现官方代码 shared-residual-distillation：仅低成本组件，完整实验不复现
自置信轨迹过滤 Learning to Reason with Selective Internal Feedback Institute of Automation, Chinese Academy of Sciences，2026-10-15 未发现官方代码 sif：仅低成本组件，完整实验不复现
RLHF / RLVR Multi-Reward Hacking: From Aggregation-Induced Drift to Trust-Anchored Defense University of British Columbia，2026-10-15 未发现官方代码 tag-reward-gating：仅低成本组件，完整实验不复现
Agentic RL OpCredit: Operation-Aware Policy Learning for Agentic Knowledge Graph Question Answering Nanjing University，2026-10-15 原作者入口；原作者已公开 BOC/FTC、训练适配与数据准备代码；配置模板未填，正式评测脚本未随仓库发布 opcredit：仅低成本组件，完整实验不复现
On-policy distillation OPRD: On-Policy Reminder Distillation for Long-Horizon Coding Agents Tsinghua University，2026-10-15 未发现官方代码 oprd-reminder：仅低成本组件，完整实验不复现
过程奖励 Progress Reward Model for Reinforcement Learning via Large Language Models Beihang University，2026-10-15 原作者入口；原作者公开 PRM4RL 仓库；本地按公式独立实现，不执行上游生成的任意 Python progress-reward：仅低成本组件，完整实验不复现
RLHF / RLVR Training Reasoning Models to Generate Diverse Scientific Ideas New York University，2026-10-15 原作者入口；原作者官方占位仓库；当前仅 README，代码、数据和模型待发布，代码许可未确认 diverge-ideation：CPU 评测接口，不执行模型
多教师能力整合 Training-Free On-Policy Distillation University of Illinois Urbana-Champaign，2026-10-15 未发现官方代码 tf-opd：仅低成本组件，完整实验不复现
On-policy distillation Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillation Alibaba Cloud Computing，2026-10-15 原作者入口；原作者公开仓库；未确认明确许可证，不vendor；从公式独立实现 waypoint-opd：仅低成本组件，完整实验不复现
On-policy distillation Divergence-Routed On-Policy Distillation for Language Models Peking University，2026-10-14 未发现官方代码 divergence-routed-opd：仅低成本组件，完整实验不复现
RLHF / RLVR Failure Is the Mother of Success: Recovering Supervision with Grounding Hindsight University of California, San Diego，2026-10-14 未发现官方代码 rshd：仅低成本组件，完整实验不复现
离策略推理 RL Gibbs-Gap Policy Optimization for Off-Policy RL with Verifiable Rewards University of Illinois Urbana-Champaign，2026-10-14 原作者入口；原作者公开Apache-2.0仓库；本地按公式独立实现 ggpo：仅低成本组件，完整实验不复现
On-policy distillation Information-Gain Calibrated On-Policy Distillation for Deep Search Agents The Chinese University of Hong Kong, Shenzhen，2026-10-14 未发现官方代码 ig-opd：仅低成本组件，完整实验不复现
扩散语言模型后训练 On-Policy Residual Learning for Diffusion Language Models Together AI / University of Illinois Urbana-Champaign，2026-10-14 原作者入口；已发布作者ORL代码；系统内核和完整实验仍需对应加速器、模型与训练预算 计划键 orl-diffusion：成本过高，不复现（审查 DONE）；仅研究资料，无可执行入口。
偏好图信用分配 When Pairwise Preferences Cycle: A Reliability-Anchored Preference Walk for Video GRPO 未确认（已读 arXiv HTML 未列机构），2026-10-14 未发现官方代码 rap-walk：仅低成本组件，完整实验不复现
on-policy-self-distillation LoOPSD: On-Policy Self-Distillation with Looped Teachers Anhui Province Key Laboratory of Digital Security，2026-10-13 未发现官方代码 计划键 loopsd：成本过高，暂不复现；待验收，未注册可执行入口。
GRPO difficulty curriculum A Step Harder, Not a Step Too Far: The Benefits and Limits of Curriculum Difficulty Progression in GRPO Dalian Maritime University，2026-10-09 已开源 grpo-difficulty-curriculum
Multi-teacher distillation Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillation Nanjing University / JD.com，2026-10-09 未发现官方代码 road-mopd
Capability boundary BoundaryEvo: Data-Model Co-Evolution at the Capability Boundary for Long-Chain Reasoning Distillation Shanghai Artificial Intelligence Laboratory，2026-10-09 未发现官方代码 boundary-evo
On-policy distillation Branching On-Policy Distillation Tsinghua University / SenseTime Research，2026-10-09 未发现官方代码 branch-opd
Cross-tokenizer distillation CAST: Credit Alignment for Cross-Tokenizer On-Policy Distillation Taobao & Tmall Group of Alibaba，2026-10-09 未发现官方代码 cast-opd
Synthetic self-distillation Continually Post-training Small Models via Synthetic Self-distillation KAIST AI / LG AI Research，2026-10-09 未发现官方代码 synthself
Proximal policy optimization Generalized Proximal Policy Optimization: Unifying LLM RL Algorithms from the Perspective of Divergence Tsinghua University AIR，2026-10-09 未发现官方代码 gppo-framework
Self-correction Inject a Turn, Not a Token: Learning Where to Redirect LLM Reasoning for Self-Correction KAIST，2026-10-09 未发现官方代码 upturn
Monte Carlo reasoning tree Learning More by Training Less: Uncertainty-Guided Progressive Contraction for Efficient Reasoning Post-Training National University of Singapore / Nanjing University，2026-10-09 未发现官方代码 progressive-contraction
Source-aware RL SOAR: Source-Aware Reinforcement Learning with Precise Credit Assignment for Search Agents Zhejiang University / Microsoft，2026-10-09 未发现官方代码 soar
On-policy distillation State-OPD: State-Grounded On-Policy Distillation for Multi-Turn Agents Alibaba Group，2026-10-09 未发现官方代码 state-opd
Trajectory-aware policy optimization Trajectory-Aware Policy Optimization: Closing the State Distribution Gap in Long-Horizon LLM RL Tsinghua University AIR，2026-10-09 未发现官方代码 tapo-trajectory
On-policy distillation DIAL-OPD: Learning More from Fewer Tokens in On-Policy Distillation Eastern Institute of Technology, Ningbo / The Hong Kong Polytechnic University，2026-10-08 已开源 dial-opd
RLVR GRPODropout: Less is More for Online Reinforcement Learning Rollouts Harbin Institute of Technology (Shenzhen)，2026-10-08 已开源 grpo-dropout
RLVR data synthesis MagicBeanStalk: Escaping the RLVR Zero Advantage Trap with Recursive Data Synthesis University of Washington，2026-10-08 未发现官方代码 计划键 magicbeanstalk：成本过高，暂不复现；待验收，未注册可执行入口。
On-policy distillation MetaOPD: Meta-Learned Token Weighting for On-Policy Distillation East China Normal University，2026-10-08 未发现官方代码 meta-opd
On-policy 蒸馏 PIVOT: Perplexity-Informed KD-to-RL Transition Scheduling for Vertical-Domain Few-Shot Distillation Ping An Technology / University of Science and Technology of China，2026-10-08 未发现官方代码 pivot
On-policy distillation ReCal: Calibrating Structured Pruning for On-Policy Distillation Recovery Zhongguancun Academy / Tencent / University of Science and Technology of China，2026-10-08 已开源 recal
RLVR Residual Advantage: Student-Relative Teacher Guidance for RL with Verifiable Rewards Harbin Engineering University / Tencent / Harbin Institute of Technology，2026-10-08 未发现官方代码 residual-advantage
On-policy distillation When Do We Need On-Policy Distillation? Distilling on Offline Student Rollouts Is Often Better UCLA / NVIDIA（论文注明实习），2026-10-08 未发现官方代码 semi-opd
组相对策略优化 When KL Regularization Misfires in Group Policy Optimization Alibaba Group，2026-10-08 未发现官方代码 zcpo
On-policy self-distillation Which Skill to Distill? SGUID: Selecting a Compact Skill Bank for Model–Skill Co-Evolution New York University，2026-10-08 未发现官方代码 sguid
On-policy distillation Why On-Policy Distillation Sometimes Fails: Vanishing Learning Signals University of Pennsylvania，2026-10-08 已开源 opd-learning-signals
Agentic RL Follow the Winners: Conservative Policy Improvement with the Cross-Entropy Method for Critic-Free RFT Trent AI Limited，2026-10-02 未发现官方代码 follow-the-winners
后训练数据选择 LESSER: Post-Training Data Selection with Output-Layer Gradients University of Pennsylvania，2026-10-02 未发现官方代码 lesser
异步 RL Asynchronous LLM Post-Training: Group-Mass Capping and Convergence Analysis The Ohio State University，2026-10-01 未发现官方代码 gmc-grpo
RLVR CARM: Cancellation-Aware Response Masking for LLM Reinforcement Learning Moore Threads AI，2026-10-01 未发现官方代码 carm
Agentic RL Dependency-Aware Reward Shaping for Agentic Reinforcement Learning University of Illinois Urbana-Champaign，2026-10-01 未发现官方代码 dars
偏好优化 GAW-PO: Preference Optimization with Gradient-Aligned Token Weights National University of Science and Technology POLITEHNICA Bucharest，2026-10-01 未发现官方代码 gaw-po
Agentic RL My FAULT: Self-Diagnosis as Credit Assignment in Self-Evolving Agentic Reinforcement Learning Alibaba Group，2026-10-01 未发现官方代码 my-fault
GRPO Range-GRPO: Policy Optimization via Pairwise Relations among Reward Intervals Yonsei University，2026-10-01 未发现官方代码 range-grpo
后验集中与长度约束 RLCPR: Rethinking Probability-Based Reinforcement Learning From Posterior Concentration National University of Singapore，2026-10-01 未发现官方代码 rlcpr
RL 后训练 Sharpening Tax in Post-Training Meta Superintelligence Labs / University of Wisconsin–Madison，2026-10-01 已开源 sharpening-tax
视频生成 Token-Level Video Reinforcement Learning 原文首页未列第一作者机构，2026-10-01 未发现官方代码 tvrl
多模态后训练 Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenes Valeo.ai，2026-10-01 已开源 where-opd
Agentic RL SHARPO: Segment-Level Credit Assignment for Agentic Reinforcement Learning Georgia Institute of Technology，2026-09-30 未发现官方代码 sharpo
轨迹到步骤信用分配 T2SPO: Trajectory-to-Step Policy Optimization for Agentic Reinforcement Learning Nanjing University，2026-09-30 未发现官方代码 t2spo
Advisor AdviSD: Learning to Advise Frontier LLMs via Targeted Multi-Turn Self-Distillation University of Southern California，2026-09-29 未发现官方代码 advisd
On-policy distillation Dr. OPD: Learning What to Follow for Optimal On-Policy Distillation of Large Language Models Rutgers University，2026-09-29 已开源 dr-opd
OPD DriftOPD: Sequence-Level Reverse-KL Distillation for One-Step VLA Policies KAIST，2026-09-29 未发现官方代码 drift-opd
On-policy distillation FlowMap-OPD: Rollout-Kernel Separation for On-Policy Distillation of Few-Step Flow-Map Generators Georgia Institute of Technology，2026-09-29 已开源 flowmap-opd
On-policy distillation From Dissonance to Orchestration: Teacher Intervention in On-Policy Distillation Nanyang Technological University，2026-09-29 已开源 maestro-opd
Agentic RL Guide, Then Let Go: Gap-Adaptive Teacher Scheduling for Sparse-Reward Agentic RL Dalian University of Technology / Kuaishou，2026-09-29 已开源 gats
策略蒸馏 Interpolated Policy Distillation: A Controllable Continuum Between Off-Policy and On-Policy Distillation WeChat Vision, Tencent，2026-09-29 未发现官方代码 ipd
RLVR Learning Beyond What You Sample: Off-Policy-Aware Cross-Model Trajectory Exchange for RLVR KAIST，2026-09-29 未发现官方代码 graft
OPD LEGO-OPD: Factorized Teacher Composition for Multimodal On-Policy Distillation KAIST，2026-09-29 未发现官方代码 lego-opd
On-policy self-distillation Overcoming Scaling Limits in On-Policy Self-Distillation for LLM Reasoning North South University，2026-09-29 未发现官方代码 oasis-opsd
Agentic RL PR-OPD: Privileged Representation On-policy Self-Distillation for Agentic Reinforcement Learning University of Florida，2026-09-29 已开源 pr-opd
RLVR SIPO: Unifying Reinforcement Learning with On-Policy Self-Distillation University of Illinois Urbana-Champaign，2026-09-29 已开源 sipo
On-policy distillation The Teacher Is a Direction, Not a Destination: Extrapolating RL-Induced Representation Residuals in On-Policy Distillation 原文未列机构，2026-09-29 已开源 ride-opd
教师前缀约束蒸馏 The Weakest Link: Distilling LLM Reasoning with Worst-Case Constrained Reinforcement Learning Huawei Noah’s Ark Lab，2026-09-29 未发现官方代码 weakest-link
RL Unlocking the Critic: Reward-Free Policy Optimization for LLM Post-Training University of Luxembourg，2026-09-29 未发现官方代码 rfpo
On-policy distillation An RL View of OPD: Least Square Policy Distillation for Sample-Efficient LLM Reasoning University of North Carolina at Chapel Hill，2026-09-28 已开源 lspd
在线蒸馏 Learning from Teacher Continuations at Student States University of Illinois Urbana-Champaign，2026-09-28 已开源 olive
多 Agent MAS-OPD: On-Policy Distillation for Multi-Agent Systems University of Science and Technology of China，2026-09-28 未发现官方代码 mas-opd
离线 SFT ROSS: Relearning from Self-Generated Rollouts through Selective Supervision AllSpark Team（原文以团队署名，未列机构），2026-09-28 未发现官方代码 ross
自我回顾 Shockingly Simple Self-retrospection Improves Agentic Models Without RL Rensselaer Polytechnic Institute，2026-09-28 未发现官方代码 roft
On-policy self-distillation Recursive Self-Improvement via On-Policy Distillation for Reasoning Meta AI / University of California, Riverside，2026-09-25 未发现官方代码 recursive-opsd
强化学习 PACT: From Credit Assignment to Critic Alignment AllSpark Team，2026-09-22 未发现官方代码 pact
On-policy distillation RetireOPD: Self-Retiring On-Policy Distillation for Agentic Reinforcement Learning Zhejiang University，2026-09-17 已开源 retire-opd
偏好优化 A Zeroth-Order Paradigm for LLM Preference Alignment University of California, Berkeley，2026-09-16 未发现官方代码 compo
On-policy distillation Trajectory Learnability for Offline On-Policy Distillation National University of Singapore，2026-09-16 未发现官方代码 trajectory-learnability
On-policy distillation Beyond Token-Local Imitation: Reward-Compatible Temporal Credit Assignment for On-Policy Distillation Tsinghua University / DiDi Voyager Labs，2026-09-15 未发现官方代码 gamma-opd
RL 训练加速 GrowMTP: Can RL Grow Its Own Draft Head? WeChat AI, Tencent / Peking University，2026-09-15 未发现官方代码 growmtp
多模态后训练 OPD-Aha: From Linguistic Momentum to Visual Reflection in Multimodal On-Policy Distillation Arizona State University / University of Virginia / Stevens Institute of Technology，2026-09-15 已开源 opd-aha
偏好优化 Style-Debiased DPO: Updating LLM Knowledge with Factuality-Aware Synthetic Preference Data Waseda University，2026-09-15 未发现官方代码 sd-dpo
强化学习 TIAO: Token Importance-Aware Policy Optimization for Text Summarization National University of Defense Technology，2026-09-15 已开源 tiao
多轮 Agent 对齐 Turn-level Multiscale Density Ratio Estimation for LLM Agents Alibaba Group，2026-09-15 未发现官方代码 tlm-dre
On-policy distillation Know When to Stop, Where to Restart: Accelerating Multi-Turn Agentic On-Policy Distillation University of Science and Technology of China / Ant Group，2026-09-13 未发现官方代码 stride-opd
On-policy distillation Data-free On-policy Distillation Institute of Automation, Chinese Academy of Sciences / Tencent，2026-09-12 未发现官方代码 df-opd
在线强化学习 MaD-RL: Matching Distributions for Calibrating LLMs with Reinforcement Learning Meta Superintelligence Labs，2026-09-12 未发现官方代码 计划键 mad-rl：成本过高，不复现（审查 DONE）；仅研究资料，无可执行入口。
On-policy distillation A Unified Per-Token Gating Family for On-Policy Distillation: FKL/RKL Mixing with Multi-Channel and Bias Coefficients Xiaohongshu，2026-09-10 未发现官方代码 adaptive-opd-gate
测试时适配 Beyond Confidence: Stability-Aware Test-Time Adaptation for LLM Reasoning Chongqing University，2026-09-10 未发现官方代码 tasco
低秩后训练 LOCUS: Task-Aware Low-Rank Post-Training for Token-Efficient Language Generation University of Washington，2026-09-10 未发现官方代码 locus
负向自蒸馏 Negative Self-Distillation: Learning to Reason by Avoiding Flaws University of Virginia，2026-09-10 已开源 nsd
跨模型族 OPD CompassOPD: Cross-Family On-Policy Distillation via Within-Family Likelihood Shifts Institute of Information Engineering, Chinese Academy of Sciences，2026-09-09 未发现官方代码 compass-opd
概率传输 OPD Distillation as Probability Transport: Routed On-Policy Distillation Tencent，2026-09-08 未发现官方代码 route-opd
弱到强反向蒸馏 Eliciting Weak-to-Strong Generalization with On-Policy Reverse Distillation KAIST AI，2026-09-08 未发现官方代码 oprd
代码测试时 RL Entropy-Regularized Rank-Masked Policy Optimization for Test-Time Reinforcement Learning in Code Generation Nanyang Technological University，2026-09-08 未发现官方代码 probe-erpo
极稀疏 On-policy distillation Extremely Sparse Supervision Incentivizes Reasoning Ability Amazon / Duke University，2026-09-03 未发现官方代码 sparse-opd
弱模型前缀探索 Boosting LLM Exploration via Weak-Model Guidance in RLVR Peking University，2026-08-27 未发现官方代码 weak-guide-rlvr
跨领域 RLVR 能力融合 Consolidating RLVR Capabilities Across Domains: A Deep Dive into Fusion Paradigms Fudan University，2026-08-27 已开源 rlvr-fusion
DPO 偏好/优化尺度解耦 Disentangling Optimization Scale from Preference Scale in DPO Affiliation not listed in the paper，2026-08-27 已开源 normalized-dpo
多教师能力整合 Preserving General Capabilities during Domain Specialization with Uncertainty-Calibrated MOPD Peking University，2026-08-27 未发现官方代码 uc-mopd
过程奖励 SPEAR: Distilling Domain-Adaptive Reasoning Skeletons via Sequential Symbolic Alignment in Reinforcement Learning University of Pittsburgh，2026-08-27 已开源 spear
测试时强化学习 TTPO: Test-Time Policy Optimization Zhejiang University，2026-08-27 已开源 ttpo
证据帧特权自蒸馏 Video-OPSD: Exploiting Privileged Visual Evidence for On-Policy Self-Distillation in Video Large Language Models Nanyang Technological University，2026-08-27 未发现官方代码 video-opsd
混合策略知识注入 RL From Memorization to Absorption: Mixed-Policy RL for Continual Knowledge Injection University of California, Merced，2026-08-26 未发现官方代码 grin
奖励引导参数插值 GRIP: Granular Reward-Guided Parameter Interpolation for Efficient Reasoning Peking University，2026-08-26 未发现官方代码 grip
多模态 rubric RL V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning S-Lab, Nanyang Technological University，2026-08-26 已开源 v-rubrics
长视频特权视图 OPD Where to Look Matters: On-Policy Self-Distillation for Long-Video Understanding University of Maryland, College Park，2026-08-26 未发现官方代码 clue-opsd
搜索增强 OPD + RL OPDSearch+: Search-Enhanced On-Policy Distillation with Reinforcement Learning University of Chinese Academy of Sciences，2026-08-25 未发现官方代码 opd-search-plus
可验证奖励 OPD OPDVR: On-Policy Distillation with Verifiable Rewards LeapLab, Tsinghua University，2026-08-25 已开源 opdvr
输入侧 Query-KL Beyond the Stability-Exploration Dilemma: Environmental Regularization for LLM Policy Optimization AMAP, Alibaba Group，2026-08-24 已开源 erpo
反思式 token 信用 SRPO: Self-Reflective Policy Optimization for Long-Horizon Reasoning Wuhan University，2026-08-24 已开源 srpo
OPD Beyond Imitation: Filtering On-Policy Distillation by Reasoning Progress Authors did not disclose affiliation，2026-08-19 未发现官方代码 r2-opd
多奖励 RL Learn What's Left, Not What's Mastered: Saturation Aware Advantage Reweighting for Multi-Reward Policy Optimization University of Florida，2026-08-17 未发现官方代码 sa-mrpo
多模态上下文偏好校准 Context Blindness in DPO: Mitigating Object Hallucination in MLLMs via Context-Calibrated Preference Optimization Korea University，2026-08-12 已开源 c2-dpo
几何约束 RL GCPO: Diagnosing and Constraining Subspace Geometry in Rollout RL for LLMs Shanghai AI Laboratory，2026-08-12 已开源 gcpo
前瞻偏好树 Preference Tree Optimization: Enhancing Goal-Oriented Dialogue with Look-Ahead Simulations Reichman University，2026-08-12 未发现官方代码 pto
Rubric RL Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL Scale AI，2026-08-12 未发现官方代码 rubric-dropout
OPD SR-OPSD: Self-Referenced On-Policy Self-Distillation Independent Researcher，2026-08-10 未发现官方代码 sr-opsd
自适应自蒸馏 DASH Nanjing University，2026-08-06 已开源 dash
OPD On-Policy Delta Distillation for Multilingual Math Reasoning NAVER AI Lab，2026-08-06 已开源 opd2
推理枢纽蒸馏 RP-OPSD Nanjing University，2026-08-06 已开源 rp-opsd
生成式奖励模型 RRC: Unlocking Generative Reward Models in LLM Reinforcement Learning via Ranking-Based Reward Construction Northeastern University，2026-08-06 已开源 rrc
无监督自蒸馏 U-OPSD University of California, San Diego，2026-08-06 未发现官方代码 u-opsd
rollout 预算分配 Optimizing What Policies Learn From: Recoverability-Aware Rollout Intervention Learning University of Notre Dame / Amazon，2026-08-05 未发现官方代码 rail
RL rollout 加速 SpecRoll: Fast-Slow Verifier-Feedback Adaptation for Speculative Reinforcement Learning Rollouts VNU University of Engineering and Technology / Viettel AI，2026-08-05 已开源 specroll
回报相关奖励塑形 ADRS University of Science and Technology of China，2026-08-04 已开源 adrs
OPD CausalOPD: First-Wrong-Step Supervision for Distilling Causal Chain Reasoning Authors did not disclose affiliation，2026-08-04 未发现官方代码 causal-opd
OPD SMOPD: Multi-Reward Reinforcement Learning via Specialize-and-Merge Online Policy Distillation Alibaba / Qwen，2026-08-04 未发现官方代码 smopd
外部 rollout Beyond On-Policy Exploration: Integrating External Policy Rollouts for Reinforcement Learning in Diffusion Language Models Seoul National University，2026-08-03 未发现官方代码 erils
持续一致性蒸馏 PCSD 论文未列机构，2026-08-03 未发现官方代码 pcsd
OPD Distill Where You Fail: Recovering Learning Signals of Negative RL-Groups from Adaptive Teacher Guidance Tianjin University，2026-08-01 未发现官方代码 rstg
OPD Contrastive Reinforced Policy Optimization via Privileged Self-Distillation Authors did not disclose affiliation，2026-07-30 未发现官方代码 crpo
Context distillation Flux-OPD Peking University，2026-07-30 未发现官方代码 flux-opd
多模态证据归因蒸馏 VAD Shanghai Jiao Tong University，2026-07-30 已开源 vad
On-policy self-distillation β-OPSD University of Maryland, College Park，2026-07-30 未发现官方代码 beta-opsd
分布保持 RL ReCo Seoul National University，2026-07-29 未发现官方代码 reco-grpo
Rubric RL SERPO: Self-Evolving Rubric Policy Optimization for Open-Ended Test-Time Reinforcement Learning Authors did not disclose affiliation，2026-07-29 未发现官方代码 serpo
Token-level credit assignment CoRT ByteDance internship，2026-07-28 未发现官方代码 cort
在线强化学习 Reinforcement Learning for Code Optimization FAIR at Meta / Inria，2026-07-28 未发现官方代码 计划键 code-optimization-rl：成本过高，不复现（审查 DONE）；仅研究资料，无可执行入口。
On-policy distillation Relay-OPD Zhejiang University，2026-07-28 已开源 relay-opd
奖励建模与评估协议 What do Reward Models Memorize? ILLC, University of Amsterdam，2026-07-27 原作者入口；作者正文直接链接ioverho/rm-shortcuts，固定SHA已核验；根树无LICENSE，GitHub license=null，新增作者源码授权未确认。本地据公开论文独立重写，不复制作者代码。 rm-memorization-protocol：仅低成本组件，完整实验不复现
post-training Co-Evolving LLM Evaluators and Policies via DynamicRubric WeChat / Tencent，2026-07-22 未发现官方代码 dynamic-rubric
过程奖励 TCR 论文未列机构，2026-07-22 未发现官方代码 tcr
RLVR ISO: An RLVR-Native Optimization Stack The University of Texas at Austin，2026-07-21 已开源 iso-rlvr
rlvr Off-Context GRPO: Learning to Reason on Hard Problems using Privileged Information Meta AI，2026-07-21 已开源 off-context-grpo
教师奖励重权重 Distilled RL Nankai University，2026-07-19 已开源 distilled-rl
Reference anchor ARMOR University of Science and Technology of China，2026-07-11 未发现官方代码 armor
几何信任域 RIPO Tsinghua University，2026-07-11 未发现官方代码 ripo
Token 信用校准 TACO Johns Hopkins University，2026-07-08 已开源 taco
post-training Turning Off-Policy Tokens On-Policy: A Plug-in Approach for Improving LLM Alignment Renmin University of China，2026-07-06 未发现官方代码 sis
多教师能力整合 MOPD Xiaomi，2026-06-29 未发现官方代码 mopd
能力边界课程 CoBA-RL Zhejiang University，2026-06-21 未发现官方代码 coba-rl
Entropy 稳定 STARE Tsinghua University，2026-06-17 已开源 stare
异步训推失配 KPop Ling / Ring Team，2026-06-13 未发现官方代码 kpop
AR-to-Diffusion 蒸馏 OPDLM Texas A&M University，2026-06-04 未发现官方代码 opd-lm
RLVR与偏好优化 POLARIS: Guiding Small Models to Write Long Stories University of Maryland，2026-06-02 原作者入口；作者训练/奖励/评估代码公开，固定commit；HRI人类故事与推理因版权不发布、主checkpoint gated，不把no-HRI公开权重当主方法完整数据 polaris-hri：仅低成本组件，完整实验不复现
多目标 RL GPRL Stanford University，2026-05-18 未发现官方代码 gprl
对比序列 RL ConSPO Beijing Institute of Technology，2026-05-13 未发现官方代码 conspo
训练流程与资源分配 Beyond GRPO and On-Policy Distillation: An Empirical Sparse-to-Dense Reward Principle for Language-Model Post-Training LinkedIn，2026-05-12 原作者入口；v5正文直接链接HJSang/OPSD_OnPolicyDistillation，固定SHA已核验；根树只有README/scripts/src，未见根LICENSE，GitHub license=null。不能把该OPD子实现等同四阶段完整流程，本地不复制源码。 计划键 sparse-to-dense-reward-principle：成本过高，不复现（审查 DONE）；仅研究资料，无可执行入口。
异步 off-policy Missing Old Logits Tianjin University，2026-05-12 未发现官方代码 missing-old-logits
直接偏好优化 Mind the Gap: Structure-Aware Consistency in Preference Learning Google Research / Courant Institute of Mathematical Sciences，2026-04-30 未发现官方代码 计划键 sa-dpo：成本过高，不复现（审查 DONE）；仅研究资料，无可执行入口。
偏好优化 GroupDPO: Memory-Efficient Group-Wise Direct Preference Optimization Carnegie Mellon University，2026-04-17 未发现官方代码 group-dpo：仅低成本组件，完整实验不复现
On-policy distillation Lightning OPD MIT HAN Lab，2026-04-14 已开源 lightning-opd
多语言后训练 Large Reasoning Models Struggle to Transfer Parametric Knowledge Across Scripts Google Research，2026-03-17 未发现官方代码 计划键 cross-script-knowledge-transfer：成本过高，不复现（审查 DONE）；仅研究资料，无可执行入口。
Context distillation Prompt-Level Distillation: A Non-Parametric Alternative to Model Fine-Tuning for Efficient Reasoning Google，2026-02-24 未发现官方代码 计划键 prompt-level-distillation：成本过高，不复现（审查 DONE）；仅研究资料，无可执行入口。
Context distillation OPCD Microsoft Research，2026-02-12 已开源 opcd
长度无偏 RL LUSPO Meituan，2026-02-05 未发现官方代码 luspo
On-policy self-distillation OPSD University of California, Los Angeles，2026-01-26 已开源 opsd
校准强化学习 Balancing Classification and Calibration Performance in Decision-Making LLMs via Calibration Aware Reinforcement Learning University of Southern California，2026-01-19 未发现官方代码 calibration-aware-rl
纯在线训推校正 Online IcePop Ant Group，2025-12-16 未发现官方代码 online-icepop
稳定 MoE RL Stabilizing RL with LLMs Alibaba Qwen Team，2025-12-01 未发现官方代码 minirl
MoE 训推失配 IcePop Ant Group，2025-10-21 未发现官方代码 icepop
奖励建模与校准 Judging with Confidence: Calibrating Autoraters to Preference Distributions Google，2025-09-30 未发现官方代码 probabilistic-autorater：仅低成本组件，完整实验不复现
LLM评估与校准 When LLMs Benchmark Themselves: Deconstructing Self-Bias in Automated Evaluation Google，2025-09-30 未发现官方代码 benchmark-self-bias：仅低成本组件，完整实验不复现
SFT-RL 动态混合 CHORD Alibaba Group，2025-08-15 已开源 chord
梯度保留 clip GPPO Alibaba Group，2025-08-11 未发现官方代码 gppo
训推失配校正 TIS University of California, San Diego，2025-08-05 未发现官方代码 tis
稳定序列 RL GSPO Alibaba Qwen Team，2025-07-24 已开源 gspo
校准奖励 Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty Massachusetts Institute of Technology，2025-07-22 已开源 rlcr
自博弈课程 SPIRAL Apple，2025-06-30 已开源 spiral
长上下文 RL CISPO / MiniMax-M1 MiniMax，2025-06-16 已开源 cispo
自置信奖励 INTUITOR University of California, Berkeley，2025-05-26 已开源 intuitor
奖励建模与信用分配 SCAR: Shapley Credit Assignment for More Efficient RLHF McGill University，2025-05-26 未发现官方代码 scar-shapley-credit：仅低成本组件，完整实验不复现
零数据自博弈 Absolute Zero Tsinghua University，2025-05-06 已开源 absolute-zero
测试时强化学习 TTRL PRIME-RL author team，2025-04-22 已开源 ttrl
离策略推理 RL LUFFY University of Washington，2025-04-21 已开源 luffy
Critic PPO VAPO ByteDance Seed，2025-04-07 未发现官方代码 vapo
GRPO 聚合偏置 Dr. GRPO SAIL 研究团队，2025-03-26 已开源 dr-grpo
长推理 RL DAPO ByteDance Seed，2025-03-18 已开源 dapo
全局优势估计 REINFORCE++ Independent researchers，2025-01-04 未发现官方代码 reinforce-plus
Reference-free 偏好 SimPO Princeton University，2024-05-23 已开源 simpo
单阶段偏好 ORPO KAIST，2024-03-12 已开源 orpo
经典 RLHF RLOO Cohere For AI，2024-02-22 未发现官方代码 rloo
在线推理 RL DeepSeekMath / GRPO DeepSeek-AI，2024-02-05 已开源 grpo
二元反馈对齐 KTO Contextual AI，2024-02-02 已开源 kto
自奖励 Self-Rewarding LM Meta AI，2024-01-18 未发现官方代码 self-rewarding
自博弈微调 SPIN University of California, Los Angeles，2024-01-02 已开源 spin
自动过程奖励 Math-Shepherd Peking University，2023-12-14 未发现官方代码 math-shepherd
偏好正则 IPO Google DeepMind，2023-10-18 未发现官方代码 ipo
经典 RLHF ReMax 香港中文大学（深圳）/ 深圳市大数据研究院，2023-10-16 已开源 remax
多属性可控 SFT SteerLM NVIDIA，2023-10-09 已开源 steerlm
AI 反馈 RLAIF Google Research，2023-09-01 未发现官方代码 rlaif
经典 On-policy distillation GKD Google DeepMind，2023-06-23 已开源 gkd
Reverse-KL distillation MiniLLM Tsinghua University，2023-06-14 已开源 minillm
过程监督 Let's Verify Step by Step OpenAI，2023-05-31 已开源 process-supervision
直接偏好优化 DPO Stanford University，2023-05-29 已开源 dpo
序列概率校准 SLiC-HF Google DeepMind，2023-05-17 未发现官方代码 slic-hf
Reward 选优微调 RAFT HKUST，2023-04-13 已开源 raft
全排序偏好 RRHF Alibaba DAMO Academy，2023-04-11 已开源 rrhf
AI 反馈安全对齐 Constitutional AI Anthropic，2022-12-15 已开源 constitutional-ai
经典 RLHF InstructGPT / PPO-RLHF OpenAI，2022-03-04 已开源 ppo-rlhf

分类浏览：