LLM 后训练论文与资料索引¶
本页由 docs/research-manifest.json 自动生成；论文元数据只在统一 manifest
维护。背景、架构、公式、原文效果和本地实验请进入独立详情页。
已实现论文与资料¶
| 方向 | 方法 | 一作机构与日期 | 原作者代码 | 本地入口 |
|---|---|---|---|---|
| On-policy distillation | RetireOPD: Self-Retiring On-Policy Distillation for Agentic Reinforcement Learning | Zhejiang University，2026-09-17 | 已开源 | retire-opd |
| 偏好优化 | A Zeroth-Order Paradigm for LLM Preference Alignment | University of California, Berkeley，2026-09-16 | 未发现官方代码 | compo |
| On-policy distillation | Trajectory Learnability for Offline On-Policy Distillation | National University of Singapore，2026-09-16 | 未发现官方代码 | trajectory-learnability |
| On-policy distillation | Beyond Token-Local Imitation: Reward-Compatible Temporal Credit Assignment for On-Policy Distillation | Tsinghua University / DiDi Voyager Labs，2026-09-15 | 未发现官方代码 | gamma-opd |
| RL 训练加速 | GrowMTP: Can RL Grow Its Own Draft Head? | WeChat AI, Tencent / Peking University，2026-09-15 | 未发现官方代码 | growmtp |
| 多模态后训练 | OPD-Aha: From Linguistic Momentum to Visual Reflection in Multimodal On-Policy Distillation | Arizona State University / University of Virginia / Stevens Institute of Technology，2026-09-15 | 已开源 | opd-aha |
| 偏好优化 | Style-Debiased DPO: Updating LLM Knowledge with Factuality-Aware Synthetic Preference Data | Waseda University，2026-09-15 | 未发现官方代码 | sd-dpo |
| 强化学习 | TIAO: Token Importance-Aware Policy Optimization for Text Summarization | National University of Defense Technology，2026-09-15 | 已开源 | tiao |
| 多轮 Agent 对齐 | Turn-level Multiscale Density Ratio Estimation for LLM Agents | Alibaba Group，2026-09-15 | 未发现官方代码 | tlm-dre |
| On-policy distillation | Know When to Stop, Where to Restart: Accelerating Multi-Turn Agentic On-Policy Distillation | University of Science and Technology of China / Ant Group，2026-09-13 | 未发现官方代码 | stride-opd |
| On-policy distillation | Data-free On-policy Distillation | Institute of Automation, Chinese Academy of Sciences / Tencent，2026-09-12 | 未发现官方代码 | df-opd |
| On-policy distillation | A Unified Per-Token Gating Family for On-Policy Distillation: FKL/RKL Mixing with Multi-Channel and Bias Coefficients | Xiaohongshu，2026-09-10 | 未发现官方代码 | adaptive-opd-gate |
| 测试时适配 | Beyond Confidence: Stability-Aware Test-Time Adaptation for LLM Reasoning | Chongqing University，2026-09-10 | 未发现官方代码 | tasco |
| 低秩后训练 | LOCUS: Task-Aware Low-Rank Post-Training for Token-Efficient Language Generation | University of Washington，2026-09-10 | 未发现官方代码 | locus |
| 负向自蒸馏 | Negative Self-Distillation: Learning to Reason by Avoiding Flaws | University of Virginia，2026-09-10 | 已开源 | nsd |
| 跨模型族 OPD | CompassOPD: Cross-Family On-Policy Distillation via Within-Family Likelihood Shifts | Institute of Information Engineering, Chinese Academy of Sciences，2026-09-09 | 未发现官方代码 | compass-opd |
| 概率传输 OPD | Distillation as Probability Transport: Routed On-Policy Distillation | Tencent，2026-09-08 | 未发现官方代码 | route-opd |
| 弱到强反向蒸馏 | Eliciting Weak-to-Strong Generalization with On-Policy Reverse Distillation | KAIST AI，2026-09-08 | 未发现官方代码 | oprd |
| 代码测试时 RL | Entropy-Regularized Rank-Masked Policy Optimization for Test-Time Reinforcement Learning in Code Generation | Nanyang Technological University，2026-09-08 | 未发现官方代码 | probe-erpo |
| 极稀疏 On-policy distillation | Extremely Sparse Supervision Incentivizes Reasoning Ability | Amazon / Duke University，2026-09-03 | 未发现官方代码 | sparse-opd |
| 弱模型前缀探索 | Boosting LLM Exploration via Weak-Model Guidance in RLVR | Peking University，2026-08-27 | 未发现官方代码 | weak-guide-rlvr |
| 跨领域 RLVR 能力融合 | Consolidating RLVR Capabilities Across Domains: A Deep Dive into Fusion Paradigms | Fudan University，2026-08-27 | 已开源 | rlvr-fusion |
| DPO 偏好/优化尺度解耦 | Disentangling Optimization Scale from Preference Scale in DPO | Affiliation not listed in the paper，2026-08-27 | 已开源 | normalized-dpo |
| 多教师能力整合 | Preserving General Capabilities during Domain Specialization with Uncertainty-Calibrated MOPD | Peking University，2026-08-27 | 未发现官方代码 | uc-mopd |
| 过程奖励 | SPEAR: Distilling Domain-Adaptive Reasoning Skeletons via Sequential Symbolic Alignment in Reinforcement Learning | University of Pittsburgh，2026-08-27 | 已开源 | spear |
| 测试时强化学习 | TTPO: Test-Time Policy Optimization | Zhejiang University，2026-08-27 | 已开源 | ttpo |
| 证据帧特权自蒸馏 | Video-OPSD: Exploiting Privileged Visual Evidence for On-Policy Self-Distillation in Video Large Language Models | Nanyang Technological University，2026-08-27 | 未发现官方代码 | video-opsd |
| 混合策略知识注入 RL | From Memorization to Absorption: Mixed-Policy RL for Continual Knowledge Injection | University of California, Merced，2026-08-26 | 未发现官方代码 | grin |
| 奖励引导参数插值 | GRIP: Granular Reward-Guided Parameter Interpolation for Efficient Reasoning | Peking University，2026-08-26 | 未发现官方代码 | grip |
| 多模态 rubric RL | V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning | S-Lab, Nanyang Technological University，2026-08-26 | 已开源 | v-rubrics |
| 长视频特权视图 OPD | Where to Look Matters: On-Policy Self-Distillation for Long-Video Understanding | University of Maryland, College Park，2026-08-26 | 未发现官方代码 | clue-opsd |
| 搜索增强 OPD + RL | OPDSearch+: Search-Enhanced On-Policy Distillation with Reinforcement Learning | University of Chinese Academy of Sciences，2026-08-25 | 未发现官方代码 | opd-search-plus |
| 可验证奖励 OPD | OPDVR: On-Policy Distillation with Verifiable Rewards | LeapLab, Tsinghua University，2026-08-25 | 已开源 | opdvr |
| 输入侧 Query-KL | Beyond the Stability-Exploration Dilemma: Environmental Regularization for LLM Policy Optimization | AMAP, Alibaba Group，2026-08-24 | 已开源 | erpo |
| 反思式 token 信用 | SRPO: Self-Reflective Policy Optimization for Long-Horizon Reasoning | Wuhan University，2026-08-24 | 已开源 | srpo |
| OPD | Beyond Imitation: Filtering On-Policy Distillation by Reasoning Progress | Authors did not disclose affiliation，2026-08-19 | 未发现官方代码 | r2-opd |
| 多奖励 RL | Learn What's Left, Not What's Mastered: Saturation Aware Advantage Reweighting for Multi-Reward Policy Optimization | University of Florida，2026-08-17 | 未发现官方代码 | sa-mrpo |
| 多模态上下文偏好校准 | Context Blindness in DPO: Mitigating Object Hallucination in MLLMs via Context-Calibrated Preference Optimization | Korea University，2026-08-12 | 已开源 | c2-dpo |
| 几何约束 RL | GCPO: Diagnosing and Constraining Subspace Geometry in Rollout RL for LLMs | Shanghai AI Laboratory，2026-08-12 | 已开源 | gcpo |
| 前瞻偏好树 | Preference Tree Optimization: Enhancing Goal-Oriented Dialogue with Look-Ahead Simulations | Reichman University，2026-08-12 | 未发现官方代码 | pto |
| Rubric RL | Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL | Scale AI，2026-08-12 | 未发现官方代码 | rubric-dropout |
| OPD | SR-OPSD: Self-Referenced On-Policy Self-Distillation | Independent Researcher，2026-08-10 | 未发现官方代码 | sr-opsd |
| 自适应自蒸馏 | DASH | Nanjing University，2026-08-06 | 已开源 | dash |
| OPD | On-Policy Delta Distillation for Multilingual Math Reasoning | NAVER AI Lab，2026-08-06 | 已开源 | opd2 |
| 推理枢纽蒸馏 | RP-OPSD | Nanjing University，2026-08-06 | 已开源 | rp-opsd |
| 生成式奖励模型 | RRC: Unlocking Generative Reward Models in LLM Reinforcement Learning via Ranking-Based Reward Construction | Northeastern University，2026-08-06 | 已开源 | rrc |
| 无监督自蒸馏 | U-OPSD | University of California, San Diego，2026-08-06 | 未发现官方代码 | u-opsd |
| rollout 预算分配 | Optimizing What Policies Learn From: Recoverability-Aware Rollout Intervention Learning | University of Notre Dame / Amazon，2026-08-05 | 未发现官方代码 | rail |
| RL rollout 加速 | SpecRoll: Fast-Slow Verifier-Feedback Adaptation for Speculative Reinforcement Learning Rollouts | VNU University of Engineering and Technology / Viettel AI，2026-08-05 | 已开源 | specroll |
| 回报相关奖励塑形 | ADRS | University of Science and Technology of China，2026-08-04 | 已开源 | adrs |
| OPD | CausalOPD: First-Wrong-Step Supervision for Distilling Causal Chain Reasoning | Authors did not disclose affiliation，2026-08-04 | 未发现官方代码 | causal-opd |
| OPD | SMOPD: Multi-Reward Reinforcement Learning via Specialize-and-Merge Online Policy Distillation | Alibaba / Qwen，2026-08-04 | 未发现官方代码 | smopd |
| 外部 rollout | Beyond On-Policy Exploration: Integrating External Policy Rollouts for Reinforcement Learning in Diffusion Language Models | Seoul National University，2026-08-03 | 未发现官方代码 | erils |
| 持续一致性蒸馏 | PCSD | 论文未列机构，2026-08-03 | 未发现官方代码 | pcsd |
| OPD | Distill Where You Fail: Recovering Learning Signals of Negative RL-Groups from Adaptive Teacher Guidance | Tianjin University，2026-08-01 | 未发现官方代码 | rstg |
| OPD | Contrastive Reinforced Policy Optimization via Privileged Self-Distillation | Authors did not disclose affiliation，2026-07-30 | 未发现官方代码 | crpo |
| Context distillation | Flux-OPD | Peking University，2026-07-30 | 未发现官方代码 | flux-opd |
| 多模态证据归因蒸馏 | VAD | Shanghai Jiao Tong University，2026-07-30 | 已开源 | vad |
| On-policy self-distillation | β-OPSD | University of Maryland, College Park，2026-07-30 | 未发现官方代码 | beta-opsd |
| 分布保持 RL | ReCo | Seoul National University，2026-07-29 | 未发现官方代码 | reco-grpo |
| Rubric RL | SERPO: Self-Evolving Rubric Policy Optimization for Open-Ended Test-Time Reinforcement Learning | Authors did not disclose affiliation，2026-07-29 | 未发现官方代码 | serpo |
| Token-level credit assignment | CoRT | ByteDance internship，2026-07-28 | 未发现官方代码 | cort |
| On-policy distillation | Relay-OPD | Zhejiang University，2026-07-28 | 已开源 | relay-opd |
| post-training | Co-Evolving LLM Evaluators and Policies via DynamicRubric | WeChat / Tencent，2026-07-22 | 未发现官方代码 | dynamic-rubric |
| 过程奖励 | TCR | 论文未列机构，2026-07-22 | 未发现官方代码 | tcr |
| RLVR | ISO: An RLVR-Native Optimization Stack | The University of Texas at Austin，2026-07-21 | 已开源 | iso-rlvr |
| rlvr | Off-Context GRPO: Learning to Reason on Hard Problems using Privileged Information | Meta AI，2026-07-21 | 已开源 | off-context-grpo |
| 教师奖励重权重 | Distilled RL | Nankai University，2026-07-19 | 已开源 | distilled-rl |
| Reference anchor | ARMOR | University of Science and Technology of China，2026-07-11 | 未发现官方代码 | armor |
| 几何信任域 | RIPO | Tsinghua University，2026-07-11 | 未发现官方代码 | ripo |
| Token 信用校准 | TACO | Johns Hopkins University，2026-07-08 | 已开源 | taco |
| post-training | Turning Off-Policy Tokens On-Policy: A Plug-in Approach for Improving LLM Alignment | Renmin University of China，2026-07-06 | 未发现官方代码 | sis |
| 多教师能力整合 | MOPD | Xiaomi，2026-06-29 | 未发现官方代码 | mopd |
| 能力边界课程 | CoBA-RL | Zhejiang University，2026-06-21 | 未发现官方代码 | coba-rl |
| Entropy 稳定 | STARE | Tsinghua University，2026-06-17 | 已开源 | stare |
| 异步训推失配 | KPop | Ling / Ring Team，2026-06-13 | 未发现官方代码 | kpop |
| AR-to-Diffusion 蒸馏 | OPDLM | Texas A&M University，2026-06-04 | 未发现官方代码 | opd-lm |
| 多目标 RL | GPRL | Stanford University，2026-05-18 | 未发现官方代码 | gprl |
| 对比序列 RL | ConSPO | Beijing Institute of Technology，2026-05-13 | 未发现官方代码 | conspo |
| 异步 off-policy | Missing Old Logits | Tianjin University，2026-05-12 | 未发现官方代码 | missing-old-logits |
| On-policy distillation | Lightning OPD | MIT HAN Lab，2026-04-14 | 已开源 | lightning-opd |
| Context distillation | OPCD | Microsoft Research，2026-02-12 | 已开源 | opcd |
| 长度无偏 RL | LUSPO | Meituan，2026-02-05 | 未发现官方代码 | luspo |
| On-policy self-distillation | OPSD | University of California, Los Angeles，2026-01-26 | 已开源 | opsd |
| 校准强化学习 | Balancing Classification and Calibration Performance in Decision-Making LLMs via Calibration Aware Reinforcement Learning | University of Southern California，2026-01-19 | 未发现官方代码 | calibration-aware-rl |
| 纯在线训推校正 | Online IcePop | Ant Group，2025-12-16 | 未发现官方代码 | online-icepop |
| 稳定 MoE RL | Stabilizing RL with LLMs | Alibaba Qwen Team，2025-12-01 | 未发现官方代码 | minirl |
| MoE 训推失配 | IcePop | Ant Group，2025-10-21 | 未发现官方代码 | icepop |
| SFT-RL 动态混合 | CHORD | Alibaba Group，2025-08-15 | 已开源 | chord |
| 梯度保留 clip | GPPO | Alibaba Group，2025-08-11 | 未发现官方代码 | gppo |
| 训推失配校正 | TIS | University of California, San Diego，2025-08-05 | 未发现官方代码 | tis |
| 稳定序列 RL | GSPO | Alibaba Qwen Team，2025-07-24 | 已开源 | gspo |
| 校准奖励 | Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty | Massachusetts Institute of Technology，2025-07-22 | 已开源 | rlcr |
| 自博弈课程 | SPIRAL | Apple，2025-06-30 | 已开源 | spiral |
| 长上下文 RL | CISPO / MiniMax-M1 | MiniMax，2025-06-16 | 已开源 | cispo |
| 自置信奖励 | INTUITOR | University of California, Berkeley，2025-05-26 | 已开源 | intuitor |
| 零数据自博弈 | Absolute Zero | Tsinghua University，2025-05-06 | 已开源 | absolute-zero |
| 测试时强化学习 | TTRL | PRIME-RL author team，2025-04-22 | 已开源 | ttrl |
| 离策略推理 RL | LUFFY | University of Washington，2025-04-21 | 已开源 | luffy |
| Critic PPO | VAPO | ByteDance Seed，2025-04-07 | 未发现官方代码 | vapo |
| GRPO 聚合偏置 | Dr. GRPO | SAIL 研究团队，2025-03-26 | 已开源 | dr-grpo |
| 长推理 RL | DAPO | ByteDance Seed，2025-03-18 | 已开源 | dapo |
| 全局优势估计 | REINFORCE++ | Independent researchers，2025-01-04 | 未发现官方代码 | reinforce-plus |
| Reference-free 偏好 | SimPO | Princeton University，2024-05-23 | 已开源 | simpo |
| 单阶段偏好 | ORPO | KAIST，2024-03-12 | 已开源 | orpo |
| 经典 RLHF | RLOO | Cohere For AI，2024-02-22 | 未发现官方代码 | rloo |
| 在线推理 RL | DeepSeekMath / GRPO | DeepSeek-AI，2024-02-05 | 已开源 | grpo |
| 二元反馈对齐 | KTO | Contextual AI，2024-02-02 | 已开源 | kto |
| 自奖励 | Self-Rewarding LM | Meta AI，2024-01-18 | 未发现官方代码 | self-rewarding |
| 自博弈微调 | SPIN | University of California, Los Angeles，2024-01-02 | 已开源 | spin |
| 自动过程奖励 | Math-Shepherd | Peking University，2023-12-14 | 未发现官方代码 | math-shepherd |
| 偏好正则 | IPO | Google DeepMind，2023-10-18 | 未发现官方代码 | ipo |
| 经典 RLHF | ReMax | 香港中文大学（深圳）/ 深圳市大数据研究院，2023-10-16 | 已开源 | remax |
| 多属性可控 SFT | SteerLM | NVIDIA，2023-10-09 | 已开源 | steerlm |
| AI 反馈 | RLAIF | Google Research，2023-09-01 | 未发现官方代码 | rlaif |
| 经典 On-policy distillation | GKD | Google DeepMind，2023-06-23 | 已开源 | gkd |
| Reverse-KL distillation | MiniLLM | Tsinghua University，2023-06-14 | 已开源 | minillm |
| 过程监督 | Let's Verify Step by Step | OpenAI，2023-05-31 | 已开源 | process-supervision |
| 直接偏好优化 | DPO | Stanford University，2023-05-29 | 已开源 | dpo |
| 序列概率校准 | SLiC-HF | Google DeepMind，2023-05-17 | 未发现官方代码 | slic-hf |
| Reward 选优微调 | RAFT | HKUST，2023-04-13 | 已开源 | raft |
| 全排序偏好 | RRHF | Alibaba DAMO Academy，2023-04-11 | 已开源 | rrhf |
| AI 反馈安全对齐 | Constitutional AI | Anthropic，2022-12-15 | 已开源 | constitutional-ai |
| 经典 RLHF | InstructGPT / PPO-RLHF | OpenAI，2022-03-04 | 已开源 | ppo-rlhf |
分类浏览：