跳转至

LLM post-training

Back to the paper catalog

Paper titles link to the original publication when available. An English write-up is project documentation, not a claim of full reproduction.

Paper First publication Project write-up
SpecOPD: Branch-Parallel On-Policy Distillation for Speculative Decoding 2026-11-02 English · Chinese
RLiD: Reinforcement Learning in the Decision Plane for Language Agents and LLMs 2026-11-02 English · Chinese
Proposing Better Rollouts for On-Policy Distillation 2026-10-27 English · Chinese
CONCUR: Concordant Auxiliary Rewards for Multi-Task, Multi-Reward Reinforcement Learning 2026-10-27 English · Chinese
When to Stop Penalizing Actions? Martingale GRPO for VLA Reinforcement Learning 2026-10-23 English · Chinese
On-Policy Self-Training: Data-Free Self-Improvement in Large Language Models 2026-10-23 English · Chinese
When to Follow the Teacher: Advantage-Based On-Policy Distillation 2026-10-22 English · Chinese
UserArena: Benchmarking Interactive User Simulation for Long-Horizon Shopping Agents 2026-10-22 English · Chinese
MOER: Mix-Policy Optimization under Expert Prefix Guidance with Selective Rollback 2026-10-22 English · Chinese
GIFT: Gradient Informed Filtering for Distillation Augmented Reasoning RL 2026-10-22 English · Chinese
Edit, Don't Imitate: Minimal-Edit On-Policy Distillation for Multimodal Reasoning 2026-10-21 English · Chinese
CORD: Conflict-Aware Optimization for Joint RL and Distillation in Agents 2026-10-21 English · Chinese
Projected Distillation: On Learning to Solve Hard Problems from Close, Correct Targets 2026-10-20 English · Chinese
On-Policy Transition Distillation 2026-10-20 English · Chinese
CSD: Common-Support Distillation for Cross-Tokenizer Knowledge Transfer 2026-10-20 English · Chinese
Ask to Distill: Learning to Synthesize Data for On-Policy Distillation 2026-10-20 English · Chinese
Adaptive Pullback: Reweighting Supervision at Penalty Jumps in On-Policy Distillation 2026-10-20 English · Chinese
Reinforcement Learning with Visually Attributable On-Policy Self-Distillation 2026-10-19 English · Chinese
Correctness-Dependent Asymmetric Target Distributions for Self-Distillation Policy Optimization 2026-10-19 English · Chinese
Beyond the Teacher: Understanding and Preserving Distillation Overshoot in Agentic On-Policy Distillation 2026-10-18 English · Chinese
Tree-Aware Posterior Distillation for Long-Horizon Agentic Reinforcement Learning 2026-10-16 English · Chinese
SimOPD: Relational Knowledge Transfer for On-Policy Distillation 2026-10-16 English · Chinese
Preserving Reasoning Diversity in Reinforcement Learning with Verifiable Rewards via Adaptive Replay 2026-10-16 English · Chinese
MultiAgent-OPD: End-to-End Training of Hierarchical Multi-Agent Systems with On-Policy Distillation 2026-10-16 English · Chinese
MGE-PO: Entropy-Aware Credit Assignment for Mathematical Reasoning 2026-10-16 English · Chinese
GR-OPD: Group-Relative Sequence Credit for On-Policy Distillation 2026-10-16 English · Chinese
Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillation 2026-10-15 English · Chinese
Training-Free On-Policy Distillation 2026-10-15 English · Chinese
Training Reasoning Models to Generate Diverse Scientific Ideas 2026-10-15 English · Chinese
Progress Reward Model for Reinforcement Learning via Large Language Models 2026-10-15 English · Chinese
OpCredit: Operation-Aware Policy Learning for Agentic Knowledge Graph Question Answering 2026-10-15 English · Chinese
OPRD: On-Policy Reminder Distillation for Long-Horizon Coding Agents 2026-10-15 English · Chinese
Multi-Reward Hacking: From Aggregation-Induced Drift to Trust-Anchored Defense 2026-10-15 English · Chinese
Learning to Reason with Selective Internal Feedback 2026-10-15 English · Chinese
Learning from Unverified Peers via Shared Residual Distillation 2026-10-15 English · Chinese
KnowGPO: Knowledge-Grouped Policy Optimization for Search Agents 2026-10-15 English · Chinese
EliteReplay: Preserving Diverse Solutions in RL Post-Training via Quality-Diversity Search 2026-10-15 English · Chinese
Each Reward to Its Relevant Tokens: Reward-Specific Token-Level Credit Assignment for Multi-Reward RL 2026-10-15 English · Chinese
Does On-Policy Distillation Need to Complete Every Rollout? 2026-10-15 English · Chinese
Condition-Contrastive Extrapolation for On-Policy Distillation of Language Models 2026-10-15 English · Chinese
Compatibility Matters: A Framework for Token-Level Teacher Selection in On-Policy Distillation 2026-10-15 English · Chinese
CleanRM: Reward Modeling from Noisy Feedback with a Few Clean Feedback for RLHF 2026-10-15 English · Chinese
When Pairwise Preferences Cycle: A Reliability-Anchored Preference Walk for Video GRPO 2026-10-14 English · Chinese
On-Policy Residual Learning for Diffusion Language Models 2026-10-14 English · Chinese
Information-Gain Calibrated On-Policy Distillation for Deep Search Agents 2026-10-14 English · Chinese
Gibbs-Gap Policy Optimization for Off-Policy RL with Verifiable Rewards 2026-10-14 English · Chinese
Failure Is the Mother of Success: Recovering Supervision with Grounding Hindsight 2026-10-14 English · Chinese
Divergence-Routed On-Policy Distillation for Language Models 2026-10-14 English · Chinese
LoOPSD: On-Policy Self-Distillation with Looped Teachers 2026-10-13 English · Chinese
Trajectory-Aware Policy Optimization: Closing the State Distribution Gap in Long-Horizon LLM RL 2026-10-09 English · Chinese
State-OPD: State-Grounded On-Policy Distillation for Multi-Turn Agents 2026-10-09 English · Chinese
SOAR: Source-Aware Reinforcement Learning with Precise Credit Assignment for Search Agents 2026-10-09 English · Chinese
Learning More by Training Less: Uncertainty-Guided Progressive Contraction for Efficient Reasoning Post-Training 2026-10-09 English · Chinese
Inject a Turn, Not a Token: Learning Where to Redirect LLM Reasoning for Self-Correction 2026-10-09 English · Chinese
Generalized Proximal Policy Optimization: Unifying LLM RL Algorithms from the Perspective of Divergence 2026-10-09 English · Chinese
Continually Post-training Small Models via Synthetic Self-distillation 2026-10-09 English · Chinese
CAST: Credit Alignment for Cross-Tokenizer On-Policy Distillation 2026-10-09 English · Chinese
Branching On-Policy Distillation 2026-10-09 English · Chinese
BoundaryEvo: Data-Model Co-Evolution at the Capability Boundary for Long-Chain Reasoning Distillation 2026-10-09 English · Chinese
Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillation 2026-10-09 English · Chinese
A Step Harder, Not a Step Too Far: The Benefits and Limits of Curriculum Difficulty Progression in GRPO 2026-10-09 English · Chinese
Why On-Policy Distillation Sometimes Fails: Vanishing Learning Signals 2026-10-08 English · Chinese
Which Skill to Distill? SGUID: Selecting a Compact Skill Bank for Model–Skill Co-Evolution 2026-10-08 English · Chinese
When KL Regularization Misfires in Group Policy Optimization 2026-10-08 English · Chinese
When Do We Need On-Policy Distillation? Distilling on Offline Student Rollouts Is Often Better 2026-10-08 English · Chinese
Residual Advantage: Student-Relative Teacher Guidance for RL with Verifiable Rewards 2026-10-08 English · Chinese
ReCal: Calibrating Structured Pruning for On-Policy Distillation Recovery 2026-10-08 English · Chinese
PIVOT: Perplexity-Informed KD-to-RL Transition Scheduling for Vertical-Domain Few-Shot Distillation 2026-10-08 English · Chinese
MetaOPD: Meta-Learned Token Weighting for On-Policy Distillation 2026-10-08 English · Chinese
MagicBeanStalk: Escaping the RLVR Zero Advantage Trap with Recursive Data Synthesis 2026-10-08 English · Chinese
GRPODropout: Less is More for Online Reinforcement Learning Rollouts 2026-10-08 English · Chinese
DIAL-OPD: Learning More from Fewer Tokens in On-Policy Distillation 2026-10-08 English · Chinese
LESSER: Post-Training Data Selection with Output-Layer Gradients 2026-10-02 English · Chinese
Follow the Winners: Conservative Policy Improvement with the Cross-Entropy Method for Critic-Free RFT 2026-10-02 English · Chinese
Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenes 2026-10-01 English · Chinese
Token-Level Video Reinforcement Learning 2026-10-01 English · Chinese
Sharpening Tax in Post-Training 2026-10-01 English · Chinese
Range-GRPO: Policy Optimization via Pairwise Relations among Reward Intervals 2026-10-01 English · Chinese
RLCPR: Rethinking Probability-Based Reinforcement Learning From Posterior Concentration 2026-10-01 English · Chinese
My FAULT: Self-Diagnosis as Credit Assignment in Self-Evolving Agentic Reinforcement Learning 2026-10-01 English · Chinese
GAW-PO: Preference Optimization with Gradient-Aligned Token Weights 2026-10-01 English · Chinese
Dependency-Aware Reward Shaping for Agentic Reinforcement Learning 2026-10-01 English · Chinese
CARM: Cancellation-Aware Response Masking for LLM Reinforcement Learning 2026-10-01 English · Chinese
Asynchronous LLM Post-Training: Group-Mass Capping and Convergence Analysis 2026-10-01 English · Chinese
T2SPO: Trajectory-to-Step Policy Optimization for Agentic Reinforcement Learning 2026-09-30 English · Chinese
SHARPO: Segment-Level Credit Assignment for Agentic Reinforcement Learning 2026-09-30 English · Chinese
Unlocking the Critic: Reward-Free Policy Optimization for LLM Post-Training 2026-09-29 English · Chinese
The Weakest Link: Distilling LLM Reasoning with Worst-Case Constrained Reinforcement Learning 2026-09-29 English · Chinese
The Teacher Is a Direction, Not a Destination: Extrapolating RL-Induced Representation Residuals in On-Policy Distillation 2026-09-29 English · Chinese
SIPO: Unifying Reinforcement Learning with On-Policy Self-Distillation 2026-09-29 English · Chinese
PR-OPD: Privileged Representation On-policy Self-Distillation for Agentic Reinforcement Learning 2026-09-29 English · Chinese
Overcoming Scaling Limits in On-Policy Self-Distillation for LLM Reasoning 2026-09-29 English · Chinese
Learning Beyond What You Sample: Off-Policy-Aware Cross-Model Trajectory Exchange for RLVR 2026-09-29 English · Chinese
LEGO-OPD: Factorized Teacher Composition for Multimodal On-Policy Distillation 2026-09-29 English · Chinese
Interpolated Policy Distillation: A Controllable Continuum Between Off-Policy and On-Policy Distillation 2026-09-29 English · Chinese
Guide, Then Let Go: Gap-Adaptive Teacher Scheduling for Sparse-Reward Agentic RL 2026-09-29 English · Chinese
From Dissonance to Orchestration: Teacher Intervention in On-Policy Distillation 2026-09-29 English · Chinese
FlowMap-OPD: Rollout-Kernel Separation for On-Policy Distillation of Few-Step Flow-Map Generators 2026-09-29 English · Chinese
DriftOPD: Sequence-Level Reverse-KL Distillation for One-Step VLA Policies 2026-09-29 English · Chinese
Dr. OPD: Learning What to Follow for Optimal On-Policy Distillation of Large Language Models 2026-09-29 English · Chinese
AdviSD: Learning to Advise Frontier LLMs via Targeted Multi-Turn Self-Distillation 2026-09-29 English · Chinese
Shockingly Simple Self-retrospection Improves Agentic Models Without RL 2026-09-28 English · Chinese
ROSS: Relearning from Self-Generated Rollouts through Selective Supervision 2026-09-28 English · Chinese
MAS-OPD: On-Policy Distillation for Multi-Agent Systems 2026-09-28 English · Chinese
Learning from Teacher Continuations at Student States 2026-09-28 English · Chinese
An RL View of OPD: Least Square Policy Distillation for Sample-Efficient LLM Reasoning 2026-09-28 English · Chinese
Recursive Self-Improvement via On-Policy Distillation for Reasoning 2026-09-25 English · Chinese
PACT: From Credit Assignment to Critic Alignment 2026-09-22 English · Chinese
RetireOPD: Self-Retiring On-Policy Distillation for Agentic Reinforcement Learning 2026-09-17 English · Chinese
Trajectory Learnability for Offline On-Policy Distillation 2026-09-16 English · Chinese
A Zeroth-Order Paradigm for LLM Preference Alignment 2026-09-16 English · Chinese
Turn-level Multiscale Density Ratio Estimation for LLM Agents 2026-09-15 English · Chinese
TIAO: Token Importance-Aware Policy Optimization for Text Summarization 2026-09-15 English · Chinese
Style-Debiased DPO: Updating LLM Knowledge with Factuality-Aware Synthetic Preference Data 2026-09-15 English · Chinese
OPD-Aha: From Linguistic Momentum to Visual Reflection in Multimodal On-Policy Distillation 2026-09-15 English · Chinese
GrowMTP: Can RL Grow Its Own Draft Head? 2026-09-15 English · Chinese
Beyond Token-Local Imitation: Reward-Compatible Temporal Credit Assignment for On-Policy Distillation 2026-09-15 English · Chinese
Know When to Stop, Where to Restart: Accelerating Multi-Turn Agentic On-Policy Distillation 2026-09-13 English · Chinese
MaD-RL: Matching Distributions for Calibrating LLMs with Reinforcement Learning 2026-09-12 English · Chinese
Data-free On-policy Distillation 2026-09-12 English · Chinese
Negative Self-Distillation: Learning to Reason by Avoiding Flaws 2026-09-10 English · Chinese
LOCUS: Task-Aware Low-Rank Post-Training for Token-Efficient Language Generation 2026-09-10 English · Chinese
Beyond Confidence: Stability-Aware Test-Time Adaptation for LLM Reasoning 2026-09-10 English · Chinese
A Unified Per-Token Gating Family for On-Policy Distillation: FKL/RKL Mixing with Multi-Channel and Bias Coefficients 2026-09-10 English · Chinese
CompassOPD: Cross-Family On-Policy Distillation via Within-Family Likelihood Shifts 2026-09-09 English · Chinese
Entropy-Regularized Rank-Masked Policy Optimization for Test-Time Reinforcement Learning in Code Generation 2026-09-08 English · Chinese
Eliciting Weak-to-Strong Generalization with On-Policy Reverse Distillation 2026-09-08 English · Chinese
Distillation as Probability Transport: Routed On-Policy Distillation 2026-09-08 English · Chinese
Extremely Sparse Supervision Incentivizes Reasoning Ability 2026-09-03 English · Chinese
Video-OPSD: Exploiting Privileged Visual Evidence for On-Policy Self-Distillation in Video Large Language Models 2026-08-27 English · Chinese
TTPO: Test-Time Policy Optimization 2026-08-27 English · Chinese
SPEAR: Distilling Domain-Adaptive Reasoning Skeletons via Sequential Symbolic Alignment in Reinforcement Learning 2026-08-27 English · Chinese
Preserving General Capabilities during Domain Specialization with Uncertainty-Calibrated MOPD 2026-08-27 English · Chinese
Disentangling Optimization Scale from Preference Scale in DPO 2026-08-27 English · Chinese
Consolidating RLVR Capabilities Across Domains: A Deep Dive into Fusion Paradigms 2026-08-27 English · Chinese
Boosting LLM Exploration via Weak-Model Guidance in RLVR 2026-08-27 English · Chinese
Where to Look Matters: On-Policy Self-Distillation for Long-Video Understanding 2026-08-26 English · Chinese
V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning 2026-08-26 English · Chinese
GRIP: Granular Reward-Guided Parameter Interpolation for Efficient Reasoning 2026-08-26 English · Chinese
From Memorization to Absorption: Mixed-Policy RL for Continual Knowledge Injection 2026-08-26 English · Chinese
OPDVR: On-Policy Distillation with Verifiable Rewards 2026-08-25 English · Chinese
OPDSearch+: Search-Enhanced On-Policy Distillation with Reinforcement Learning 2026-08-25 English · Chinese
SRPO: Self-Reflective Policy Optimization for Long-Horizon Reasoning 2026-08-24 English · Chinese
Beyond the Stability-Exploration Dilemma: Environmental Regularization for LLM Policy Optimization 2026-08-24 English · Chinese
Beyond Imitation: Filtering On-Policy Distillation by Reasoning Progress 2026-08-19 English · Chinese
Learn What's Left, Not What's Mastered: Saturation Aware Advantage Reweighting for Multi-Reward Policy Optimization 2026-08-17 English · Chinese
Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL 2026-08-12 English · Chinese
Preference Tree Optimization: Enhancing Goal-Oriented Dialogue with Look-Ahead Simulations 2026-08-12 English · Chinese
GCPO: Diagnosing and Constraining Subspace Geometry in Rollout RL for LLMs 2026-08-12 English · Chinese
Context Blindness in DPO: Mitigating Object Hallucination in MLLMs via Context-Calibrated Preference Optimization 2026-08-12 English · Chinese
SR-OPSD: Self-Referenced On-Policy Self-Distillation 2026-08-10 English · Chinese
U-OPSD 2026-08-06 English · Chinese
RRC: Unlocking Generative Reward Models in LLM Reinforcement Learning via Ranking-Based Reward Construction 2026-08-06 English · Chinese
RP-OPSD 2026-08-06 English · Chinese
On-Policy Delta Distillation for Multilingual Math Reasoning 2026-08-06 English · Chinese
DASH 2026-08-06 English · Chinese
SpecRoll: Fast-Slow Verifier-Feedback Adaptation for Speculative Reinforcement Learning Rollouts 2026-08-05 English · Chinese
Optimizing What Policies Learn From: Recoverability-Aware Rollout Intervention Learning 2026-08-05 English · Chinese
SMOPD: Multi-Reward Reinforcement Learning via Specialize-and-Merge Online Policy Distillation 2026-08-04 English · Chinese
CausalOPD: First-Wrong-Step Supervision for Distilling Causal Chain Reasoning 2026-08-04 English · Chinese
ADRS 2026-08-04 English · Chinese
PCSD 2026-08-03 English · Chinese
Beyond On-Policy Exploration: Integrating External Policy Rollouts for Reinforcement Learning in Diffusion Language Models 2026-08-03 English · Chinese
Distill Where You Fail: Recovering Learning Signals of Negative RL-Groups from Adaptive Teacher Guidance 2026-08-01 English · Chinese
β-OPSD 2026-07-30 English · Chinese
VAD 2026-07-30 English · Chinese
Flux-OPD 2026-07-30 English · Chinese
Contrastive Reinforced Policy Optimization via Privileged Self-Distillation 2026-07-30 English · Chinese
SERPO: Self-Evolving Rubric Policy Optimization for Open-Ended Test-Time Reinforcement Learning 2026-07-29 English · Chinese
ReCo 2026-07-29 English · Chinese
Relay-OPD 2026-07-28 English · Chinese
Reinforcement Learning for Code Optimization 2026-07-28 English · Chinese
CoRT 2026-07-28 English · Chinese
What do Reward Models Memorize? 2026-07-27 English · Chinese
TCR 2026-07-22 English · Chinese
Co-Evolving LLM Evaluators and Policies via DynamicRubric 2026-07-22 English · Chinese
Off-Context GRPO: Learning to Reason on Hard Problems using Privileged Information 2026-07-21 English · Chinese
ISO: An RLVR-Native Optimization Stack 2026-07-21 English · Chinese
Distilled RL 2026-07-19 English · Chinese
RIPO 2026-07-11 English · Chinese
ARMOR 2026-07-11 English · Chinese
TACO 2026-07-08 English · Chinese
Turning Off-Policy Tokens On-Policy: A Plug-in Approach for Improving LLM Alignment 2026-07-06 English · Chinese
MOPD 2026-06-29 English · Chinese
CoBA-RL 2026-06-21 English · Chinese
STARE 2026-06-17 English · Chinese
KPop 2026-06-13 English · Chinese
OPDLM 2026-06-04 English · Chinese
POLARIS: Guiding Small Models to Write Long Stories 2026-06-02 English · Chinese
GPRL 2026-05-18 English · Chinese
ConSPO 2026-05-13 English · Chinese
Missing Old Logits 2026-05-12 English · Chinese
Beyond GRPO and On-Policy Distillation: An Empirical Sparse-to-Dense Reward Principle for Language-Model Post-Training 2026-05-12 English · Chinese
Mind the Gap: Structure-Aware Consistency in Preference Learning 2026-04-30 English · Chinese
GroupDPO: Memory-Efficient Group-Wise Direct Preference Optimization 2026-04-17 English · Chinese
Lightning OPD 2026-04-14 English · Chinese
Large Reasoning Models Struggle to Transfer Parametric Knowledge Across Scripts 2026-03-17 English · Chinese
Prompt-Level Distillation: A Non-Parametric Alternative to Model Fine-Tuning for Efficient Reasoning 2026-02-24 English · Chinese
OPCD 2026-02-12 English · Chinese
LUSPO 2026-02-05 English · Chinese
OPSD 2026-01-26 English · Chinese
Balancing Classification and Calibration Performance in Decision-Making LLMs via Calibration Aware Reinforcement Learning 2026-01-19 English · Chinese
Online IcePop 2025-12-16 English · Chinese
Stabilizing RL with LLMs 2025-12-01 English · Chinese
IcePop 2025-10-21 English · Chinese
When LLMs Benchmark Themselves: Deconstructing Self-Bias in Automated Evaluation 2025-09-30 English · Chinese
Judging with Confidence: Calibrating Autoraters to Preference Distributions 2025-09-30 English · Chinese
CHORD 2025-08-15 English · Chinese
GPPO 2025-08-11 English · Chinese
TIS 2025-08-05 English · Chinese
GSPO 2025-07-24 English · Chinese
Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty 2025-07-22 English · Chinese
SPIRAL 2025-06-30 English · Chinese
CISPO / MiniMax-M1 2025-06-16 English · Chinese
SCAR: Shapley Credit Assignment for More Efficient RLHF 2025-05-26 English · Chinese
INTUITOR 2025-05-26 English · Chinese
Absolute Zero 2025-05-06 English · Chinese
TTRL 2025-04-22 English · Chinese
LUFFY 2025-04-21 English · Chinese
VAPO 2025-04-07 English · Chinese
Dr. GRPO 2025-03-26 English · Chinese
DAPO 2025-03-18 English · Chinese
REINFORCE++ 2025-01-04 English · Chinese
SimPO 2024-05-23 English · Chinese
ORPO 2024-03-12 English · Chinese
RLOO 2024-02-22 English · Chinese
DeepSeekMath / GRPO 2024-02-05 English · Chinese
KTO 2024-02-02 English · Chinese
Self-Rewarding LM 2024-01-18 English · Chinese
SPIN 2024-01-02 English · Chinese
Math-Shepherd 2023-12-14 English · Chinese
IPO 2023-10-18 English · Chinese
ReMax 2023-10-16 English · Chinese
SteerLM 2023-10-09 English · Chinese
RLAIF 2023-09-01 English · Chinese
GKD 2023-06-23 English · Chinese
MiniLLM 2023-06-14 English · Chinese
Let's Verify Step by Step 2023-05-31 English · Chinese
DPO 2023-05-29 English · Chinese
SLiC-HF 2023-05-17 English · Chinese
RAFT 2023-04-13 English · Chinese
RRHF 2023-04-11 English · Chinese
Constitutional AI 2022-12-15 English · Chinese
InstructGPT / PPO-RLHF 2022-03-04 English · Chinese