LLM post-training papers and materials

简体中文 · English

This index is built from docs/research-manifest.json, generated from the shared paper specifications. Maintain metadata in packaged paper.yaml files and per-domain catalog specifications, not generated pages. Open a paper's write-up for background, architecture, equations, reported results, and local evidence.

Implemented papers and research materials

Topic	Method	First author's institution and first publication	Author code	Local entry point
Policy optimization	RLiD: Reinforcement Learning in the Decision Plane for Language Agents and LLMs	University of Illinois Urbana-Champaign, 2026-11-02	No verified public repository found as of 2026-11-04. The official abs/HTML has no author repository URL. ldmrl/ and llmdistill/raw/ are artifact paths, not verified open-source links; exact-title/ID search did not establish one	rlid — Low-cost component only; full experiments not reproduced.
On-policy distillation	SpecOPD: Branch-Parallel On-Policy Distillation for Speculative Decoding	Qwen Business Unit of Alibaba / University of Science and Technology of China, 2026-11-02	No verified repository found as of 2026-11-04 in the official abs/HTML or exact-title/GitHub search. Referenced DFlash/Draft-OPD repositories do not establish a SpecOPD code release	specopd — Low-cost component only; full experiments not reproduced.
Policy optimization	CONCUR: Concordant Auxiliary Rewards for Multi-Task, Multi-Reward Reinforcement Learning	Carnegie Mellon University, 2026-10-27	No author repository for CONCUR was found in this review. A training framework or baseline repository is not CONCUR source.	concur — Low-cost component only; full experiments not reproduced.
On-policy distillation	Proposing Better Rollouts for On-Policy Distillation	University of Illinois Urbana-Champaign, 2026-10-27	The pinned author repository contains a README and assets, but no training source or LICENSE. It is not a runnable open-source implementation.	mpopd — Low-cost component only; full experiments not reproduced.
On-policy distillation	On-Policy Self-Training: Data-Free Self-Improvement in Large Language Models	Google, 2026-10-23	No author repository was found in the reviewed v1 abstract/HTML as of 2026-10-26. OPSD and evaluation-harness links are not author source. The paper is CC BY 4.0.	opst — Low-cost component only; full experiments not reproduced.
Policy optimization	When to Stop Penalizing Actions? Martingale GRPO for VLA Reinforcement Learning	The University of Hong Kong, 2026-10-23	No method repository in the reviewed v1 as of 2026-10-26. Referenced SimpleVLA-RL is not author source. No source license/revision verified; the paper's arXiv license is not a software license.	mart-grpo — Low-cost component only; full experiments not reproduced.
On-policy distillation	GIFT: Gradient Informed Filtering for Distillation Augmented Reasoning RL	Shanghai Jiao Tong University, 2026-10-22	lilifree/GIFT at 5e9a9dfdb371b7368949abbdc3a846d23a546557. No first-party license found; vendored licenses do not cover author source. Independent implementation.	gift — Low-cost component only; full experiments not reproduced.
Post-training algorithm components	MOER: Mix-Policy Optimization under Expert Prefix Guidance with Selective Rollback	City University of Hong Kong, 2026-10-22	CityU-MLO/MOER at 7dafbe33118e61597a4f0186d538c1630430c0be. No root license verified; nested luffy/verl licenses do not cover all new code. Independent formula implementation.	moer — Low-cost component only; full experiments not reproduced.
On-policy distillation	UserArena: Benchmarking Interactive User Simulation for Long-Horizon Shopping Agents	Leiden University, 2026-10-22	RecGPT-Team/UserArena, environment/evaluation/OPTD at d37e3c062d30950f2a3744c66a18fda9aa72c1f7; non-commercial research/data agreement, no unauthorized redistribution. Independent formula implementation; no source, user records or images copied.	userarena-optd — Low-cost component only; full experiments not reproduced.
On-policy distillation	When to Follow the Teacher: Advantage-Based On-Policy Distillation	University of Pennsylvania, 2026-10-22	Helloworld10011/AOPD at 3f205368f225bfc4ed4ad007ccbae7879d1b38f2; Apache-2.0. Independent implementation, not the upstream training environment.	aopd — Low-cost component only; full experiments not reproduced.
Agentic RL	CORD: Conflict-Aware Optimization for Joint RL and Distillation in Agents	The Chinese University of Hong Kong, 2026-10-21	CORD-release, Apache-2.0 at ba09683.	cord — Low-cost component only; full experiments not reproduced.
On-policy distillation	Edit, Don't Imitate: Minimal-Edit On-Policy Distillation for Multimodal Reasoning	Meta Superintelligence Labs, 2026-10-21	No verifiable author source in v1 or exact-ID GitHub search as of 2026-10-25. This is not proof no release exists elsewhere. Independent implementation; paper and figure CC BY 4.0.	minimal-edit-opd — Low-cost component only; full experiments not reproduced.
On-policy distillation	Adaptive Pullback: Reweighting Supervision at Penalty Jumps in On-Policy Distillation	Nanjing University, 2026-10-20	No author source link verified in reviewed HTML/PDF; referenced code is not this method.	ap-opd — Low-cost component only; full experiments not reproduced.
On-policy distillation	Ask to Distill: Learning to Synthesize Data for On-Policy Distillation	University of Illinois Urbana-Champaign, 2026-10-20	Ask_to_Distill, Apache-2.0, 1edaf22fc8e2ba35efe6c02fa1a3665875f1f98a.	ask-to-distill — Low-cost component only; full experiments not reproduced.
Knowledge distillation	CSD: Common-Support Distillation for Cross-Tokenizer Knowledge Transfer	NVIDIA, 2026-10-20	NVIDIA-NeMo/RL PR #4680, Apache-2.0, pinned head. Loss/subtoken utilities visible in the open PR, not a merged release or completed upstream GPU qualification.	csd — Low-cost component only; full experiments not reproduced.
On-policy distillation	On-Policy Transition Distillation	The Hong Kong Polytechnic University, 2026-10-20	ModalityDance/optd, MIT at eb8a29a5f6eeb64b48714bd31d880aeae1ea5ff7, with verl implementation/config/tests and separate third-party notices.	optd — Low-cost component only; full experiments not reproduced.
On-policy distillation	Projected Distillation: On Learning to Solve Hard Problems from Close, Correct Targets	University of Massachusetts, 2026-10-20	No author source found in reviewed text/abstract as of 2026-10-21. Referenced TRL is not this method.	projected-distillation — Low-cost component only; full experiments not reproduced.
On-policy distillation	Correctness-Dependent Asymmetric Target Distributions for Self-Distillation Policy Optimization	Korea University, 2026-10-19	The paper links to an anonymous APSD materials repository. Its code could not be accessed and verified in this review; not confirmed as open source.	apsd — Low-cost component only; full experiments not reproduced.
On-policy distillation	Reinforcement Learning with Visually Attributable On-Policy Self-Distillation	Baidu Inc., 2026-10-19	No author training source was found. The authors released VAG-Qwen3.5-4B and a 9B checkpoint; weights are not open-source training code.	Planned key visually-attributable-opd — Not reproduced due to cost (review complete); research material only, no executable entry.
On-policy distillation	Beyond the Teacher: Understanding and Preserving Distillation Overshoot in Agentic On-Policy Distillation	University of Southern California, 2026-10-18	The official SPOD repository includes configs, spod training code, and sgate method code.	spod — Low-cost component only; full experiments not reproduced.
On-policy distillation	GR-OPD: Group-Relative Sequence Credit for On-Policy Distillation	Shanghai Jiao Tong University, 2026-10-16	No verifiable author GR-OPD method repository was found in this review.	gr-opd — Low-cost component only; full experiments not reproduced.
RLVR	MGE-PO: Entropy-Aware Credit Assignment for Mathematical Reasoning	Institution not stated in the paper; not inferred from email, 2026-10-16	No verifiable author method repository found in this review.	Planned key mge-po — Not reproduced due to cost (review complete); research material only, no executable entry.
Agentic RL	MultiAgent-OPD: End-to-End Training of Hierarchical Multi-Agent Systems with On-Policy Distillation	Alibaba Group, 2026-10-16	The paper mentions anonymous materials. No verifiable public training-source URL or license was found in this review.	Planned key multiagent-opd — Not reproduced due to cost (review complete); research material only, no executable entry.
RLVR	Preserving Reasoning Diversity in Reinforcement Learning with Verifiable Rewards via Adaptive Replay	Cornell University, 2026-10-16	Official TAR repository, including the verl-based recipe/tar training and memory-bank construction code	tar-adaptive-replay — Low-cost component only; full experiments not reproduced.
On-policy distillation	SimOPD: Relational Knowledge Transfer for On-Policy Distillation	Zhejiang University, 2026-10-16	No verifiable author method repository found in this review. Third-party paper lists are not author implementations.	simopd — Low-cost component only; full experiments not reproduced.
Agentic RL	Tree-Aware Posterior Distillation for Long-Horizon Agentic Reinforcement Learning	Tsinghua University, 2026-10-16	No verifiable author method repository found; the full text and abstract provide no verifiable training-code link.	treeopsd — Low-cost component only; full experiments not reproduced.
Alignment from binary feedback	CleanRM: Reward Modeling from Noisy Feedback with a Few Clean Feedback for RLHF	Zhejiang University, 2026-10-15	The paper links to anonymous CAN-RM-46BB, but it was inaccessible during this review. Its contents, license, and executability were not verified, so it is not marked as integrated.	clean-rm — Low-cost component only; full experiments not reproduced.
On-policy distillation	Compatibility Matters: A Framework for Token-Level Teacher Selection in On-Policy Distillation	Princeton University, 2026-10-15	No official repository for this method was found in the paper or through author searches.	token-teacher-selection — Low-cost component only; full experiments not reproduced.
On-policy distillation	Condition-Contrastive Extrapolation for On-Policy Distillation of Language Models	The Chinese University of Hong Kong, Shenzhen, 2026-10-15	An official repository for the method was not found or confirmed as released. Independently implemented from the published equations.	cce-opd — Low-cost component only; full experiments not reproduced.
On-policy distillation	Does On-Policy Distillation Need to Complete Every Rollout?	Cornell University, 2026-10-15	No author method repository was found or confirmed; independently implemented from the public equations.	stopd — Low-cost component only; full experiments not reproduced.
RLHF / RLVR	Each Reward to Its Relevant Tokens: Reward-Specific Token-Level Credit Assignment for Multi-Reward RL	University of Minnesota, 2026-10-15	Official MRTPO, with training for three tasks, verifier replay, and visualization (Apache-2.0).	mrtpo — Low-cost component only; full experiments not reproduced.
Reference anchor	EliteReplay: Preserving Diverse Solutions in RL Post-Training via Quality-Diversity Search	The Ohio State University, 2026-10-15	Public skai-research/EliteReplay	elite-replay — Low-cost component only; full experiments not reproduced.
Agentic RL	KnowGPO: Knowledge-Grouped Policy Optimization for Search Agents	Zhejiang University, 2026-10-15	Official placeholder repository has assets, LICENSE, and README only. The README says method code is forthcoming; the paper's availability claim does not establish training implementation release.	knowgpo — Low-cost component only; full experiments not reproduced.
On-policy self-distillation	Learning from Unverified Peers via Shared Residual Distillation	Shanghai Jiao Tong University, 2026-10-15	No SRD method source found. The linked OPSD repository is a baseline, not SRD code.	shared-residual-distillation — Low-cost component only; full experiments not reproduced.
Self-confidence trajectory filtering	Learning to Reason with Selective Internal Feedback	Institute of Automation, Chinese Academy of Sciences, 2026-10-15	No released author method repository was found or confirmed. The component was independently implemented from the equations.	sif — Low-cost component only; full experiments not reproduced.
RLHF / RLVR	Multi-Reward Hacking: From Aggregation-Induced Drift to Trust-Anchored Defense	University of British Columbia, 2026-10-15	No official TAG repository was found in the paper or author search; release status and license are unconfirmed.	tag-reward-gating — Low-cost component only; full experiments not reproduced.
Agentic RL	OpCredit: Operation-Aware Policy Learning for Agentic Knowledge Graph Question Answering	Nanjing University, 2026-10-15	kaka1215/OpCredit, Apache-2.0; includes BOC/FTC and training adaptations, but configuration templates are unfilled and formal evaluation scripts are absent. See its THIRD_PARTY_NOTICES.md for third-party attribution.	opcredit — Low-cost component only; full experiments not reproduced.
On-policy distillation	OPRD: On-Policy Reminder Distillation for Long-Horizon Coding Agents	Tsinghua University, 2026-10-15	No official OPRD method code was found. VeOmni is the training framework, not a release of this method.	oprd-reminder — Low-cost component only; full experiments not reproduced.
Process rewards	Progress Reward Model for Reinforcement Learning via Large Language Models	Beihang University, 2026-10-15	deng-ai-lab/PRM4RL, the authors' public implementation	progress-reward — Low-cost component only; full experiments not reproduced.
RLHF / RLVR	Training Reasoning Models to Generate Diverse Scientific Ideas	New York University, 2026-10-15	Official placeholder contains a README only, with code, data, and models forthcoming. Not a released training implementation.	diverge-ideation — CPU evaluation interface; does not run a model.
Multi-teacher capability integration	Training-Free On-Policy Distillation	University of Illinois Urbana-Champaign, 2026-10-15	No verified public method repository found. Dependencies and third-party implementations are not author code.	tf-opd — Low-cost component only; full experiments not reproduced.
On-policy distillation	Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillation	Alibaba Cloud Computing, 2026-10-15	WaypointOPD is public. An explicit license was not confirmed, so no source was vendored; the component independently implements the equations.	waypoint-opd — Low-cost component only; full experiments not reproduced.
On-policy distillation	Divergence-Routed On-Policy Distillation for Language Models	Peking University, 2026-10-14	No repository released by the authors for this method was found. Third-party or related-work repositories do not count.	divergence-routed-opd — Low-cost component only; full experiments not reproduced.
RLHF / RLVR	Failure Is the Mother of Success: Recovering Supervision with Grounding Hindsight	University of California, San Diego, 2026-10-14	No official RSHD method repository was found in the paper or author searches.	rshd — Low-cost component only; full experiments not reproduced.
Off-policy reasoning RL	Gibbs-Gap Policy Optimization for Off-Policy RL with Verifiable Rewards	University of Illinois Urbana-Champaign, 2026-10-14	thunderous77/GGPO, public under Apache-2.0. The local component independently implements the equations and does not vendor upstream training code.	ggpo — Low-cost component only; full experiments not reproduced.
On-policy distillation	Information-Gain Calibrated On-Policy Distillation for Deep Search Agents	The Chinese University of Hong Kong, Shenzhen, 2026-10-14	No official IG-OPD method repository was found in the paper or author searches. Framework dependencies do not establish an open-source release of the method.	ig-opd — Low-cost component only; full experiments not reproduced.
Diffusion language model post-training	On-Policy Residual Learning for Diffusion Language Models	Together AI / University of Illinois Urbana-Champaign, 2026-10-14	Public author/upstream code. The authors released ORL code, but system kernels and full experiments still require the corresponding accelerator, model, and training budget.	Planned key orl-diffusion — Not reproduced due to cost (review complete); research material only, no executable entry.
Preference-graph credit assignment	When Pairwise Preferences Cycle: A Reliability-Anchored Preference Walk for Video GRPO	Unconfirmed (reviewed arXiv HTML lists no institution), 2026-10-14	No released author method repository was found.	rap-walk — Low-cost component only; full experiments not reproduced.
on-policy-self-distillation	LoOPSD: On-Policy Self-Distillation with Looped Teachers	Anhui Province Key Laboratory of Digital Security, 2026-10-13	Not found. The pinned v1 did not link official code for this method when checked on 2026-10-14; this is not a claim that no code exists anywhere.	Planned key loopsd — Not reproduced for now due to cost; validation pending, no executable entry.
GRPO difficulty curriculum	A Step Harder, Not a Step Too Far: The Benefits and Limits of Curriculum Difficulty Progression in GRPO	Dalian Maritime University, 2026-10-09	Yes: authors' code.	grpo-difficulty-curriculum
Multi-teacher distillation	Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillation	Nanjing University / JD.com, 2026-10-09	No public implementation found as of 2026-10-12	road-mopd
Capability boundary	BoundaryEvo: Data-Model Co-Evolution at the Capability Boundary for Long-Chain Reasoning Distillation	Shanghai Artificial Intelligence Laboratory, 2026-10-09	Not released at the time of review (2026-10-12). The paper promises a later code and data release, but no usable implementation was found.	boundary-evo
On-policy distillation	Branching On-Policy Distillation	Tsinghua University / SenseTime Research, 2026-10-09	No implementation found: the authors' repository contained only LICENSE and README on 2026-10-12.	branch-opd
Cross-tokenizer distillation	CAST: Credit Alignment for Cross-Tokenizer On-Policy Distillation	Taobao & Tmall Group of Alibaba, 2026-10-09	No public author implementation found as of 2026-10-12.	cast-opd
Synthetic self-distillation	Continually Post-training Small Models via Synthetic Self-distillation	KAIST AI / LG AI Research, 2026-10-09	No release found in paper-linked primary sources as of 2026-10-13.	synthself
Proximal policy optimization	Generalized Proximal Policy Optimization: Unifying LLM RL Algorithms from the Perspective of Divergence	Tsinghua University AIR, 2026-10-09	No release found in the paper-linked primary sources as of 2026-10-13.	gppo-framework
Self-correction	Inject a Turn, Not a Token: Learning Where to Redirect LLM Reasoning for Self-Correction	KAIST, 2026-10-09	No release found in paper-linked primary sources as of 2026-10-13.	upturn
Monte Carlo reasoning tree	Learning More by Training Less: Uncertainty-Guided Progressive Contraction for Efficient Reasoning Post-Training	National University of Singapore / Nanjing University, 2026-10-09	Not released or not found in paper-linked primary sources, checked 2026-10-13	progressive-contraction
Source-aware RL	SOAR: Source-Aware Reinforcement Learning with Precise Credit Assignment for Search Agents	Zhejiang University / Microsoft, 2026-10-09	No release found in paper-linked primary sources as of 2026-10-13.	soar
On-policy distillation	State-OPD: State-Grounded On-Policy Distillation for Multi-Turn Agents	Alibaba Group, 2026-10-09	No release found in paper-linked primary sources as of 2026-10-13.	state-opd
Trajectory-aware policy optimization	Trajectory-Aware Policy Optimization: Closing the State Distribution Gap in Long-Horizon LLM RL	Tsinghua University AIR, 2026-10-09	No release found in paper-linked primary sources as of 2026-10-13.	tapo-trajectory
On-policy distillation	DIAL-OPD: Learning More from Fewer Tokens in On-Policy Distillation	Eastern Institute of Technology, Ningbo / The Hong Kong Polytechnic University, 2026-10-08	Yes: EIT-NLP/DIAL-OPD.	dial-opd
RLVR	GRPODropout: Less is More for Online Reinforcement Learning Rollouts	Harbin Institute of Technology (Shenzhen), 2026-10-08	Public implementation	grpo-dropout
RLVR data synthesis	MagicBeanStalk: Escaping the RLVR Zero Advantage Trap with Recursive Data Synthesis	University of Washington, 2026-10-08	No released implementation found in the paper or linked primary sources as of 2026-10-13; this is not proof of universal absence.	Planned key magicbeanstalk — Not reproduced for now due to cost; validation pending, no executable entry.
On-policy distillation	MetaOPD: Meta-Learned Token Weighting for On-Policy Distillation	East China Normal University, 2026-10-08	No author code found or released as of 2026-10-10.	meta-opd
On-policy distillation	PIVOT: Perplexity-Informed KD-to-RL Transition Scheduling for Vertical-Domain Few-Shot Distillation	Ping An Technology / University of Science and Technology of China, 2026-10-08	No public author repository found.	pivot
On-policy distillation	ReCal: Calibrating Structured Pruning for On-Policy Distillation Recovery	Zhongguancun Academy / Tencent / University of Science and Technology of China, 2026-10-08	Released	recal
RLVR	Residual Advantage: Student-Relative Teacher Guidance for RL with Verifiable Rewards	Harbin Engineering University / Tencent / Harbin Institute of Technology, 2026-10-08	No author-released code was found as of 2026-10-10.	residual-advantage
On-policy distillation	When Do We Need On-Policy Distillation? Distilling on Offline Student Rollouts Is Often Better	UCLA / NVIDIA (internship noted in the paper), 2026-10-08	No author-released code was found as of 2026-10-10.	semi-opd
Group relative policy optimization	When KL Regularization Misfires in Group Policy Optimization	Alibaba Group, 2026-10-08	No public author implementation found.	zcpo
On-policy self-distillation	Which Skill to Distill? SGUID: Selecting a Compact Skill Bank for Model–Skill Co-Evolution	New York University, 2026-10-08	Not yet released. The paper plans to release code, skill-generation prompts, and the full candidate bank after acceptance.	sguid
On-policy distillation	Why On-Policy Distillation Sometimes Fails: Vanishing Learning Signals	University of Pennsylvania, 2026-10-08	Author code.	opd-learning-signals
Agentic RL	Follow the Winners: Conservative Policy Improvement with the Cross-Entropy Method for Critic-Free RFT	Trent AI Limited, 2026-10-02	Not released. The reproducibility statement says code, models, and data will not be released before acceptance and internal legal review.	follow-the-winners
Post-training data selection	LESSER: Post-Training Data Selection with Output-Layer Gradients	University of Pennsylvania, 2026-10-02	No public code found in official HTML as of 2026-10-05.	lesser
Asynchronous RL	Asynchronous LLM Post-Training: Group-Mass Capping and Convergence Analysis	The Ohio State University, 2026-10-01	No public author implementation was found as of 2026-10-03.	gmc-grpo
RLVR	CARM: Cancellation-Aware Response Masking for LLM Reinforcement Learning	Moore Threads AI, 2026-10-01	No public author implementation found as of 2026-10-03.	carm
Agentic RL	Dependency-Aware Reward Shaping for Agentic Reinforcement Learning	University of Illinois Urbana-Champaign, 2026-10-01	The official placeholder said the implementation was in preparation as of 2026-10-02; interface notes are not released implementation.	dars
Preference optimization	GAW-PO: Preference Optimization with Gradient-Aligned Token Weights	National University of Science and Technology POLITEHNICA Bucharest, 2026-10-01	No public author implementation found as of 2026-10-03.	gaw-po
Agentic RL	My FAULT: Self-Diagnosis as Credit Assignment in Self-Evolving Agentic Reinforcement Learning	Alibaba Group, 2026-10-01	No public author implementation found as of 2026-10-02.	my-fault
GRPO	Range-GRPO: Policy Optimization via Pairwise Relations among Reward Intervals	Yonsei University, 2026-10-01	No public author implementation found as of 2026-10-02.	range-grpo
Posterior concentration and length constraints	RLCPR: Rethinking Probability-Based Reinforcement Learning From Posterior Concentration	National University of Singapore, 2026-10-01	None found in full-text review on 2026-10-04.	rlcpr
RL post-training	Sharpening Tax in Post-Training	Meta Superintelligence Labs / University of Wisconsin–Madison, 2026-10-01	changdaeoh/sharpening-tax is public.	sharpening-tax
Video generation	Token-Level Video Reinforcement Learning	First author's affiliation not listed on the first page, 2026-10-01	No public author implementation was found as of 2026-10-03.	tvrl
Multimodal post-training	Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenes	Valeo.ai, 2026-10-01	Where-OPD	where-opd
Agentic RL	SHARPO: Segment-Level Credit Assignment for Agentic Reinforcement Learning	Georgia Institute of Technology, 2026-09-30	No public author implementation found as of 2026-10-03.	sharpo
Trajectory-to-step credit assignment	T2SPO: Trajectory-to-Step Policy Optimization for Agentic Reinforcement Learning	Nanjing University, 2026-09-30	No author implementation found in the full-text review on 2026-10-04. TabPFN is a separate dependency, not official T2SPO code.	t2spo
Advisor	AdviSD: Learning to Advise Frontier LLMs via Targeted Multi-Turn Self-Distillation	University of Southern California, 2026-09-29	No author repository found as of 2026-10-01.	advisd
On-policy distillation	Dr. OPD: Learning What to Follow for Optimal On-Policy Distillation of Large Language Models	Rutgers University, 2026-09-29	zywang0701/Dr-OPD	dr-opd
OPD	DriftOPD: Sequence-Level Reverse-KL Distillation for One-Step VLA Policies	KAIST, 2026-09-29	No public author implementation found as of 2026-10-03.	drift-opd
On-policy distillation	FlowMap-OPD: Rollout-Kernel Separation for On-Policy Distillation of Few-Step Flow-Map Generators	Georgia Institute of Technology, 2026-09-29	Public implementation	flowmap-opd
On-policy distillation	From Dissonance to Orchestration: Teacher Intervention in On-Policy Distillation	Nanyang Technological University, 2026-09-29	MAESTRO	maestro-opd
Agentic RL	Guide, Then Let Go: Gap-Adaptive Teacher Scheduling for Sparse-Reward Agentic RL	Dalian University of Technology / Kuaishou, 2026-09-29	Yes: Ricardo-H/guide-then-let-go	gats
Policy distillation	Interpolated Policy Distillation: A Controllable Continuum Between Off-Policy and On-Policy Distillation	WeChat Vision, Tencent, 2026-09-29	No public author repository found as of 2026-10-01.	ipd
RLVR	Learning Beyond What You Sample: Off-Policy-Aware Cross-Model Trajectory Exchange for RLVR	KAIST, 2026-09-29	No author repository found as of 2026-09-30.	graft
OPD	LEGO-OPD: Factorized Teacher Composition for Multimodal On-Policy Distillation	KAIST, 2026-09-29	No public author implementation was found as of 2026-10-03.	lego-opd
On-policy self-distillation	Overcoming Scaling Limits in On-Policy Self-Distillation for LLM Reasoning	North South University, 2026-09-29	No author repository was found as of 2026-09-30.	oasis-opsd
Agentic RL	PR-OPD: Privileged Representation On-policy Self-Distillation for Agentic Reinforcement Learning	University of Florida, 2026-09-29	Yes: balibata/PR-OPD	pr-opd
RLVR	SIPO: Unifying Reinforcement Learning with On-Policy Self-Distillation	University of Illinois Urbana-Champaign, 2026-09-29	Yueeeeeeee/SIPO.	sipo
On-policy distillation	The Teacher Is a Direction, Not a Destination: Extrapolating RL-Induced Representation Residuals in On-Policy Distillation	Affiliation not listed in the paper, 2026-09-29	xixixixixxxx/RIDE.	ride-opd
Teacher prefix-constrained distillation	The Weakest Link: Distilling LLM Reasoning with Worst-Case Constrained Reinforcement Learning	Huawei Noah’s Ark Lab, 2026-09-29	No author code found in the 2026-10-04 full-text review.	weakest-link
RL	Unlocking the Critic: Reward-Free Policy Optimization for LLM Post-Training	University of Luxembourg, 2026-09-29	No public author repository found as of 2026-09-30.	rfpo
On-policy distillation	An RL View of OPD: Least Square Policy Distillation for Sample-Efficient LLM Reasoning	University of North Carolina at Chapel Hill, 2026-09-28	UNCSciML/LSPD.	lspd
Online distillation	Learning from Teacher Continuations at Student States	University of Illinois Urbana-Champaign, 2026-09-28	The source lists the author project as the release link.	olive
Multi-agent systems	MAS-OPD: On-Policy Distillation for Multi-Agent Systems	University of Science and Technology of China, 2026-09-28	No public author repository found as of 2026-09-30.	mas-opd
Offline SFT	ROSS: Relearning from Self-Generated Rollouts through Selective Supervision	AllSpark Team (team byline; no institution listed), 2026-09-28	No public author repository was found as of 2026-09-30.	ross
Self-review	Shockingly Simple Self-retrospection Improves Agentic Models Without RL	Rensselaer Polytechnic Institute, 2026-09-28	No author-released implementation was found as of 2026-09-30.	roft
On-policy self-distillation	Recursive Self-Improvement via On-Policy Distillation for Reasoning	Meta AI / University of California, Riverside, 2026-09-25	No author training repository found as of 2026-09-29.	recursive-opsd
Reinforcement learning	PACT: From Credit Assignment to Critic Alignment	AllSpark Team, 2026-09-22	The project repository was empty as of 2026-09-27; no training source was released.	pact
On-policy distillation	RetireOPD: Self-Retiring On-Policy Distillation for Agentic Reinforcement Learning	Zhejiang University, 2026-09-17	ZJU-REAL/SDAR is public.	retire-opd
Preference optimization	A Zeroth-Order Paradigm for LLM Preference Alignment	University of California, Berkeley, 2026-09-16	No public author implementation found as of 2026-09-19.	compo
On-policy distillation	Trajectory Learnability for Offline On-Policy Distillation	National University of Singapore, 2026-09-16	No public author repository found as of 2026-09-19.	trajectory-learnability
On-policy distillation	Beyond Token-Local Imitation: Reward-Compatible Temporal Credit Assignment for On-Policy Distillation	Tsinghua University / DiDi Voyager Labs, 2026-09-15	No public author repository found as of 2026-09-16.	gamma-opd
RL training acceleration	GrowMTP: Can RL Grow Its Own Draft Head?	WeChat AI, Tencent / Peking University, 2026-09-15	No public author repository found as of 2026-09-16.	growmtp
Multimodal post-training	OPD-Aha: From Linguistic Momentum to Visual Reflection in Multimodal On-Policy Distillation	Arizona State University / University of Virginia / Stevens Institute of Technology, 2026-09-15	Yes: Echochef/OPD-Aha	opd-aha
Preference optimization	Style-Debiased DPO: Updating LLM Knowledge with Factuality-Aware Synthetic Preference Data	Waseda University, 2026-09-15	No public author repository was found as of 2026-09-16.	sd-dpo
Reinforcement learning	TIAO: Token Importance-Aware Policy Optimization for Text Summarization	National University of Defense Technology, 2026-09-15	Public implementation	tiao
Multi-turn agent alignment	Turn-level Multiscale Density Ratio Estimation for LLM Agents	Alibaba Group, 2026-09-15	No public author repository found as of 2026-09-16.	tlm-dre
On-policy distillation	Know When to Stop, Where to Restart: Accelerating Multi-Turn Agentic On-Policy Distillation	University of Science and Technology of China / Ant Group, 2026-09-13	No public author repository was found as of 2026-09-16.	stride-opd
On-policy distillation	Data-free On-policy Distillation	Institute of Automation, Chinese Academy of Sciences / Tencent, 2026-09-12	No public author repository found as of 2026-09-16.	df-opd
Online reinforcement learning	MaD-RL: Matching Distributions for Calibrating LLMs with Reinforcement Learning	Meta Superintelligence Labs, 2026-09-12	No release found in checked paper and official sources; unrelated robotics MAD is not this implementation.	Planned key mad-rl — Not reproduced due to cost (review complete); research material only, no executable entry.
On-policy distillation	A Unified Per-Token Gating Family for On-Policy Distillation: FKL/RKL Mixing with Multi-Channel and Bias Coefficients	Xiaohongshu, 2026-09-10	No public author repository was found on 2026-09-14.	adaptive-opd-gate
Test-time adaptation	Beyond Confidence: Stability-Aware Test-Time Adaptation for LLM Reasoning	Chongqing University, 2026-09-10	No public author repository found as of 2026-09-14.	tasco
Low-rank post-training	LOCUS: Task-Aware Low-Rank Post-Training for Token-Efficient Language Generation	University of Washington, 2026-09-10	No public author repository was found on 2026-09-14.	locus
Negative self-distillation	Negative Self-Distillation: Learning to Reason by Avoiding Flaws	University of Virginia, 2026-09-10	Authors' code.	nsd
Cross-model-family OPD	CompassOPD: Cross-Family On-Policy Distillation via Within-Family Likelihood Shifts	Institute of Information Engineering, Chinese Academy of Sciences, 2026-09-09	No public author repository found as of 2026-09-12.	compass-opd
Probability transport for OPD	Distillation as Probability Transport: Routed On-Policy Distillation	Tencent, 2026-09-08	No public author repository was found on 2026-09-12.	route-opd
Weak-to-strong reverse distillation	Eliciting Weak-to-Strong Generalization with On-Policy Reverse Distillation	KAIST AI, 2026-09-08	No public repository found as of 2026-09-12.	oprd
Test-time RL for code	Entropy-Regularized Rank-Masked Policy Optimization for Test-Time Reinforcement Learning in Code Generation	Nanyang Technological University, 2026-09-08	No public author repository found as of 2026-09-12.	probe-erpo
Extremely sparse on-policy distillation	Extremely Sparse Supervision Incentivizes Reasoning Ability	Amazon / Duke University, 2026-09-03	No public code repository found as of 2026-09-07.	sparse-opd
Weak-model prefix exploration	Boosting LLM Exploration via Weak-Model Guidance in RLVR	Peking University, 2026-08-27	No public author implementation found as of 2026-08-29.	weak-guide-rlvr
Cross-domain RLVR capability fusion	Consolidating RLVR Capabilities Across Domains: A Deep Dive into Fusion Paradigms	Fudan University, 2026-08-27	Released: LLM-Fusion	rlvr-fusion
Decoupling preference and optimization scales in DPO	Disentangling Optimization Scale from Preference Scale in DPO	Affiliation not listed in the paper, 2026-08-27	ivankru/bayesian_dpo.	normalized-dpo
Multi-teacher capability integration	Preserving General Capabilities during Domain Specialization with Uncertainty-Calibrated MOPD	Peking University, 2026-08-27	No public author code was found when checked on 2026-08-29.	uc-mopd
Process rewards	SPEAR: Distilling Domain-Adaptive Reasoning Skeletons via Sequential Symbolic Alignment in Reinforcement Learning	University of Pittsburgh, 2026-08-27	zhuochunli/SPEAR.	spear
Test-time reinforcement learning	TTPO: Test-Time Policy Optimization	Zhejiang University, 2026-08-27	ZJU-REAL/TTPO	ttpo
Privileged self-distillation with evidence frames	Video-OPSD: Exploiting Privileged Visual Evidence for On-Policy Self-Distillation in Video Large Language Models	Nanyang Technological University, 2026-08-27	No public author code found on 2026-08-31.	video-opsd
Mixed-policy knowledge injection in RL	From Memorization to Absorption: Mixed-Policy RL for Continual Knowledge Injection	University of California, Merced, 2026-08-26	No public author code found as of 2026-08-28.	grin
Reward-guided parameter interpolation	GRIP: Granular Reward-Guided Parameter Interpolation for Efficient Reasoning	Peking University, 2026-08-26	The paper uses a public training framework, but no method-specific author repository was found as of 2026-08-28.	grip
Multimodal rubric RL	V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning	S-Lab, Nanyang Technological University, 2026-08-26	Author project.	v-rubrics
Long-video privileged-view OPD	Where to Look Matters: On-Policy Self-Distillation for Long-Video Understanding	University of Maryland, College Park, 2026-08-26	No public author code was found on 2026-08-28.	clue-opsd
Search-augmented OPD and RL	OPDSearch+: Search-Enhanced On-Policy Distillation with Reinforcement Learning	University of Chinese Academy of Sciences, 2026-08-25	No public code found on 2026-08-26.	opd-search-plus
OPD with verifiable rewards	OPDVR: On-Policy Distillation with Verifiable Rewards	LeapLab, Tsinghua University, 2026-08-25	LeapLabTHU/OPDVR.	opdvr
Input-side Query-KL	Beyond the Stability-Exploration Dilemma: Environmental Regularization for LLM Policy Optimization	AMAP, Alibaba Group, 2026-08-24	Yes: alibaba/ERPO.	erpo
Reflective token credit	SRPO: Self-Reflective Policy Optimization for Long-Horizon Reasoning	Wuhan University, 2026-08-24	Released: Galleons2029/SRPO	srpo
OPD	Beyond Imitation: Filtering On-Policy Distillation by Reasoning Progress	Authors did not disclose affiliation, 2026-08-19	No public author code found as of 2026-08-24.	r2-opd
Multi-reward RL	Learn What's Left, Not What's Mastered: Saturation Aware Advantage Reweighting for Multi-Reward Policy Optimization	University of Florida, 2026-08-17	No public author code was found as of 2026-08-24.	sa-mrpo
Multimodal contextual preference calibration	Context Blindness in DPO: Mitigating Object Hallucination in MLLMs via Context-Calibrated Preference Optimization	Korea University, 2026-08-12	Released: mlvlab/C2-DPO	c2-dpo
Geometrically constrained RL	GCPO: Diagnosing and Constraining Subspace Geometry in Rollout RL for LLMs	Shanghai AI Laboratory, 2026-08-12	Released: Icarus1411/GCPO.	gcpo
Lookahead preference trees	Preference Tree Optimization: Enhancing Goal-Oriented Dialogue with Look-Ahead Simulations	Reichman University, 2026-08-12	No public author code found on 2026-08-13.	pto
Rubric RL	Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL	Scale AI, 2026-08-12	No public author code was found as of 2026-08-24.	rubric-dropout
OPD	SR-OPSD: Self-Referenced On-Policy Self-Distillation	Independent Researcher, 2026-08-10	No public author code found as of 2026-08-24.	sr-opsd
Adaptive self-distillation	DASH	Nanjing University, 2026-08-06	DBtxy/DASH-OPSD is public.	dash
OPD	On-Policy Delta Distillation for Multilingual Math Reasoning	NAVER AI Lab, 2026-08-06	naver-ai/opd2.	opd2
Reasoning-pivot distillation	RP-OPSD	Nanjing University, 2026-08-06	Released: NJUNLP/RP-OPSD.	rp-opsd
Generative reward models	RRC: Unlocking Generative Reward Models in LLM Reinforcement Learning via Ranking-Based Reward Construction	Northeastern University, 2026-08-06	wangclnlp/RRC.	rrc
Unsupervised self-distillation	U-OPSD	University of California, San Diego, 2026-08-06	No released official code found on 2026-08-08.	u-opsd
Rollout budget allocation	Optimizing What Policies Learn From: Recoverability-Aware Rollout Intervention Learning	University of Notre Dame / Amazon, 2026-08-05	No public author code was found on 2026-08-09.	rail
RL rollout acceleration	SpecRoll: Fast-Slow Verifier-Feedback Adaptation for Speculative Reinforcement Learning Rollouts	VNU University of Engineering and Technology / Viettel AI, 2026-08-05	Released through the author artifact.	specroll
Return-dependent reward shaping	ADRS	University of Science and Technology of China, 2026-08-04	Released code: gitrxh/ADRS-arxiv.	adrs
OPD	CausalOPD: First-Wrong-Step Supervision for Distilling Causal Chain Reasoning	Authors did not disclose affiliation, 2026-08-04	No public author code was found as of 2026-08-24.	causal-opd
OPD	SMOPD: Multi-Reward Reinforcement Learning via Specialize-and-Merge Online Policy Distillation	Alibaba / Qwen, 2026-08-04	No public author code found as of 2026-08-24.	smopd
External rollouts	Beyond On-Policy Exploration: Integrating External Policy Rollouts for Reinforcement Learning in Diffusion Language Models	Seoul National University, 2026-08-03	No public author code found as of 2026-08-24.	erils
Continual consistency distillation	PCSD	Institution not stated in the paper, 2026-08-03	No official implementation was found on 2026-08-08.	pcsd
OPD	Distill Where You Fail: Recovering Learning Signals of Negative RL-Groups from Adaptive Teacher Guidance	Tianjin University, 2026-08-01	No public author code was found as of 2026-08-24.	rstg
OPD	Contrastive Reinforced Policy Optimization via Privileged Self-Distillation	Authors did not disclose affiliation, 2026-07-30	No public author code found as of 2026-08-24.	crpo
Context distillation	Flux-OPD	Peking University, 2026-07-30	No official author implementation was found when checked on 2026-08-01.	flux-opd
Multimodal evidence attribution distillation	VAD	Shanghai Jiao Tong University, 2026-07-30	Yes: author implementation and model weights.	vad
On-policy self-distillation	β-OPSD	University of Maryland, College Park, 2026-07-30	No official author implementation found as of 2026-08-01.	beta-opsd
Distribution-preserving RL	ReCo	Seoul National University, 2026-07-29	No public author implementation found.	reco-grpo
Rubric RL	SERPO: Self-Evolving Rubric Policy Optimization for Open-Ended Test-Time Reinforcement Learning	Authors did not disclose affiliation, 2026-07-29	No public author code found as of 2026-08-24.	serpo
Token-level credit assignment	CoRT	ByteDance internship, 2026-07-28	No official public repository was found as of 2026-07-29.	cort
Online reinforcement learning	Reinforcement Learning for Code Optimization	FAIR at Meta / Inria, 2026-07-28	No verified complete training source for this paper was found. Its cited BigOBench sandbox belongs to prior work, not a release of this method.	Planned key code-optimization-rl — Not reproduced due to cost (review complete); research material only, no executable entry.
On-policy distillation	Relay-OPD	Zhejiang University, 2026-07-28	ZJU-REAL/Relay-OPD.	relay-opd
Reward modeling and evaluation protocols	What do Reward Models Memorize?	ILLC, University of Amsterdam, 2026-07-27	The text links ioverho/rm-shortcuts; pinned SHA e737a3b26393b59ab354f7bbac8a4744ea441904 checked. No root LICENSE; GitHub license=null, author code license unconfirmed. Independently reimplemented. Paper and figure: CC BY 4.0.	rm-memorization-protocol — Low-cost component only; full experiments not reproduced.
post-training	Co-Evolving LLM Evaluators and Policies via DynamicRubric	WeChat / Tencent, 2026-07-22	No official or author-released code found in the paper (checked 2026-08-09).	dynamic-rubric
Process rewards	TCR	Institution not stated in the paper, 2026-07-22	No public repository found as of 2026-07-27.	tcr
RLVR	ISO: An RLVR-Native Optimization Stack	The University of Texas at Austin, 2026-07-21	zhuhanqing/ISO.	iso-rlvr
rlvr	Off-Context GRPO: Learning to Reason on Hard Problems using Privileged Information	Meta AI, 2026-07-21	Official author code	off-context-grpo
Teacher reward reweighting	Distilled RL	Nankai University, 2026-07-19	Released code: 597358816/Distilled-RL.	distilled-rl
Reference anchor	ARMOR	University of Science and Technology of China, 2026-07-11	No official code was found.	armor
Geometric trust regions	RIPO	Tsinghua University, 2026-07-11	No official author code found.	ripo
Token credit calibration	TACO	Johns Hopkins University, 2026-07-08	Author code.	taco
post-training	Turning Off-Policy Tokens On-Policy: A Plug-in Approach for Improving LLM Alignment	Renmin University of China, 2026-07-06	No official author code identified (checked 2026-08-09).	sis
Multi-teacher capability integration	MOPD	Xiaomi, 2026-06-29	No official author implementation found or released as of 2026-08-08.	mopd
Capability-frontier curricula	CoBA-RL	Zhejiang University, 2026-06-21	No separately released official implementation was found.	coba-rl
Entropy stabilization	STARE	Tsinghua University, 2026-06-17	Author code.	stare
Asynchronous training–inference mismatch	KPop	Ling / Ring Team, 2026-06-13	No standalone algorithm repository found.	kpop
Autoregressive-to-diffusion distillation	OPDLM	Texas A&M University, 2026-06-04	No released official code found on 2026-08-08.	opd-lm
RLVR and preference optimization	POLARIS: Guiding Small Models to Write Long Stories	University of Maryland, 2026-06-02	Author resources at 50177c1854d83ef9d93339ba7d87e95583e73340. Training, reward, and evaluation code are public; copyrighted HRI human stories and reasoning traces are not released and the main checkpoint is gated. The public no-HRI weights do not supply main-method data. Root MIT; vendored verl and related changes are Apache-2.0.	polaris-hri — Low-cost component only; full experiments not reproduced.
Multi-objective RL	GPRL	Stanford University, 2026-05-18	No public repository found on the paper page as of 2026-07-27.	gprl
Contrastive sequence RL	ConSPO	Beijing Institute of Technology, 2026-05-13	No official author code was found.	conspo
Training workflows and resource allocation	Beyond GRPO and On-Policy Distillation: An Empirical Sparse-to-Dense Reward Principle for Language-Model Post-Training	LinkedIn, 2026-05-12	The v5 text links HJSang/OPSD_OnPolicyDistillation and the pinned SHA was verified. The root contains only README/scripts/src, with no root LICENSE and GitHub license=null. The OPD subset is not the complete four-stage pipeline; no source is copied locally. Author repository; pinned SHA 39b3b0a5dccbc02b8f1908468bb685d2ea386d9a	Planned key sparse-to-dense-reward-principle — Not reproduced due to cost (review complete); research material only, no executable entry.
Asynchronous off-policy training	Missing Old Logits	Tianjin University, 2026-05-12	No official author code found or released.	missing-old-logits
Direct preference optimization	Mind the Gap: Structure-Aware Consistency in Preference Learning	Google Research / Courant Institute of Mathematical Sciences, 2026-04-30	No author repository was found in the paper or official Google page. This is not proof of non-release.	Planned key sa-dpo — Not reproduced due to cost (review complete); research material only, no executable entry.
Preference optimization	GroupDPO: Memory-Efficient Group-Wise Direct Preference Optimization	Carnegie Mellon University, 2026-04-17	No verified training code in the pinned paper, Google's paper page or the first author's publications page. No verified SHA/license; no unattributed code reused.	group-dpo — Low-cost component only; full experiments not reproduced.
On-policy distillation	Lightning OPD	MIT HAN Lab, 2026-04-14	jet-ai-projects/Lightning-OPD	lightning-opd
Multilingual post-training	Large Reasoning Models Struggle to Transfer Parametric Knowledge Across Scripts	Google Research, 2026-03-17	No author code link in the pinned v2 or abstract. Llama-Factory is upstream training tooling, not this paper's author implementation.	Planned key cross-script-knowledge-transfer — Not reproduced due to cost (review complete); research material only, no executable entry.
Context distillation	Prompt-Level Distillation: A Non-Parametric Alternative to Model Fine-Tuning for Efficient Reasoning	Google, 2026-02-24	No author implementation repository was found in the source reviewed. Not finding it does not prove it was never released.	Planned key prompt-level-distillation — Not reproduced due to cost (review complete); research material only, no executable entry.
Context distillation	OPCD	Microsoft Research, 2026-02-12	microsoft/LMOps/opcd.	opcd
Length-unbiased RL	LUSPO	Meituan, 2026-02-05	No separate official repository found or released.	luspo
On-policy self-distillation	OPSD	University of California, Los Angeles, 2026-01-26	Yes: siyan-zhao/OPSD.	opsd
Calibration-aware RL	Balancing Classification and Calibration Performance in Decision-Making LLMs via Calibration Aware Reinforcement Learning	University of Southern California, 2026-01-19	No public author implementation was found as of 2026-09-20.	calibration-aware-rl
Online-only training–inference correction	Online IcePop	Ant Group, 2025-12-16	No standalone source released; a training-schedule variant of IcePop.	online-icepop
Stable MoE reinforcement learning	Stabilizing RL with LLMs	Alibaba Qwen Team, 2025-12-01	No official author repository found or released.	minirl
MoE training–inference mismatch	IcePop	Ant Group, 2025-10-21	No standalone IcePop algorithm source found or released.	icepop
Reward modeling and calibration	Judging with Confidence: Calibrating Autoraters to Preference Distributions	Google, 2025-09-30	No method-specific code link in the pinned abstract or full text, and exact-title search did not verify a repository. No verifiable author parser was released; the local implementation documents its independent convention.	probabilistic-autorater — Low-cost component only; full experiments not reproduced.
LLM evaluation and calibration	When LLMs Benchmark Themselves: Deconstructing Self-Bias in Automated Evaluation	Google, 2025-09-30	No author source link verified in the pinned v3, abstract, or exact-title search. Third-party evaluation frameworks are not author implementations.	benchmark-self-bias — Low-cost component only; full experiments not reproduced.
Dynamic SFT–RL mixture	CHORD	Alibaba Group, 2025-08-15	Trinity-RFT CHORD example.	chord
Gradient-preserving clipping	GPPO	Alibaba Group, 2025-08-11	No standalone public implementation of the algorithm was found.	gppo
Training–inference mismatch correction	TIS	University of California, San Diego, 2025-08-05	No standalone algorithm repository was released; the article lists subsequent OAT, SkyRL, and OpenRLHF implementations.	tis
Stable sequence-level reinforcement learning	GSPO	Alibaba Qwen Team, 2025-07-24	No standalone repository was released with the paper. Alibaba ROLL later provided a GSPO implementation.	gspo
Calibration rewards	Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty	Massachusetts Institute of Technology, 2025-07-22	damanimehul/RLCR.	rlcr
Self-play curricula	SPIRAL	Apple, 2025-06-30	Author code.	spiral
Long-context RL	CISPO / MiniMax-M1	MiniMax, 2025-06-16	Public implementation	cispo
Self-confidence rewards	INTUITOR	University of California, Berkeley, 2025-05-26	Released: sunblaze-ucb/Intuitor.	intuitor
Reward modeling and credit assignment	SCAR: Shapley Credit Assignment for More Efficient RLHF	McGill University, 2025-05-26	Pinned v1 links upstream SHAP, TRL, and baseline tools only; no author repository was verified in the paper or by exact-title search. This is not proof none was released.	scar-shapley-credit — Low-cost component only; full experiments not reproduced.
Zero-data self-play	Absolute Zero	Tsinghua University, 2025-05-06	LeapLabTHU/Absolute-Zero-Reasoner.	absolute-zero
Test-time reinforcement learning	TTRL	PRIME-RL author team, 2025-04-22	Public implementation	ttrl
Off-policy reasoning RL	LUFFY	University of Washington, 2025-04-21	Public implementation	luffy
Critic PPO	VAPO	ByteDance Seed, 2025-04-07	No official code found.	vapo
GRPO aggregation bias	Dr. GRPO	SAIL research team, 2025-03-26	sail-sg/understand-r1-zero.	dr-grpo
Long-reasoning RL	DAPO	ByteDance Seed, 2025-03-18	Released: BytedTsinghua-SIA/DAPO	dapo
Global advantage estimation	REINFORCE++	Independent researchers, 2025-01-04	No official code was found.	reinforce-plus
Reference-free preferences	SimPO	Princeton University, 2024-05-23	princeton-nlp/SimPO.	simpo
Single-stage preference learning	ORPO	KAIST, 2024-03-12	Released: xfactlab/orpo	orpo
Classic RLHF	RLOO	Cohere For AI, 2024-02-22	No standalone official training repository. The authors released data links; TRL and OpenRLHF later provided third-party implementations.	rloo
Online reasoning RL	DeepSeekMath / GRPO	DeepSeek-AI, 2024-02-05	Released	grpo
Alignment from binary feedback	KTO	Contextual AI, 2024-02-02	Released: ContextualAI/HALOs	kto
Self-rewarding	Self-Rewarding LM	Meta AI, 2024-01-18	No standalone author training repository found or released.	self-rewarding
Self-play fine-tuning	SPIN	University of California, Los Angeles, 2024-01-02	uclaml/SPIN	spin
Automatic process rewards	Math-Shepherd	Peking University, 2023-12-14	No standalone author repository found or released.	math-shepherd
Preference regularization	IPO	Google DeepMind, 2023-10-18	No standalone official repository was released or found.	ipo
Classic RLHF	ReMax	The Chinese University of Hong Kong, Shenzhen / Shenzhen Research Institute of Big Data, 2023-10-16	liziniu/ReMax.	remax
Multi-attribute controllable SFT	SteerLM	NVIDIA, 2023-10-09	NVIDIA NeMo-Aligner	steerlm
AI feedback	RLAIF	Google Research, 2023-09-01	No standalone author training repository found or released.	rlaif
Classic on-policy distillation	GKD	Google DeepMind, 2023-06-23	The authors did not release a standalone repository. Hugging Face TRL subsequently implemented it.	gkd
Reverse-KL distillation	MiniLLM	Tsinghua University, 2023-06-14	microsoft/LMOps/minillm	minillm
Process supervision	Let's Verify Step by Step	OpenAI, 2023-05-31	The authors released PRM800K data.	process-supervision
Direct preference optimization	DPO	Stanford University, 2023-05-29	Public author code.	dpo
Sequence probability calibration	SLiC-HF	Google DeepMind, 2023-05-17	No standalone author implementation found.	slic-hf
Reward-ranked fine-tuning	RAFT	HKUST, 2023-04-13	Integrated in the authors' LMFlow	raft
Full-ranking preferences	RRHF	Alibaba DAMO Academy, 2023-04-11	Public author code.	rrhf
Safety alignment from AI feedback	Constitutional AI	Anthropic, 2022-12-15	Supplementary materials released; not a complete training framework.	constitutional-ai
Classic RLHF	InstructGPT / PPO-RLHF	OpenAI, 2022-03-04	Partially released: evaluation examples and model cards. Full SFT/RM/PPO training code was not released.	ppo-rlhf

Browse by:

• Browse by institution, company, or university
• Browse by topic
• Browse by year