From one cheaper forward pass to an auditable long-term research loop¶
简体中文 · English
Updated 2026-10-27 · 19 topic entries / 5 representative comparisons
ROCS removes duplicate computation within a request. Rec-Distill moves expensive teacher knowledge into training. Effective Training Time tracks training lifecycle losses. FORTE controls feature retirement risk. Self-Evolving RecSys links proposals, experiments, and online feedback over time. They optimize different timescales; lower GPU cost, feature removal, and automated proposals are not evidence of recommendation gains.
Mechanisms · Trade-offs · Cross-paper questions · Updates
How to read this card
The table compares representative methods. It is not a shared-benchmark leaderboard. Teaching examples are not measured results.
Mechanism comparison¶
Axes: Training, within-request inference, and long-term research costs · Dependencies and consumers that must stay consistent under reuse or deletion · Connecting offline gates, retraining, and real online feedback · How failed gates and negative results inform later decisions
| Paper and source | Mechanism | Benefits and trade-offs |
|---|---|---|
| ROCS: Request-Oriented Compute Sharing for Efficient Large-Scale Recommendation · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 复现边界 |
Encodes a user request once under candidate isolation, then scores candidates via deferred DCA interaction. Production IKBO broadcasts the request representation within the kernel. | Sharing requires a candidate-independent user representation. Local work does not reproduce GLM/IKBO CUDA kernels or production clusters. Analytic reuse is not measured throughput. |
| Rec-Distill: An Industrial Distillation Pipeline for Large-Scale Recommendation Models · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 本地复现 |
Materializes raw logits from the teacher's own forward pass, with sampling correction on the student side. The main tower trains only on true labels and serves; the auxiliary tower takes task and distillation losses and shares the backbone. | Extra teacher and auxiliary-tower training lowers deployment burden, but a stronger teacher does not guarantee student gains. Align teacher/student probability spaces and compare against a same-budget student. |
| Optimizing Effective Training Time for Large-Scale Recommendation Systems · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 复现边界 |
Divides training time that consumes new data by end-to-end wall time, attributing losses to initialization, compilation, checkpointing, publishing, and recovery. | Higher ETT is not higher model accuracy or GPU busy time. The local component decomposes lifecycle phases and ownership, not recommender capability. |
| FORTE: A Production Framework for Feature Evaluation and Deprecation in Industrial Recommender Systems · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 复现边界 |
Perturbs dependencies in the frozen model, screens all heads and time windows, then retires features through joint ablation, real retraining, online experiments, and consumer lineage audits. | Weak single-feature dependence does not imply safe joint removal. No candidate passed every gate in the local three-seed A100 run, so the baseline remained. Completing the process does not establish gains from deletion. |
| Self-Evolving Recommendation System: End-To-End Autonomous Model Optimization With LLM Agents · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 本地复现 |
An inner research agent proposes optimizers, architecture, and rewards and executes tools. An outer state machine enforces constraints, offline gates, and real A/B promotion; conclusions return to a journal. | Executable experiments and independent evaluation are necessary. A proposal or registered name is not a capability. Small local tests do not replace production feedback, and active exploration does not waive gates. |
Trade-off analysis¶
Charge costs to the right stage¶
ROCS shares user encoding across candidates in one request, saving online duplicate work. Rec-Distill moves the larger model's influence into training and deploys a lightweight main tower, which may increase training work. ETT measures the share of end-to-end training advancing on new data, not request latency or device utilization. The three metrics may improve together or trade off; record the stage where each change acts.
Sources: ROCS: Request-Oriented Compute Sharing for Efficient Large-Scale Recommendation · Rec-Distill: An Industrial Distillation Pipeline for Large-Scale Recommendation Models · Optimizing Effective Training Time for Large-Scale Recommendation Systems
Automated research must be able to choose no change¶
FORTE puts cost-saving proposals through quality and lineage gates. Retaining the original model when no candidate passes is a valid result. The Self-Evolving RecSys outer state machine also cannot treat completion of an inner experiment as production readiness. ETT reminds us to count failure recovery and duplicate work. Long-term memory should retain failure reasons, budgets, and consumer constraints, not merely names of successful proposals.
Sources: FORTE: A Production Framework for Feature Evaluation and Deprecation in Industrial Recommender Systems · Self-Evolving Recommendation System: End-To-End Autonomous Model Optimization With LLM Agents · Optimizing Effective Training Time for Large-Scale Recommendation Systems
Cross-paper questions¶
Try answering before expanding the explanation. Each question relates at least two papers.
ROCS and Rec-Distill both lower online cost. Why not add their savings directly?¶
Show answer and example
Answer: ROCS eliminates repeated shareable encoding within a request. Rec-Distill changes the deployed model and leaves teacher and auxiliary work in training. The distilled architecture may change the shareable fraction, while kernels and bandwidth affect latency. Measure the combined path instead of multiplying or adding independent speedups.
Illustrative example: A student with much lighter user encoding may leave less duplication for ROCS to remove. Extra teacher training does not appear in per-request latency.
Sources: ROCS: Request-Oriented Compute Sharing for Efficient Large-Scale Recommendation · Rec-Distill: An Industrial Distillation Pipeline for Large-Scale Recommendation Models
Why might Effective Training Time be low even when the GPU stays busy and ROCS speeds up requests?¶
Show answer and example
Answer: ETT counts only training time consuming new data in the numerator. Its denominator includes setup, compilation, checkpointing, publishing, and recovery. GPUs can be busy in those phases, and ROCS targets online inference rather than training. Attribute losses by phase and owner, not device busy time or inference latency.
Illustrative example: If 8 of 10 wall-clock hours train on new data and 2 compile or recover, ETT is 80% even if the GPU is fully occupied for the remaining 2 hours.
Sources: Optimizing Effective Training Time for Large-Scale Recommendation Systems · ROCS: Request-Oriented Compute Sharing for Efficient Large-Scale Recommendation
If two features show almost no loss under separate FORTE perturbations, why can't a research agent delete both?¶
Show answer and example
Answer: The features may back each other up; deleting one leaves the other signal, while deleting both exposes non-additive loss. FORTE also requires joint ablation, retraining, and an audit of all consumers. Send the proposal through actual Self-Evolving RecSys gates and retain failures in the journal instead of inferring safety from single-feature tests.
Illustrative example: A brand ID and a brand word can both identify the brand. Deleting either may barely matter, while deleting both hurts a sparse task. This is illustrative, not a paper measurement.
Sources: FORTE: A Production Framework for Feature Evaluation and Deprecation in Industrial Recommender Systems · Self-Evolving Recommendation System: End-To-End Autonomous Model Optimization With LLM Agents
If a Rec-Distill student scores slightly better offline and a research agent journals it, has it completed the launch loop?¶
Show answer and example
Answer: No. Check student sampling correction, main and auxiliary roles, and matched budgets so the gain is not a protocol change. Then require independent validation and applicable online gates under the Self-Evolving RecSys outer controls. A journal preserves auditable runs and conclusions but does not grant promotion or replace execution evidence.
Illustrative example: If AUC changes after adjusting the student's negative sampling rate, repair any mismatch with the teacher logits' probability space before comparing. A “distillation succeeded” entry cannot fix it.
Sources: Rec-Distill: An Industrial Distillation Pipeline for Large-Scale Recommendation Models · Self-Evolving Recommendation System: End-To-End Autonomous Model Optimization With LLM Agents
Update log¶
Updates record new approaches, changed trade-offs, and follow-up evidence.
2026-10-27 · Initial version: compared ROCS, Rec-Distill, Effective Training Time, FORTE, and Self-Evolving RecSys. Added analysis and questions on the stages where costs occur and the dependencies that must remain consistent for sharing or deletion.
Covers 19 topic entries; see the full list below.
All 19 topic entries
The full catalog follows. The main comparison covers the representative papers listed above.
- AgenticRecTune: Multi-Agent with Self-Evolving Skillhub for Recommendation System Optimization
- AutoLR: Automating the Path from Research to Launch Review in Industrial Recommender Systems
- CORAL: An LLM-Native Harness for Production Recommender Systems
- Optimizing Effective Training Time for Large-Scale Recommendation Systems
- Verify, Don't Trust: Agentic Model Development for Video Discovery Retrieval at Scale
- EvoRec: Self-Evolving Agentic Recommender Systems
- EvoSkillRec: Skill-Genome Evolution for Recommender Architecture Discovery
- FORTE: A Production Framework for Feature Evaluation and Deprecation in Industrial Recommender Systems
- Memory Layer: Train the In-Model Cache for Recommendation Models
- NOVA: A Verification-Aware Agent Harness for Architecture Evolution in Industrial Recommender Systems
- Prompt Generation Technical Report
- RADAR: Recall Augmentation through Deferred Asynchronous Retrieval
- Rec-Distill: An Industrial Distillation Pipeline for Large-Scale Recommendation Models
- RecEvolve: A Knowledge-Driven Autonomous Agent System for Recommender Systems
- RecHarness: A Bandit-Routed Agentic Harness for Self-Evolving Recommender Systems
- ROCS: Request-Oriented Compute Sharing for Efficient Large-Scale Recommendation
- Self-Evolving Recommendation System: End-To-End Autonomous Model Optimization With LLM Agents
- SOLARIS: Speculative Offloading of Latent-bAsed Representation for Inference Scaling
- Compute Only Once: UG-Separation for Efficient Large Recommendation Models