DIN, HSTU, LONGER, SlimPer, and RankMixer: what interacts with what?¶
简体中文 · English
Updated 2026-10-27 · 73 topic entries / 5 representative comparisons
Before comparing ranking backbones, trace history-to-history, candidate-to-history, and field-to-field edges. DIN activates history by candidate; HSTU updates historical states across layers; LONGER merges tokens before a short query reads long KV; SlimPer repeatedly reads the user pool but propagates only a small knowledge base; RankMixer replaces heterogeneous field interactions. These mechanisms may occupy different stages in one pipeline. A single length variable or local leaderboard cannot rank them all.
Mechanisms · Trade-offs · Cross-paper questions · Updates
How to read this card
The table compares representative methods. It is not a shared-benchmark leaderboard. Teaching examples are not measured results.
Mechanism comparison¶
Axes: Interactions within history, target readout, or field mixing · Propagated state and continued access to original history · How candidate dependencies limit user-cache sharing · Complexity variables and limits of local and production evidence
| Paper and source | Mechanism | Benefits and trade-offs |
|---|---|---|
| Deep Interest Network for Click-Through Rate Prediction · Original source Sections in the Chinese source: 背景与主要改动, 核心公式 |
Computes local activation between candidate q and each historical action, then pools a target-specific interest rather than one final user interest for all candidates. | Direct target relevance, but each candidate needs its own activation and aggregation. This main path does not create HSTU-style layer-by-layer history interactions. |
| Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 本地复现 |
Updates history tokens inside the window through causal pointwise aggregation, post-pooling normalization, and a U gate, preserving historical dependencies. | Removing softmax does not make all-pairs interactions linear. The paper's M-FALCON can reuse candidate-independent historical KV, but target-side queries and outputs are still dependent. M-FALCON has not been implemented locally; the reuse is not measured here. |
| LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 本地复现 |
Merges adjacent behavior tokens. In the first layer, a few global/recent queries read long KV and yield query-length outputs; later self-attention operates only on the short output. | Merging and short-query compression are separate and may lose fine-grained addressing. Merging increases width and projection cost. The local implementation only has InnerTrans/ordinary self-attention after aggregation, not the paper's first-layer short-Q/long-KV cross-causal attention or causal mask. Prior metrics cannot validate that path or cache throughput. |
| SlimPer: Make Personalization Model Slim and Smart · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 复现边界 |
Refines a fixed-slot, candidate-dependent knowledge base with Select–Match–Refine. Each layer rereads the shared user token pool and propagates the small KB instead of all historical hidden states. | The user pool still grows with history length, and initial encoding may include local interactions. Candidate-independent user encodings can be shared; final candidate KBs cannot. Short local histories do not establish production long-sequence QPS. |
| RankMixer: Scaling Up Ranking Models in Industrial Recommenders · Original source Sections in the Chinese source: 背景与主要改动, 核心公式, 本地复现 |
Rearranges heterogeneous feature tokens by head and token, then applies a separate FFN at each output position. Learns combinations when rearranged components meet, replacing attention for field interactions. | Token counts must match head and width constraints. Upstream history modules may still use attention; field mixing does not compress very long histories. Check sparse routing and implementation scope separately before generalizing that the full model is attention-free. |
Trade-off analysis¶
Compare layer-to-layer state before FLOPs¶
HSTU updates the history window in every layer; LONGER continues on short query outputs after the first; SlimPer keeps original user tokens readable while propagating a small KB. They save different things. Eliminating full-length propagated state does not eliminate long-history reads, and eliminating softmax does not eliminate pairwise history interactions. DIN's target pooling and RankMixer's field mixing also do not fit one N²-versus-N table.
Sources: Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations · LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders · SlimPer: Make Personalization Model Slim and Smart · Deep Interest Network for Click-Through Rate Prediction · RankMixer: Scaling Up Ranking Models in Industrial Recommenders
A user-side label does not guarantee reuse across candidates¶
DIN pooling already reads the candidate. SlimPer's KB begins with item/cross information, so candidate B cannot reuse A's refined KB. HSTU M-FALCON and LONGER user KV reuse require a shared path that has not consumed target-dependent information. RankMixer often mixes candidates, sequence summaries, and fields late, so its final state is not inherently a reusable user cache. Trace dependencies rather than module names.
Sources: Deep Interest Network for Click-Through Rate Prediction · SlimPer: Make Personalization Model Slim and Smart · Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations · LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders · RankMixer: Scaling Up Ranking Models in Industrial Recommenders
Cross-paper questions¶
Try answering before expanding the explanation. Each question relates at least two papers.
LONGER and SlimPer both compress state. Why does compressing once and never reading the full history again describe only some paths?¶
Show answer and example
Answer: LONGER's first short queries read long KV; later layers mostly work on the short output. SlimPer's small KB queries the user token pool at every layer, retaining access to original evidence. HSTU instead updates history within the window across layers. Ask what each layer can still access instead of calling all three short-sequence compression.
Illustrative example: If an old gardening interaction was missed in layer one, SlimPer can query it again later. LONGER cannot reread it from the full history if the short sequence did not retain it.
Sources: LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders · SlimPer: Make Personalization Model Slim and Smart · Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations
When scoring 100 candidates for one user, can DIN interests, SlimPer KBs, and HSTU historical KV all be cached once?¶
Show answer and example
Answer: No. DIN interests and SlimPer's final KB depend on the candidate. Only base encodings that have not read the target are shareable. Under M-FALCON, HSTU reuses candidate-independent historical KV, but each target still needs its own query and output. A cache requires fixed dependencies, not just the same user.
Illustrative example: A cat-food candidate makes the KB focus on cat ownership. A gardening-tool candidate should attend to flower growing. Reusing the cat-food KB would mistake one target-conditioned state for a general user state.
Sources: Deep Interest Network for Click-Through Rate Prediction · SlimPer: Make Personalization Model Slim and Smart · Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations
If RankMixer does not use attention for interactions, why can it coexist with LONGER or HSTU?¶
Show answer and example
Answer: RankMixer handles interactions across heterogeneous field tokens. LONGER and HSTU focus on behavioral history. A history module can produce a summary or state as input fields for TokenMix and PFFN. Field count T and history length N are different variables, so replacing field attention does not determine full-model cost or long-history capability.
Illustrative example: A history module emits recent and long-term interests. Alongside price, device, and candidate category, these make five field tokens. Late TokenMix does not replace upstream history retrieval and encoding.
Sources: RankMixer: Scaling Up Ranking Models in Industrial Recommenders · LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders · Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations
Update log¶
Updates record new approaches, changed trade-offs, and follow-up evidence.
2026-10-27 · Initial version: compared DIN, HSTU, LONGER, SlimPer, and RankMixer. Added analysis and questions on interaction targets, layerwise state, and continued access to original history.
Covers 73 topic entries; see the full list below.
All 73 topic entries
The full catalog follows. The main comparison covers the representative papers listed above.
- AdaF²M²: Comprehensive Learning and Responsive Leveraging Features in Recommendation System
- Scaling Recommender Transformers to One Billion Parameters
- Behavior Sequence Transformer for E-commerce Recommendation in Alibaba
- CADET: Context-Conditioned Ads CTR Prediction With a Decoder-Only Transformer
- CCFormer: Efficient Cross-Field Interaction and Hierarchical Sequence Compression for Industrial Recommendation at Tencent
- ChronicleRec: Pre-training Temporally Anchored Tokens for Lifelong User Modeling
- ClockRoPE: Random Fourier Rotations for Temporal Routine Modeling
- CMSL: Constructive Multi-Sequence Learning for Recommendation Systems
- DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems
- DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
- Deep Interest Evolution Network for Click-Through Rate Prediction
- Deep Interest Network for Click-Through Rate Prediction
- ECHO: Efficient Cascade with Homogeneous Operators for Distillation-Guided 100K-Sequence Modeling
- FuXi-α: Scaling Recommendation Model with Feature Interaction Enhanced Transformer
- Towards Large-scale Generative Ranking
- GenRec: An LLM-Backed Recommendation Ranker at Netflix
- Heterogeneous Ranking in Industrial-Scale Recommender Systems: A Case Study
- HELIX: Purified and Unified - Rethinking Feature Interaction and Sequence Modeling for Large-Scale Recommendation
- HiSAC: Hierarchical Sparse Activation Compression for Ultra-long Sequence Modeling in Recommenders
- Beyond the Flat Sequence: Hierarchical and Preference-Aware Generative Recommendations
- Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations
- HyFormer: Revisiting the Roles of Sequence Modeling and Feature Interaction in CTR Prediction
- IntHQ: Task-Interactive Hierarchical Query on Dual-Stream Representations for Generative Recommendation
- JourneyFormer: Encoding Airbnb Guest Journey with Sequence Modeling
- Kunlun: Establishing Scaling Laws for Massive-Scale Recommendation Systems through Unified Architecture Design
- LazFormer: Scaling Transformers for Industrial Recommendation via Transferable Generative Pre-training
- An Industrial-Scale Sequential Recommender for LinkedIn Feed Ranking
- LLaTTE: Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendation
- Long-History User Transformers for Real-Time Ad Ranking
- LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders
- MDL: A Unified Multi-Distribution Learner in Large-scale Industrial Recommendation through Tokenization
- Memento: Personalized RAG-Style Long-Retention Data Scaling for Online Ads Recommendation
- MixFormer: Co-Scaling Up Dense and Sequence in Industrial Recommenders
- ML-DCN: Masked Low-Rank Deep Crossing Network Towards Scalable Ads Click-through Rate Prediction at Pinterest
- Compress, Cross and Scale: Multi-Level Compression Cross Networks for Efficient Scaling in Recommender Systems
- MSN: A Memory-based Sparse Activation Scaling Framework for Large-scale Industrial Recommendation
- Not Only NTP: Extending Training Signal Coverage for Generative Recommendation
- OneModel: A Unified Foundation for Platform-Scale Multi-Scenario Ranking
- OnePiece: Bringing Context Engineering and Reasoning to Industrial Cascade Ranking
- OneRank: Unified Transformer-Native Ranking Architecture for Multi-Task Recommendation
- OneTrans: Unified Feature Interaction and Sequence Modeling with One Transformer in Industrial Recommender
- OneTrans-V2: Unifying Retrieval, Pre-rank, and Fine-rank with One Transformer in Industrial Recommender
- PinFM: Foundation Model for User Activity Sequences at a Billion-scale Visual Discovery Platform
- POEM: Partial-Order Enhanced Real-Time Sequential Modeling for Recommendation
- RankMixer: Scaling Up Ranking Models in Industrial Recommenders
- RankUp: Towards High-rank Representations for Large Scale Advertising Recommender Systems
- Beyond Interaction Capacity: Estimator Scaling with Recursive Models for CTR Prediction
- Not All Is Lost: Repairing Lossy User Preference States of Personalization Encoders
- From Language to Behavior: Scaling Sequence Transformers for Industrial Recommendation Ranking with Rec-Native Designs
- Learning to Forget: Satiation-Aware Long-Sequence Transducers for Mitigating Post-Purchase Redundancy
- Self-Attentive Sequential Recommendation
- Beyond Item IDs: Scaling Short-Form-Video Recommendation via Semantic-Native Long Sequence Modeling
- SequenceO1: End-to-End Ultra-Long (100K) Sequence Modeling in Recommendation with Low-Rank Caching
- Search-based User Interest Modeling with Lifelong Sequential Behavior Data for CTR Prediction
- SlimPer: Make Personalization Model Slim and Smart
- SORT: A Systematically Optimized Ranking Transformer for Industrial-scale Recommenders
- Unleashing the Potential of Sparse Attention on Long-term Behaviors for CTR Prediction
- SpecFormer: Mitigating Embedding and Attention Collapse via Spectral-Aware Transformer for Recommendation
- Teacher Retains Full Tokens, Student Merges Efficiently: TM20K for E-Commerce Sequence Modeling in Ad Recommendation
- TMallGS: Scaling Unified Feature and Sequence Modeling for Generative E-commerce Search
- TokenMixer-Large: Scaling Up Large Ranking Models in Industrial Recommenders
- TransAct V2: Lifelong User Action Sequence Modeling on Pinterest Recommendation
- TransX: Scaling Transformer-based Recommendation via Behavioral and Serving Stream Crossings
- TubiFM: Unified Item, Carousel, and Search Ranking for Streaming Discovery
- TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou
- Bending the Scaling Law Curve in Large-Scale Recommendation Systems
- UniFormer: Efficient and Unified Model-Centric Scaling for Industrial Recommendation
- UniRec: Cross-Stage Multi-Task Fusion with Preference Optimization for Cascaded Recommender Systems
- UniScale: Synergistic Entire Space Data and Model Scaling for Search Ranking
- UxSID: Semantic-Aware User Interests Modeling for Ultra-Long Sequence
- WHALE: A Scalable Unified Model for Recommendation with Wukong-HSTU Architecture
- Wide & Deep Learning for Recommender Systems
- Zenith: Scaling up Ranking Models for Billion-scale Livestreaming Recommendation