User histories and ranking backbones¶
简体中文 · English
Search, ads, and recommendation / 299 questions · 73 papers
Try answering each question before opening the explanation. The bank grows as papers and questions are added.
Read this topic's comparisons, analysis, and cross-paper Q&A
299 questions · 73 papers
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why does state-aware fusion vary across examples?
AdaF²M²: Comprehensive Learning and Responsive Leveraging Features in Recommendation System
Show answer and example
Answer: Activity and feedback change which signals are trustworthy. The adapter weights complementary masks per state; one fusion coefficient is not permanent importance.
Illustrative example: Behavioral history can dominate for an established user and content for a new one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
How do AdaF²M² multiple masked forwards differ from dropping features once?
AdaF²M²: Comprehensive Learning and Responsive Leveraging Features in Recommendation System
Show answer and example
Answer: Several masks retain complementary views. A state-aware adapter combines them by user/item reliability; one perturbed output or uniform averaging does not implement this.
Illustrative example: Combine behavior-masked and content-masked views differently for new and mature items.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does multi-forward fusion differ from feature dropout?
AdaF²M²: Comprehensive Learning and Responsive Leveraging Features in Recommendation System
Show answer and example
Answer: Single dropout trains robustness to missing inputs. Multi-forward retains alternatives and learns state-specific fusion, adding execution cost even at equal parameter count.
Illustrative example: Keeping price-present and price-absent predictions for fusion differs from randomly deleting price once.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Which axis should AdaF²M² softmax normalize?
AdaF²M²: Comprehensive Learning and Responsive Leveraging Features in Recommendation System
Show answer and example
Answer: Normalize branches separately for each example, with compatible representation widths. Do not couple different users through a batch-level softmax.
Illustrative example: Two users with three views each need two independent three-weight sums.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
How read ARGUS's feedback–item joint distribution?
Scaling Recommender Transformers to One Billion Parameters
Show answer and example
Answer: Predict feedback from history, then condition item probability on it and multiply. True feedback is training supervision, not an inference input unless specified externally under protocol.
Illustrative example: Likes can favor relatable clips while long watches favor tutorials.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
How does feedback-then-item differ from direct next-item prediction?
Scaling Recommender Transformers to One Billion Parameters
Show answer and example
Answer: ARGUS factors joint action into feedback type then conditional item, sharing a trunk while preserving feedback distinctions. Inference cannot read future feedback. Local module ablation is negative.
Illustrative example: Expected long watch and quick skip can favor different items for the same history.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does conditional factorization differ from a parallel auxiliary head?
Scaling Recommender Transformers to One Billion Parameters
Show answer and example
Answer: A parallel head adds loss without necessarily changing item prediction. ARGUS explicitly conditions items on feedback, so feedback errors can propagate. Measure both.
Illustrative example: Separate “will like” and item outputs are multitask, not a feedback-conditioned generator.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why distinguish DIN and next-item-only baselines?
Scaling Recommender Transformers to One Billion Parameters
Show answer and example
Answer: DIN compares whole models; matched next-item-only isolates feedback factorization. Both local comparisons lack gains, and neither should hide the other's negative evidence.
Illustrative example: Overall proximity to DIN can coexist with degradation after adding the feedback path.
Sections in the Chinese source: 本地复现. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why include BST's candidate token in history encoding rather than concatenating it only in the final MLP?
Behavior Sequence Transformer for E-commerce Recommendation in Alibaba
Show answer and example
Answer: Attention with the candidate lets history representations adapt early and learns both behavior-to-behavior and target-to-history relations. Late concatenation only interacts with an already compressed representation. The result is candidate-conditioned, not universal.
Illustrative example: Hiking shoes can amplify a related camping and hiking history, while a camera should refocus the encoding on photography.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
How does BST jointly attending over a candidate and history differ from DIN using a candidate to weight history?
Behavior Sequence Transformer for E-commerce Recommendation in Alibaba
Show answer and example
Answer: DIN mainly learns candidate-to-action activation. BST also enables interactions among past actions and embeds the candidate jointly, allowing context to change an action's meaning. This costs more sequence computation and can limit reuse across candidates; compare matched budgets and history windows.
Illustrative example: A camera click might be incidental, but later lens and tutorial views reinforce photography intent. A different candidate can change this jointly encoded state.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What does BST's self-attention add over fixed history pooling, and at what cost?
Behavior Sequence Transformer for E-commerce Recommendation in Alibaba
Show answer and example
Answer: Fixed pooling uses predetermined or simplified action weights and can miss combinations. BST learns pairwise contextual dependencies, with sequence-length-related computation and possible re-encoding for each candidate. Flexibility alone is not a win; compare matching data, budgets, and candidate protocols.
Illustrative example: A camera view followed by a lens view can reflect a different purchase stage than a camera view alone. Attention can combine them but costs more with thousands of events.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why not cache BST's candidate-conditioned state as a user embedding shared by all candidates?
Behavior Sequence Transformer for E-commerce Recommendation in Alibaba
Show answer and example
Answer: A candidate participating in self-attention can write into hidden history states; substituting it changes downstream inputs. Without an isolation mask, caching is valid only for the specific dependency conditions. The local candidate-aware Transformer versus DIN is not a demonstration of shared user-side caching.
Illustrative example: A state computed for shoes can emphasize shoe history and bias a movie score. A shared cache must stop before candidate information is written.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why does CADET emphasize when post-scoring context becomes visible?
CADET: Context-Conditioned Ads CTR Prediction With a Decoder-Only Transformer
Show answer and example
Answer: Training logs may include information created only after scoring. Using it before scoring misaligns training and serving. Encode cacheable history first, then introduce context only at its legitimate stage; timing is part of the feature contract.
Illustrative example: A user's reaction after an impression can enter the next history, but cannot predict the current click before that impression.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does CADET put cacheable history and post-scoring context in different information-flow stages?
CADET: Context-Conditioned Ads CTR Prediction With a Decoder-Only Transformer
Show answer and example
Answer: The cache must remain valid when candidates or later context change. CADET encodes history and scores candidates before injecting post-scoring context at the allowed stage, preventing use of information unavailable at serving time. Mixing everything from the start can invalidate reuse and create training-serving mismatch despite better offline scores.
Illustrative example: Do not write ad A's post-scoring context into shared history before evaluating ad B. Keep the shared prefix distinct from later conditional state.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does staged context in CADET differ from feeding all log fields into a Transformer together?
CADET: Context-Conditioned Ads CTR Prediction With a Decoder-Only Transformer
Show answer and example
Answer: Indiscriminate concatenation can expose unavailable fields and leak candidate computations into shared history. Separating cacheable state from post-scoring context limits arbitrary interactions but preserves serving consistency. Local public-data proxies do not verify the full timing of production advertising features.
Illustrative example: Mixing current-request features with the next action in one sequence invites a shortcut; stage boundaries allow only evidence available when the request arrives.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
When does new CADET context invalidate cached history K/V?
CADET: Context-Conditioned Ads CTR Prediction With a Decoder-Only Transformer
Show answer and example
Answer: Check whether it changes history inputs, masks, or model parameters. The shared K/V stays valid only if context enters a later candidate-query path. The source formula lets a context query read prior history; it does not authorize feeding arbitrary context back into the cached backbone. Inspect actual tensor dependencies.
Illustrative example: A change to the query can reuse the history K/V; changing history time encodings requires recomputation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why project CCFormer fields separately instead of treating IDs, content, and history as homogeneous tokens?
Show answer and example
Answer: Different sources and scales benefit from their own projections, followed by cross-attention, which limits dominance by a strong field. Hierarchical compression separately controls history cost. This jointly addresses feature interaction and sequence length, not simply Transformer depth.
Illustrative example: A user ID is a sparse lookup and content represents semantics. Separate projections preserve both before interaction instead of favoring frequent IDs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does CCFormer expand compression receptive fields for distant history while retaining recent detail?
Show answer and example
Answer: Old sequences are large, and per-event interactions can consume ranking budget. Recent behavior can signal changing intent and lose detail if merged early. CCFormer hierarchically mixes old tokens while keeping field-specific projections and cross-attention. Sequence shortening and field interaction solve different problems, so length alone cannot measure effectiveness.
Illustrative example: Merge many similar views from last month but preserve today's views of two different sizes; candidate fields can still query compressed long-term interests.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What do CCFormer's progressive compression and fixed last-N truncation each lose?
Show answer and example
Answer: Truncation removes distant events entirely; compression retains coarse older signals at less cost but can erase rare details. Keeping recent resolution while broadening old receptive fields is not lossless. A local GRU comparison does not establish a ranking of all long-sequence methods.
Illustrative example: Compression can retain last year's photography interest and three recent camping actions but forget one camera model. Last-ten truncation may forget photography entirely.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How does hierarchical token mixing in CCFormer differ from compressing the whole sequence uniformly?
Show answer and example
Answer: Expand the old-history receptive field by layer and temporal region, keep the recent fine-grained path, and use field-specific query projections for interactions. Uniform merging loses temporal resolution allocation. For the small local core, verify connectivity and length changes rather than just runnable tensor shapes.
Illustrative example: The first layer can keep four recent actions as individual tokens and merge eight older ones. A later layer expands the old window without averaging away a just-recorded key click.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why does ChronicleRec represent near, middle, and distant history instead of a few unordered interest tokens?
ChronicleRec: Pre-training Temporally Anchored Tokens for Lifelong User Modeling
Show answer and example
Answer: Multi-scale pooling preserves temporal structure. Query anchors delimit the past being summarized so stable preferences and recent changes remain distinct. Candidate-independent Chronicle Tokens can be cached, but public compression can still discard target-specific details.
Illustrative example: A long-term science-fiction preference and this week's parenting interest mean more with time scales than as two unordered keywords.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why must a Chronicle Token have a time anchor and summarize only preceding events?
ChronicleRec: Pre-training Temporally Anchored Tokens for Lifelong User Modeling
Show answer and example
Answer: Without the boundary, an earlier representation could include future events, violating temporal causality and making cache meaning unclear. ChronicleRec compresses different ranges, anchors queries on the timeline, reads earlier events only, and masks recent behavior in multiple ways to align short-term intent. Anchored candidate-independent summaries are reusable across candidates.
Illustrative example: A Wednesday summary must not include Friday's camera purchase when scoring Thursday lens recommendations.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does ChronicleRec alignment pretraining differ from ordinary next-item prediction?
ChronicleRec: Pre-training Temporally Anchored Tokens for Lifelong User Modeling
Show answer and example
Answer: Branches hide different lengths of recent actions so compressed older history must predict recent intent without nearby evidence. Ordinary next-item prediction may lean on adjacent actions. Do not return hidden targets to evaluation inputs, or the compression test loses its meaning.
Illustrative example: Hide the latest three parenting views and predict the new interest from older history instead of guessing the next item from the last parenting item.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why normalize Chronicle Token pooling by valid mask count and read at the query position?
ChronicleRec: Pre-training Temporally Anchored Tokens for Lifelong User Modeling
Show answer and example
Answer: Padding should not shrink short histories; masks and a safe denominator average real events only. Read after causal encoding at the query anchor so the state summarizes only earlier events. Other positions and padded events can alter semantics without causing shape errors.
Illustrative example: A four-slot window with two real actions divides by two. Put the query after both to summarize both; a query between them cannot see the later one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why use a periodic kernel in ClockRoPE rather than assuming relevance always decays with distance?
ClockRoPE: Random Fourier Rotations for Temporal Routine Modeling
Show answer and example
Answer: Habits may repeat by hour or weekday. The periodic kernel restores similarity at specific offsets through its rotary frequencies. It is a prior, and proxies without real timestamps are not evidence of periodic behavior.
Illustrative example: A Friday film habit makes a seven-day gap informative; for an irregular viewer the same periodic bonus can add noise.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why can ClockRoPE give higher relevance to a one-week gap than to a two-day gap?
ClockRoPE: Random Fourier Rotations for Temporal Routine Modeling
Show answer and example
Answer: Behavior may recur periodically rather than decay monotonically. ClockRoPE samples rotary frequencies from the desired periodic kernel's spectrum so query-key dot products strengthen again at matching times of day or week. Ordinary RoPE can share channels to retain recency. Periodicity is a bias, not a universal user habit.
Illustrative example: Last Monday morning commute news can be more relevant to this Monday than Saturday-night entertainment.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does ClockRoPE differ from adding a weekday feature?
ClockRoPE: Random Fourier Rotations for Temporal Routine Modeling
Show answer and example
Answer: A weekday feature is an absolute time label. ClockRoPE changes pairwise attention directly through relative query-key rotations and a periodic similarity kernel. They may overlap or complement one another; the mechanism alone does not guarantee better results.
Illustrative example: A Friday label says what day it is; rotation directly relates this Friday to last Friday in attention.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why must ClockRoPE's day and week periods match the timestamp unit?
ClockRoPE: Random Fourier Rotations for Temporal Routine Modeling
Show answer and example
Answer: The local kernel uses periods 24 and 168 when gaps are in hours. Raw seconds shift phase, and sampled frequencies and mixed channels must respect rotary dimension pairing. Test re-enhancement one period apart, not just tensor shapes.
Illustrative example: Convert seven days to 168 hours. Feeding the raw second count interprets the same interval as many incorrect periods.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why does CMSL condition contextual lenses on current nonsequence context?
CMSL: Constructive Multi-Sequence Learning for Recommendation Systems
Show answer and example
Answer: Context determines which historical events explain a latent intent together. Soft routing creates multiple strands for encoding and context-query aggregation, avoiding fixed business categories that hide more specific needs. Human-interpretable themes are not guaranteed; check redundancy and collapse.
Illustrative example: Travel can emphasize light equipment, while a home context emphasizes storage within the same shopping history.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does CMSL's contextual lens not prebucket events into click and purchase types?
CMSL: Constructive Multi-Sequence Learning for Recommendation Systems
Show answer and example
Answer: Fixed event types need not match current latent intent. CMSL learns context-conditioned soft assignment into semantic strands, encodes them separately, and aggregates them using nonsequence context. Check duplication and loss of effective history rather than assuming strands are complementary.
Illustrative example: A kitchen-item click can indicate restocking or preparing to move. Different lenses can weight it differently without a fixed click bucket.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How do CMSL strands differ from simply adding attention heads?
CMSL: Constructive Multi-Sequence Learning for Recommendation Systems
Show answer and example
Answer: Heads project the same input sequence into different subspaces. A lens first reweights or groups events under context, creating distinct latent sequences before encoding. CMSL's linear HSTU uses a specific kernel approximation; strands do not automatically linearize standard HSTU.
Illustrative example: More heads read the same ledger in different ways. Lenses first reorganize it into separate ledgers, which still need effective encoding.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why check width costs separately in CMSL's second-order feature-map approximation?
CMSL: Constructive Multi-Sequence Learning for Recommendation Systems
Show answer and example
Answer: Decomposing the SiLU dot product can reorder multiplication to reduce quadratic dependence on history length. Second-order cross terms grow with feature width, so linear length scaling does not settle FLOPs or constants. The local run checks learned lenses and an HSTU-style path; verify the production kernel separately.
Illustrative example: Doubling history from 1,000 to 2,000 can roughly double the length term, while doubling feature width can expand the feature map by much more.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why does depth control the order of explicit crosses in DCN-V2?
DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems
Show answer and example
Answer: Each layer elementwise multiplies a transform of current state by the original input, building higher orders while residuals retain lower ones. This gives traceable polynomial interaction paths, unlike generic MLP fitting. Rank, width, and training still constrain effective expressivity.
Illustrative example: A user age–category interaction can gain another original feature at the next layer without losing its lower-order residual.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does each DCN-V2 cross layer multiply the current transform by the original input x₀?
DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems
Show answer and example
Answer: The original anchor creates traceable higher-order combinations as depth increases instead of relying solely on implicit MLP interactions. Matrix crosses increase expressivity; low-rank factors and expert gating control cost. Compression can reduce capacity and must be tested with the deep path.
Illustrative example: Cross user and category first, then multiply by the original time feature to form a third-order interaction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What is the capacity-compute trade-off between DCN-V2 low-rank expert mixing and a full cross matrix?
DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems
Show answer and example
Answer: U and V factor the large matrix through a smaller subspace. Multiple gated experts add input-dependent flexibility but also storage and routing. A single low-rank expert limits representable directions; parameter savings do not imply equal end-to-end latency savings because embeddings and other modules still cost time.
Illustrative example: Project thousands of dimensions to dozens and back, with separate expert crosses. Too little rank can lose rare, important relationships.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Which layers provide x0 and xl in DCN-V2?
DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems
Show answer and example
Answer: x0 is always the cross stack's original input and xl its current state; replacing x0 with xl changes the cross. Map each low-rank expert back to original width before multiplication and normalize gates over experts. Matching residual shapes does not verify the operator.
Illustrative example: Layer two still multiplies the original user and item vectors rather than squaring the previous state, despite matching shapes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
What do DeepFM's first-order, second-order, and DNN outputs contribute?
DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
Show answer and example
Answer: First-order terms give individual effects, FM vector products capture pairs, and the DNN learns richer nonlinear interactions. They meet in one click logit with shared embeddings; orders can overlap and probabilities should not be added.
Illustrative example: A city effect, a city–category pair, and a more complex price/time condition all contribute to the same logit.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why do DeepFM's FM and DNN share field embeddings?
DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
Show answer and example
Answer: FM explicitly models first- and second-order terms and the DNN learns higher-order nonlinear interactions. Shared field semantics reduce duplicated parameters and manual crosses and let both paths train the embeddings. Their coupled gradients need not be conflict-free.
Illustrative example: FM learns occupation–category pairs, while DNN combines occupation, category, and price; both update the same field vectors.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How do DeepFM's engineering assumptions differ from hand-crossed Wide & Deep?
DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
Show answer and example
Answer: FM learns pairwise relations from vector inner products and shares fields with the deep path, reducing manual crosses. Explicit crosses offer exact rules but cost coverage and maintenance. Embedding limits remain, as do field design and governance.
Illustrative example: FM can learn city–category effects without one column per pair, but an illegal business combination still needs an explicit constraint.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why sum only i<j in DeepFM's second-order term?
DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
Show answer and example
Answer: FM models distinct feature pairs. Self-products add squares and counting both orders changes scale. Efficient computation uses half the difference between the square of the sum and the sum of squares. The FM and DNN must share parameters, not copies with the same initial values.
Illustrative example: Count age–category once, not age–age. Copies of vectors would diverge once the two paths train.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why separate interest extraction and target-related evolution in DIEN?
Deep Interest Evolution Network for Click-Through Rate Prediction
Show answer and example
Answer: The first stage extracts predictive interest states from behavior with auxiliary supervision. The second uses candidate relevance to control their update strength for the target. This avoids equating raw clicks with interests and remains sensitive to auxiliary negatives and noisy histories.
Illustrative example: Camera and lens views establish photography interest, which matters more for a travel camera than for a novel.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
How does DIEN multiplying target relevance into a GRU update gate differ from final history pooling?
Deep Interest Evolution Network for Click-Through Rate Prediction
Show answer and example
Answer: Auxiliary prediction learns per-step interests, and candidate attention modulates each state update, letting relevant events revise memory while suppressing irrelevant changes. Late pooling cannot undo earlier overwrites. More recurrent computation need not help short histories.
Illustrative example: For a lens recommendation, a random food-ad click should weakly update a strong photography state rather than overwrite it before final pooling.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
When does candidate information enter DIEN compared with DIN target attention?
Deep Interest Evolution Network for Click-Through Rate Prediction
Show answer and example
Answer: DIN weights history at aggregation, whereas DIEN multiplies relevance into each recurrent update gate, controlling retained and new state throughout evolution. Extra computation and training are not guaranteed improvements; the local gain did not reproduce.
Illustrative example: A lens view can revise recurrent state immediately for a camera candidate; late weighting only reads what survives existing compression.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How should DIEN auxiliary next-event supervision avoid confusing training labels with test futures?
Deep Interest Evolution Network for Click-Through Rate Prediction
Show answer and example
Answer: Use next events as training labels while extraction inputs stop at current time and negatives follow training rules. Do not feed a held-out event into the GRU at evaluation. Target gating must update recurrent state step by step, not merely weight the final vector.
Illustrative example: Predict event three from position two during training; feeding event three when evaluating that prediction leaks the answer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why is DIN local interest candidate-conditioned rather than a single short-term interest?
Deep Interest Network for Click-Through Rate Prediction
Show answer and example
Answer: Local means the relevant subset of all history for the current target. A user can have several interests; candidate changes alter activations and aggregation without requiring one label. Attention cannot create missing evidence, and low activation does not imply permanent disinterest.
Illustrative example: The same user can like science fiction and cooking, activating different histories for a movie and cookware.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why can DIN produce different interest vectors for the same user?
Deep Interest Network for Click-Through Rate Prediction
Show answer and example
Answer: Which historical actions matter depends on the candidate. DIN learns activations from each candidate-action feature combination and pools accordingly. Greater expressivity prevents blanket reuse of the resulting state across candidates; compare against matched-budget pooling.
Illustrative example: Running socks emphasize running-shoe history, while baking tools emphasize oven activity in the same user history.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does DIN local activation differ from dot-product attention?
Deep Interest Network for Click-Through Rate Prediction
Show answer and example
Answer: It takes the candidate, action, difference, and elementwise product to learn richer matching. Dice adapts its boundary to feature distributions. This does not imply causality or guaranteed gain; the local matched comparison showed no improvement over mean pooling.
Illustrative example: Near-equal dot products can hide price differences and shared categories that the difference and product features highlight.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Over which axis should DIN normalize weights, and how should it handle padding?
Deep Interest Network for Click-Through Rate Prediction
Show answer and example
Answer: Normalize over valid history positions for one user and candidate, masking padding first. Do not normalize across candidates or batch examples. Keep Dice training statistics and serving state consistent without adjusting from test labels.
Illustrative example: With three actions and five pads, distribute weights only over the three valid positions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why retain history record addresses rather than only latent interest vectors in ECHO?
ECHO: Efficient Cascade with Homogeneous Operators for Distillation-Guided 100K-Sequence Modeling
Show answer and example
Answer: Latents are compact but cannot recover concrete candidate-specific events. Addressable records enable targeted small-subset retrieval and user-side reuse. This does not imply scanning all history per candidate; serving must follow the exported index and parameter path, beyond the local loss component.
Illustrative example: A camera query can retrieve a recent macro-photography view and running shoes can retrieve exercise evidence, neither recoverable from a vague outdoors vector.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does ECHO distill the full teacher distribution and also penalize misses near the selection boundary?
ECHO: Efficient Cascade with Homogeneous Operators for Distillation-Guided 100K-Sequence Modeling
Show answer and example
Answer: Soft targets communicate relative history relevance but can leave crucial records outside a limited budget. Boundary pairs compare a better missed record against a weaker selected one, using detached teacher-score differences to drive swaps. The component retains addresses, and the local run verifies loss and gradients, not 100K-history performance.
Illustrative example: With budget two, the student misses a highly valued record and keeps a weak one. KD conveys overall order; the pair directly promotes the swap.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How do ECHO boundary pairs and full-distribution KD supervise different errors?
ECHO: Efficient Cascade with Homogeneous Operators for Distillation-Guided 100K-Sequence Modeling
Show answer and example
Answer: KD transfers overall teacher preference; pairing missed strong records with selected weak ones focuses on swaps that change Top-K. Teacher quality gaps weight pairs. The discrete selection remains nondifferentiable; selected student logits receive gradients.
Illustrative example: Positions ten and eleven can have similar global probabilities yet determine whether a vital record is kept. Boundary loss targets that pair.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why normalize ECHO by valid pair weights across the batch and detach teacher values and weights?
ECHO: Efficient Cascade with Homogeneous Operators for Distillation-Guided 100K-Sequence Modeling
Show answer and example
Answer: Averaging by row can inflate requests with few pairs. Batch normalization preserves defined pair contributions. Freeze teacher and mining weights, update matching student scores only, and exclude padding from probabilities, selection, and gradients. A finite scalar alone is insufficient verification.
Illustrative example: With ten pairs on one row and one on another, per-row means can give the lone pair tenfold relative weight; a shared denominator avoids that.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why divide feature interactions across FuXi-α channels?
FuXi-α: Scaling Recommendation Model with Feature Interaction Enhanced Transformer
Show answer and example
Answer: Heterogeneous fields encode different relations; widening one space can still favor the strongest. Separate adaptive channels and staged FFN fusion can retain complementary structure, not guarantee it.
Illustrative example: Channels can specialize in user–item fit and context–price interaction, or all collapse to popularity.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does FuXi-α use multiple interaction channels and staged FFNs instead of widening one attention path?
FuXi-α: Scaling Recommendation Model with Feature Interaction Enhanced Transformer
Show answer and example
Answer: One path can mix unlike relationships in the same space. Adaptive channels model different interactions and multistage FFNs combine them gradually. Additional channels can still learn redundant patterns; compare single-channel and matched-budget configurations.
Illustrative example: User–item, context–item, and history–candidate relations can be learned separately, but identical attention channels only add cost.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Where does FuXi-α's staged fusion differ from concatenating channel outputs only at the final head?
FuXi-α: Scaling Recommendation Model with Feature Interaction Enhanced Transformer
Show answer and example
Answer: Intermediate FFNs fuse current state and channel outputs before later layers, allowing further relation composition. Late fusion has one opportunity. Additional capacity costs more and did not prevent local NDCG decline.
Illustrative example: A later layer can combine nighttime context and price sensitivity into demand for evening promotions rather than only merging final summaries.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
What shapes and update order does FuXi-α require for progressive residual fusion?
FuXi-α: Scaling Recommendation Model with Feature Interaction Enhanced Transformer
Show answer and example
Answer: Channels may have distinct Q/K/V. Project their concatenated state back to backbone width for residual addition, preserve layer and old/new state order, and train channel weights rather than concatenate fixed similarities. Local comparisons only cover the implemented budget.
Illustrative example: Three d-wide channel outputs plus backbone features must pass through an FFN back to d before addition.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why can GenRank retain multi-action learning with items and actions at one position?
Towards Large-scale Generative Ranking
Show answer and example
Answer: Actions remain input features or supervision, just not separate positions. Position and time biases preserve temporal cues. This compresses organization, not behavioral relevance, and equivalence needs testing.
Illustrative example: An impression and its known click can share a position while clicks and skips remain different labels.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does merging actions into item tokens change HSTU-style ranking cost in GenRank?
Towards Large-scale Generative Ranking
Show answer and example
Answer: Interleaving doubles positions per event, increasing quadratic interactions. GenRank keeps action features or supervision on item positions and adds position/time biases. Fewer tokens do not guarantee equivalent expression; local quality changed alongside speed.
Illustrative example: Ten events become ten instead of twenty positions, reducing the attention matrix while potentially losing action dependencies.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Why compare latency and AUC for GenRank versus interleaved item/action ranking?
Towards Large-scale Generative Ranking
Show answer and example
Answer: Fewer positions reduce attention work but can sacrifice interaction expressivity. Local latency and AUC both decreased, so efficiency is not unchanged accuracy or superiority to HSTU. Hardware and candidates limit extrapolation.
Illustrative example: One hundred events can use 100 rather than 200 positions but lose action-order detail; speed alone hides that.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why not input GenRank's current prediction target like historical actions?
Towards Large-scale Generative Ranking
Show answer and example
Answer: History actions are known and can accompany items; the current action is the label. Separate fields and causal masks and use only request-time information in position/time bias. Matching shapes do not rule out leakage.
Illustrative example: Include a prior camera click, not the true future click on the shoe being scored.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
How does Netflix GenRec's catalog-aware head prevent out-of-catalog hallucinations?
GenRec: An LLM-Backed Recommendation Ranker at Netflix
Show answer and example
Answer: A pooled prefill state scores only fixed catalog item IDs, not generated names. Text context and a language objective supply semantics. In-catalog outputs can still be irrelevant or unavailable; inventory and serving mappings need separate checks.
Illustrative example: An unreleased title known to the LLM cannot be recommended if there is no output column for it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does Netflix GenRec not generate a recommendation list token by token despite using a decoder-only LLM?
GenRec: An LLM-Backed Recommendation Ranker at Netflix
Show answer and example
Answer: One prefill encodes text context, then a catalog-aware head scores items directly. This keeps semantic interactions while avoiding multi-step decoding and out-of-catalog generation. A joint language objective improves representation, not free-form serving; the local single seed does not establish Netflix-scale gains.
Illustrative example: Score every catalog movie once instead of writing one title and feeding it back to generate the next.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does Netflix GenRec reward-weighted ranking differ from online RL over generated lists?
GenRec: An LLM-Backed Recommendation Ranker at Netflix
Show answer and example
Answer: Multiple rewards weight examples in supervised catalog ranking, alongside a language objective. There is no token-by-token rollout or list policy gradient; serving uses prefill scoring. The two reward uses have different probability and compute paths.
Illustrative example: Giving a satisfied example more loss weight is not generating ten online results and running PPO.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why isn't Netflix GenRec's local GRU comparison a parameter-matched architecture ablation?
GenRec: An LLM-Backed Recommendation Ranker at Netflix
Show answer and example
Answer: The experiment loads a larger public causal LLM and trains LoRA, while the baseline is ID-only GRU. Parameters and semantic inputs differ; it compares pipelines, not one isolated operator. Single seed, reduced catalog, and proxy rewards also preclude comparing directly to reported online percentages.
Illustrative example: LLM gains may come from title semantics or pretrained weights, not entirely from the catalog head.
Sections in the Chinese source: 本地复现, 对照口径. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why choose HA-MoE experts by heterogeneity instead of one fixed model per content type?
Heterogeneous Ranking in Industrial-Scale Recommender Systems: A Case Study
Show answer and example
Answer: Independent models cannot share common knowledge, while one tower can favor high-volume content. Gated combinations share useful general signals while retaining specialization, which must be measured rather than assumed from expert names.
Illustrative example: Sports news can use both a freshness expert and a sports specialist.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why route HA-MoE experts by content heterogeneity and monitor domain DL-AUC?
Heterogeneous Ranking in Industrial-Scale Recommender Systems: A Case Study
Show answer and example
Answer: In mixed feeds, frequent domains can dominate shared gradients. Contextual gates combine general and specialist experts, and per-domain metrics plus utilization reveal degradation masked by averages. More experts do not prove specialization.
Illustrative example: News volume can hide sports-video losses in aggregate scores, which domain monitoring exposes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Why can LENS per-domain DL-AUC disagree with aggregate AUC?
Heterogeneous Ranking in Industrial-Scale Recommender Systems: A Case Study
Show answer and example
Answer: High-volume domains dominate aggregates. Slice metrics and expert usage show which types are ignored and where gains originate. Monitoring does not guarantee learning; local NDCG did not rise, and lower head share is not a quality substitute.
Illustrative example: If video is 90% of traffic, a slight video gain can mask a large news regression.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Over what axis should HA-MoE apply gate softmax, and what should LENS group by?
Heterogeneous Ranking in Industrial-Scale Recommender Systems: A Case Study
Show answer and example
Answer: Normalize per example over experts and align expert outputs before mixing. Record utilization by domain and time. Local session entropy proxies heterogeneity and measures a fixed expert set, not all open-web content.
Illustrative example: Give one news request weights over four experts; do not normalize news and video examples together.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why does HELIX expand both sequence modeling and heterogeneous feature interaction?
Show answer and example
Answer: History and feature combinations supply distinct information. Interleaved modules let candidate states read history at multiple depths while preserving user-only state. This is not late concatenation, and caching constrains direction.
Illustrative example: Layers can combine travel history with price and device features; enlarging either path alone cannot provide the other's evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does HELIX forbid candidate tokens from writing back to shared sequence state?
Show answer and example
Answer: One-way dependence keeps expensive user-history encoding candidate-independent and reusable within a request while candidates repeatedly read it for deep interaction. Writes back would make history candidate-dependent. Local checks cover sensitivity and cache invariance, not production throughput or gains.
Illustrative example: A shoe can reuse the history read by a camera only if the camera did not modify that state.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How do HELIX one-way communication and WHALE-style bidirectional exchange differ in cacheability?
Show answer and example
Answer: HELIX blocks candidate-conditioned writes into public sequence state. Bidirectional exchange allows deeper co-evolution but creates candidate dependence. Both can use cross-attention without equivalent dependencies, and no universal quality ordering follows.
Illustrative example: After a camera reads shared history, shoes can read the same cache only if the camera state never wrote back.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why check candidate deltas, invariant user caches, and finite gradients in HELIX?
Show answer and example
Answer: Candidate changes should alter scores without changing public user state. Missing either indicates ignored candidates or cache pollution; finite loss and gradients test trainability. These are diagnostics, not quality metrics.
Illustrative example: Identical scores suggest candidate insensitivity; changing scores and cache together breaks sharing despite successful forward execution.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why aren't HiSAC interest agents a group of live language agents?
HiSAC: Hierarchical Sparse Activation Compression for Ultra-long Sequence Modeling in Recommenders
Show answer and example
Answer: Each is a representation aggregated from hierarchical codes and behavior. Quantization, voting, routing, and weighting operate over related nodes without independent LLM calls. The name describes structure, not agent reasoning or serving cost.
Illustrative example: Photography and camping agents can be two vectors; selecting one needs no inter-agent messages or generated text.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does HiSAC form hierarchical interest agents and route each query to only a few?
HiSAC: Hierarchical Sparse Activation Compression for Ultra-long Sequence Modeling in Recommenders
Show answer and example
Answer: Full-history attention is expensive and one summary can lose long-tail interests. Hierarchical codes and voting create personalized compressed units; queries read only relevant ones. These are not autonomous LLMs, and routing cannot recover interests lost in compression or assignment.
Illustrative example: For lenses, read the photography agent rather than cooking or cycling; discarded astrophotography evidence remains unavailable.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What choices does HiSAC's hierarchical interest routing retain compared with a single fixed user vector?
HiSAC: Hierarchical Sparse Activation Compression for Ultra-long Sequence Modeling in Recommenders
Show answer and example
Answer: Separate nodes preserve interests at different levels of granularity, and a query can use them as needed instead of averaging away multiple modes. A single vector gives a simpler interface but makes individual sub-interests harder to recover. Routing may miss rare, low-weight needs, and quantization can lose detail. The source's small NDCG increase and Hit decrease should be interpreted separately.
Illustrative example: Illustration: A camera query can read the photography node for a user who likes photography and running, while running shoes can read the sports node. Averaging the two interests into one vector may dilute both.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why must HiSAC explicitly handle empty nodes and repeated votes when aggregating behavior by code?
HiSAC: Hierarchical Sparse Activation Compression for Ultra-long Sequence Modeling in Recommenders
Show answer and example
Answer: The node mean is normalized by total valid weight. An empty node provides no evidence of an interest and must not produce a strong signal. Hierarchical votes and query-selected subsets must preserve identity. Repeated behavior should count as defined, but expanding one event across levels must not turn it into several independent consumption events.
Illustrative example: Illustration: One photography event can populate both a coarse and a fine node without implying two purchases. An empty diving node is not a real interest just because its initial vector happens to look similar.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why does HPGR use masked session modeling to learn a behavioral hierarchy?
Beyond the Flat Sequence: Hierarchical and Preference-Aware Generative Recommendations
Show answer and example
Answer: Events in a session often share intent, while a change of session may reflect a shift in intent. Learning both levels gives the later preference selector a more meaningful state. A flat sequence based on adjacent positions can mix separate visits. Pretraining supervision comes from observed training data; test input must not reveal a target that should be unknown.
Illustrative example: Illustration: One visit focuses on tents and sleeping bags, the next on hiking boots. A session encoder can distinguish the two needs rather than treating every item as part of one continuous intent.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does HPGR use masked session pretraining before selecting history with preference-guided sparse attention?
Beyond the Flat Sequence: Hierarchical and Preference-Aware Generative Recommendations
Show answer and example
Answer: Session structure captures relationships within a stretch of behavior. Learning it with masking avoids treating the entire long history as a flat sequence of interchangeable events. At fine-tuning time, the current preference selects related history so computation focuses on useful evidence rather than just cutting by recency. Sparse selection can still miss rare but decisive interests, so check selection coverage along with final predictions.
Illustrative example: Illustration: Last week's camera comparisons belong to one session and can be retrieved together for today's lens candidate. The latest unrelated purchase should not displace that organized history just because it is newer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What older evidence can HPGR use that a fixed recent window might miss?
Beyond the Flat Sequence: Hierarchical and Preference-Aware Generative Recommendations
Show answer and example
Answer: A recent window retains events by time. Preference-guided sparse selection picks events for the current intent, letting older relevant history compete for a limited budget. It depends on the selector: downstream layers cannot read omitted events, and hierarchical pretraining does not guarantee correct selection. The public proxy gains in the source apply only to its protocol.
Illustrative example: Illustration: If the latest browsing was for novels but the current candidate is a camera, the selector may find older photography history that the last ten events omit. A bad selector could also revive an outdated interest.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How should HPGR keep padding and future targets out of its top-k attention?
Beyond the Flat Sequence: Hierarchical and Preference-Aware Generative Recommendations
Show answer and example
Answer: Score only eligible historical events, mask padding before top-k, and enforce the current-time cutoff. If there are fewer than k events, use the valid count instead of duplicating dummy records. Align session boundaries with the preference query; do not use the gold next item to create a condition unavailable at test time.
Illustrative example: Illustration: With three valid events and k=5, attention can read only those three. Unmasked zero-scored padding can beat real events with negative scores.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Do HSTU history tokens interact, unlike a static history vector?
Show answer and example
Answer: Yes. Each position updates by reading visible earlier tokens, producing contextual history that a target later reads. Causality restricts direction, not history interaction; outside-window events remain absent.
Illustrative example: A lens event can incorporate earlier camera context before a tripod candidate reads it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why do ranking and retrieval predict different objects in HSTU?
Show answer and example
Answer: Retrieval predicts a relevant next item. Ranking conditions on the candidate to predict actions, interleaving item/action tokens. Full-catalog next-item metrics do not validate every action task.
Illustrative example: Guess a film versus predict clicks or long watch for an already given film.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why remove softmax but normalize after aggregation and apply a U gate?
Show answer and example
Answer: Softmax fixes weight mass and can suppress behavior-frequency intensity. Pointwise SiLU aggregation preserves variation; post-normalization stabilizes and the token-conditioned U gate modulates output. Causal window interactions are not linear complexity.
Illustrative example: One versus ten related events can differ before the gate controls how much information passes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Why not interpret local matched-budget SASRec losses as industrial superiority?
Show answer and example
Answer: Matched width, layers, loss, and steps isolate a local core. MovieLens lacks heterogeneous events, timing, long histories, and scale. Preserve local negatives without dismissing paper production evidence or importing it.
Illustrative example: A short-film-sequence loss neither predicts 100,000-event failure nor becomes a local win from online results.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
What does M-FALCON share, and why not cache every ranking score?
Show answer and example
Answer: Section 3.4/Appendix H isolate candidate microbatches and share per-layer candidate-independent history KV. Each candidate still projects and interacts separately. Versions and history constrain reuse; local HSTU omits this serving optimization.
Illustrative example: A thousand items can share history while cameras and shoes retain different scores.
Sections in the Chinese source: 背景与主要改动, 本地复现. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How does all-position HSTU training preserve causality?
Show answer and example
Answer: Each valid state reads its prefix and predicts the next item; mask padding and invalid targets. One encoding supplies multiple losses. Candidate action interleaving is a separate contract.
Illustrative example: A predicts B and A/B predicts C without A seeing B or C.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why initialize HyFormer's Global Queries from non-sequence fields and the sequence mean?
HyFormer: Revisiting the Roles of Sequence Modeling and Feature Interaction in CTR Prediction
Show answer and example
Answer: The query needs the current global context and a history overview so it can read the long sequence purposefully instead of starting from an unconditional slot. Each decoding step is followed by boosting to combine fields with history. The initial summary can lose details; later reads can help but cannot guarantee complete recovery.
Illustrative example: Illustration: A travel setting and a photography-heavy history can steer the initial query toward lightweight camera needs. A constant query sees the current setting only in a later layer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does HyFormer apply Query Boosting after every history read and pass the queries back to the next layer?
HyFormer: Revisiting the Roles of Sequence Modeling and Feature Interaction in CTR Prediction
Show answer and example
Answer: If history is compressed once before field interactions, those interactions cannot change what evidence is read next. HyFormer alternates sequence reads and non-sequence token mixing, letting new evidence enter field interactions and boosted queries return to history. The compact queries and field representations co-evolve; this does not require rewriting the raw history for every candidate.
Illustrative example: Illustration: The first layer finds an outdoor interest. After mixing in rainy weather and item material, the next query can focus on waterproof gear. A one-shot summary followed by an MLP cannot easily ask that new question of the history.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
When do interactions occur in HyFormer's query co-evolution versus LONGER followed by RankMixer?
HyFormer: Revisiting the Roles of Sequence Modeling and Feature Interaction in CTR Prediction
Show answer and example
Answer: The serial path summarizes the sequence before field mixing, so the initial history read cannot react to later field results. HyFormer sends boosted queries back for another read, conditioning it over several layers. Earlier interaction adds expressiveness but may also strengthen popularity bias: the source reports a large local increase in head share, not unequivocally better personalization.
Illustrative example: Illustration: One round finds camping signals, field mixing adds a budget constraint, and the next round seeks lightweight gear. A completed one-shot summary generally cannot reread raw history under the new constraint.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why must HyFormer feed the boosted query back into the next cross-attention layer?
HyFormer: Revisiting the Roles of Sequence Modeling and Feature Interaction in CTR Prediction
Show answer and example
Answer: Reusing the original Q would make each layer repeat the same question and eliminate co-evolution. Sequence K/V, decoded queries, and non-sequence tokens also need aligned layouts; per-token transforms operate on the mixed layout. The local two-query reduction is a specific implementation, not an exact match for production parameters and cache paths.
Illustrative example: Illustration: The second query should carry the first layer's budget and use-case information. If code accidentally reuses Q0, a deeper model can still reduce loss without implementing the paper's iterative information loop.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why are when, where, how, and via query roles rather than just labels in IntHQ?
Show answer and example
Answer: Queries extract different evidence from behavior and context streams, affecting intermediate representations instead of merely producing four final predictions. The two streams preserve distinct sources; multilayer interactions connect time, location, manner, and channel. The names suggest specialization, which still needs to be checked.
Illustrative example: Illustration: When someone returns depends more on intervals, whereas where they go depends more on location history. Two heads attached to the same fixed vector cannot separately guide the early read.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
What does IntHQ gain by using when, where, how, and via queries within multiple encoder layers rather than only four output heads?
Show answer and example
Answer: With heads only at the end, the encoder may compress every kind of evidence the same way. Task queries read separate behavior and context streams at multiple layers, so the task can guide evidence selection: time can favor periodicity and channel can favor entry points. Shared lower layers can capture common structure, but task conflict does not disappear automatically; verify that the queries really specialize.
Illustrative example: Illustration: In the same lunchtime search event, the time query cares about a lunch-break pattern and the channel query about search as the entry point. Letting both ask inside the encoder is more flexible than giving each a head on one summary.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What structure does IntHQ's two-stream interaction preserve compared with concatenating behavior and context once?
Show answer and example
Answer: The streams distinguish events from context before task queries read across layers, avoiding undifferentiated mixing from the outset. One concatenation is simpler, but it is harder to track source and task conditions explicitly. Stronger structure does not guarantee results: the local public proxy regressed, and query names cannot replace evaluation.
Illustrative example: Illustration: A weekday lunch hour is context; recent restaurant clicks are behavior. The two streams can encode them separately before answering when and where, without forcing both to affect every target equally.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why must IntHQ keep the task axis separate from the history axis in multilayer query aggregation?
Show answer and example
Answer: Each task query reads the relevant stream or layer's K/V and forms a task state. Softmax belongs on the candidate evidence axis. Normalizing over the four tasks as if they were historical records would make tasks compete. Role alignment matters more than matching final dimensions, and the public proxy does not cover all real spatiotemporal tasks.
Illustrative example: Illustration: The when query should distribute weight over historical times, not share a sum of one with the where query; otherwise adding a location task changes the time task's evidence budget.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why encode a guest journey rather than a set of viewed listings in JourneyFormer?
JourneyFormer: Encoding Airbnb Guest Journey with Sequence Modeling
Show answer and example
Answer: Searches, views, saves, and bookings mark different stages, and elapsed time affects whether a need still applies. Type and time embeddings let the same property mean different things in different paths and jointly represent long- and short-term behavior. Event labels are not causal explanations, and a completed past trip need not match current demand.
Illustrative example: Illustration: A seaside listing booked three months ago and an urban apartment saved today should not count equally as unmet needs. A set of listing IDs loses that distinction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does JourneyFormer encode event types and time in the guest journey rather than just listing viewed properties?
JourneyFormer: Encoding Airbnb Guest Journey with Sequence Modeling
Show answer and example
Answer: The same property means different things during search, viewing, saving, and booking. Time gaps distinguish one trip from older plans. A shared journey sequence learns relations between stages while retaining both long-term preferences and immediate travel needs. Caching and sparsity reduce cost but do not relax which events were visible at request time.
Illustrative example: Illustration: A user saved a seaside listing last year but now searches downtown and views business apartments. Listing IDs alone do not distinguish the trips; event type and timing can reveal the current one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What does a unified long and short guest journey retain, and what does it cost compared with a recent window?
JourneyFormer: Encoding Airbnb Guest Journey with Sequence Modeling
Show answer and example
Answer: Long history preserves stable destinations and accommodation preferences, while the short window shows the current trip stage. Joint encoding can connect them but adds attention and update costs, motivating caches and sparsity. Older trips can be stale, and more history need not be more accurate. The source's local negative result applies only to its proxy.
Illustrative example: Illustration: A family-room preference may remain relevant during a business trip, but the new context also matters. The newest events alone can forget the stable need, while equal weighting of every old trip can distract.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why must listing, event type, and time use one version in JourneyFormer's incremental cache?
JourneyFormer: Encoding Airbnb Guest Journey with Sequence Modeling
Show answer and example
Answer: All three determine state, so changing an event type or time encoding makes the old cache inconsistent with new inputs. A causal sequence can contain only events already observed, not future bookings. The local proxy lacks real lodging inventory and full production refreshes, so its latency does not establish online performance.
Illustrative example: Illustration: If a view is corrected to a booking, a stale cache will still treat the user as exploring and may keep recommending. Refreshing the listing vector alone will not fix the event state.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why does Kunlun put a sequence block and an interaction block in the same layer?
Show answer and example
Answer: The sequence block extracts behavioral dependencies and the interaction block combines heterogeneous fields. Global Interaction passes information between layers to prevent isolated towers from hitting a communication bottleneck. HSP summarizes sequence seeds, while system parallelism and CompSkip address deep scaling. Enlarging one FFN does not reproduce these separate mechanisms.
Illustrative example: Illustration: History reveals camping intent, and a budget field in the interaction block changes its interpretation and later fusion. Towers concatenated only at the end see each other's clues much later.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why interleave sequence and interaction blocks in Kunlun rather than making only one tower larger?
Show answer and example
Answer: The sequence model handles historical dependencies and the heterogeneous interaction model handles field relationships. Scaling one side alone can leave the other as an information bottleneck. Kunlun exchanges sequence seeds and non-sequence information between layers, uses gating and residuals to control deep updates, and relies on parallel engineering to scale. Joint structure does not make every module free to reuse: distinguish candidate dependence from serving cost.
Illustrative example: Illustration: A long history reveals outdoor interest, but price and context interactions arriving only at the end cannot guide deeper interpretation. Interleaving lets them cooperate across layers instead of concatenating once.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Where does Kunlun's gated GDPA affect evidence compared with ordinary softmax attention?
Show answer and example
Answer: The gate's log-sigmoid is added to attention logits, suppressing relations before normalization rather than scaling the final vector afterward. It can suppress noise or rare important events. CompSkip separately changes residual depth.
Illustrative example: Illustration: Downweighting an accidental click before normalization reallocates attention to other events; multiplying the whole user vector afterward cannot reproduce that change.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why must Kunlun's CompSkip align the dimensions of the old state and F output?
Show answer and example
Answer: h(l+1)=αhl+(1−α)Fl(hl) combines states in the same representation space. Specify the range and broadcast axis of α rather than gating across candidates or batch examples accidentally. Respect synchronized sequence and interaction updates. The four-layer local compact core's quality variation is not production throughput evidence.
Illustrative example: Illustration: A per-layer α must not accidentally become candidate-dependent and contaminate a shared state merely because broadcasting succeeds.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why distinguish transferable generative pretraining from the ranking residual adapter in LazFormer?
Show answer and example
Answer: Pretraining learns general sequence and semantic structure. Ranking needs candidate value and serving objectives; the residual adds a task-specific update without rewriting the base. Dense recent and sparse distant inputs and sparse attention separately control history cost. Verify training and serving mechanisms individually.
Illustrative example: Illustration: Pretraining learns a camera-to-lens progression; adaptation adds the current candidate's price, a business trade-off not necessarily learned by transfer alone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why keep a ranking residual adapter alongside LazFormer's transferable pretrained backbone?
Show answer and example
Answer: Generative pretraining learns broad sequence patterns, but industrial ranking needs candidate interaction and frequent updates. A residual adds ranking-specific behavior without wholesale overwrite; dense recent and sparse distant tokens with sparse attention control long-history cost. Local deterministic mapping and adapter composition do not reproduce full generative pretraining.
Illustrative example: Illustration: The backbone knows long-term consumption, while a promotion needs extra price interactions. A fixed weighted scorer cannot prove the original training pipeline ran.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What long-range information do dense recent and sparse distant tokens retain versus fixed recent truncation?
Show answer and example
Answer: Truncation drops old history entirely. The mixed representation keeps coarse older evidence and detailed recent events, and sparse connections lower pairwise cost. Compression can lose rare events. The deterministic mapping and residual in the local version are interfaces, not original long-sequence pretraining.
Illustrative example: Illustration: Keep every click from the last two days but summarize last year's history by groups. This remembers more than ten recent clicks, yet an important old exception may be lost.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why must the LazFormer local implementation not report the paper's two-week A/B numbers as its own training results?
Show answer and example
Answer: The local version has deterministic candidate scoring, transferable mapping, and a ranking residual with fusion selected on validation. It lacks full large-model training and online serving. Report the negative public-protocol result and separately cite the paper's business gains.
Illustrative example: Illustration: A residual in local movie scores does not imply generative pretraining; copying the paper's GMV percentages into local metrics would mix evidence sources.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why does LinkedIn Feed combine candidates, interaction history, and context into a sequence?
An Industrial-Scale Sequential Recommender for LinkedIn Feed Ranking
Show answer and example
Answer: They jointly determine current content value, so a common backbone lets evidence interact before the final stage. Very long history retains career interests but adds compute and candidate dependence, motivating incremental serving. A unified representation does not make all states reusable across candidates.
Illustrative example: Illustration: Recent job seeking, a long-running interest in engineering, and a current job-listing candidate together explain value better than a set of liked articles.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does LinkedIn Feed's sequential model stress evaluation without random negative sampling?
An Industrial-Scale Sequential Recommender for LinkedIn Feed Ranking
Show answer and example
Answer: A few easy random negatives simplify ranking and conceal hard distinctions in a large pool. A model that jointly processes candidates, history, and context needs a consistent candidate protocol to test scaling and incremental serving. A local proxy is not LinkedIn's production distribution, and gains cannot be compared across different negative counts.
Illustrative example: Illustration: Finding one positive among nine obviously irrelevant items differs from finding it among thousands of similar Feed candidates, even with the same model.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Why is evaluation without random negatives harder than testing one positive and a few random negatives?
An Industrial-Scale Sequential Recommender for LinkedIn Feed Ranking
Show answer and example
Answer: Sampled sets often omit real competitors. Metrics vary with negative count and difficulty, potentially overstating ranking ability. A fuller pool better represents serving competition but costs more. Evaluation without random negatives does not imply no training negatives or justify comparing percentages across protocols.
Illustrative example: Illustration: Ranking among 99 irrelevant articles is easier than beating many similar articles in the full professional corpus. The metric can be named NDCG in both without measuring the same task.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why validate candidate-independent history state before incremental serving in LinkedIn Feed?
An Industrial-Scale Sequential Recommender for LinkedIn Feed Ranking
Show answer and example
Answer: A reusable prefix must not depend on the candidate being swapped. History and model changes also invalidate caches. Candidate attention may read shared state but must not reuse candidate-conditioned history for other candidates. The local core proxy lacks the full LinkedIn system and does not measure true long-sequence throughput.
Illustrative example: Illustration: A new article may read old history K/V through a different query. If the old article was written into that state, sharing it with the new one leaks candidate information despite fast execution.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why scale LLaTTE's async long history separately from the online short sequence?
LLaTTE: Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendation
Show answer and example
Answer: The upstream model processes more history and caches an overview. The downstream model uses fresh events and candidate-aware interactions without rerunning the big model on each request. Each stage has a bottleneck, so measure compression and transfer efficiency rather than assuming all upstream gains reach serving.
Illustrative example: Illustration: A cached profile retains years of interests, while recent baby-product browsing can change ranking immediately instead of waiting for the next refresh.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does LLaTTE still read recent history with candidate-aware attention after caching LLM user representations?
LLaTTE: Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendation
Show answer and example
Answer: Async long-history encoding supports more compute but refreshes periodically and compresses details. The online short sequence adds recent behavior, and the candidate query picks relevant evidence. Stable knowledge and real-time relevance are distinct responsibilities. Measure transfer of upstream gains downstream instead of assuming all of them survive.
Illustrative example: Illustration: An overnight cache remembers photography, but a new backpack search signals travel. Online history adds the new intent and lets the current candidate read relevant knowledge.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Why isn't LLaTTE's transfer efficiency τ just the upstream NE improvement?
LLaTTE: Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendation
Show answer and example
Answer: A better upstream model need not yield proportional downstream gains: cache size, fusion capacity, and task objectives can discard information. τ compares the two improvements to help decide where to spend compute. A tiny denominator or mismatched protocols makes the ratio unstable; an empirical trend is not a theorem.
Illustrative example: Illustration: If upstream NE improves and downstream NE barely changes, candidate-aware fusion may be more useful than adding upstream parameters.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why can't LLaTTE's local four latent queries and recent-12 path establish the value of a 2,000-dimensional production cache?
LLaTTE: Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendation
Show answer and example
Answer: The small setup trains long summaries, short history, and gating but does not fully measure production refreshes, roughly 400 online events, or multisource features. Verify gradient flow and candidate dependence, then report the matched short baseline and head exposure. Size differences cannot become measured QPS or conversion gains.
Illustrative example: Illustration: Four summary vectors reading 24 movie events test information flow, not the latency of a production cache over tens of thousands of multisource events.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why pair the offline Long-History Transformer cache with a recent online path?
Long-History User Transformers for Real-Time Ad Ranking
Show answer and example
Answer: Async long history retains stable interests but lags. A small online model reads recent changes and the cache, combining coverage with responsiveness. Compression can lose details, and auxiliary pretraining and ad fine-tuning are distinct stages. Concatenating a long vector does not reproduce all of this.
Illustrative example: Illustration: Baby-product browsing can change a prediction immediately even while the background cache still emphasizes photography.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why keep recent events in a lightweight online Transformer when a long-history cache already exists?
Long-History User Transformers for Real-Time Ad Ranking
Show answer and example
Answer: The cache covers stable patterns but refresh lag excludes recent events. A short online window plus cached state responds without re-encoding all history. Auxiliary objectives support both representations. A small cache can lose information; do not mix fresh offline states with stale online caches in evaluation.
Illustrative example: Illustration: Overnight history records photography, but today's sudden childcare searches need a recent path without waiting for a refresh or re-encoding all past events.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What costs move to the background in a two-stage async model compared with request-time full history encoding?
Long-History User Transformers for Real-Time Ad Ranking
Show answer and example
Answer: The background encodes long history and caches fixed-width state, avoiding repeated long self-attention at serving time. Refresh, storage, invalidation, and compression costs remain. The source lacks actual Yandex feature-store measurements, so structure alone cannot establish latency gains.
Illustrative example: Illustration: Requests can share a daily summary, but if updates are frequent and requests rare, background compute may not pay off.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How should the local 24-step offline, four-step online model enforce input cutoffs?
Long-History User Transformers for Real-Time Ad Ranking
Show answer and example
Answer: Both branches use only events before the prediction. Define overlap handling and never build a cache from the full sequence including test events. Next-item and feedback auxiliary labels stay on the training side. A 32-dimensional cache is a scale-down, not the original history length or feature scale.
Illustrative example: Illustration: A cache containing the final film leaks it even if the online branch reads only four earlier events.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why isn't a LONGER global token a fixed user summary?
LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders
Show answer and example
Answer: Globals can include CLS, user, and candidate fields. Short queries propagate their first-layer states; candidate globals remain request-specific. Cacheability depends on actual dependency.
Illustrative example: A user ID can be shared but a price-derived global belongs to one item.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Do later LONGER layers keep updating long history after using it as KV?
LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders
Show answer and example
Answer: The first layer uses short global/recent Q against merged long KV. Output follows Q length; later self-attention runs only on the short states. This is inter-layer compression, not upfront truncation. The local namesake lacks full fidelity.
Illustrative example: Short query cards inspect a long reference once, then only the cards are refined.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Where does LONGER compress compared with candidate top-k history?
LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders
Show answer and example
Answer: Top-k discards events before fine encoding. LONGER lets short queries read merged long KV once, then propagates short output. Merging is still lossy and the local first-layer path is not fully faithful.
Illustrative example: Read a broad compressed source once rather than discard all but ten pages up front.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why can larger LONGER merge factor K fail to reduce total FLOPs?
LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders
Show answer and example
Answer: Section 3.4 shortens positions while widening merged features, increasing projection cost but reducing attention cost. InnerTrans supplies within-group interaction. This is not fixed-width mean pooling or monotone quality.
Illustrative example: Moving from four to eight events per wider token can make matrix work rebound.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How cache user KV if some global tokens contain candidates?
LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders
Show answer and example
Answer: Sections 3.5.2/3.6.3 mask candidate-specific globals from user sequences. Cache independent user KV and compute candidate query outputs separately. Global means aggregation role, not independence.
Illustrative example: Two products share behavior KV but use distinct globals; do not cache a user state already conditioned on product A for B.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why verify first-layer Q length and subsequent state length?
LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders
Show answer and example
Answer: The defining path is short global/recent Q over long KV, then short-only self-attention. Concatenating every input and self-attending each layer changes information flow. Inspect actual shapes and masks.
Illustrative example: With KV length 500 and Q length 100, the next layer should receive 100 positions, not over 600.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why turn scenario and task into tokens in MDL?
Show answer and example
Answer: They determine which field relations matter, not only which head to use. Domain-feature attention lets scenario queries read features deep in the model, and task tokens and aggregation form specific outputs. Token roles permit interaction but do not ensure intended semantics emerge.
Illustrative example: Illustration: A user's features matter differently for live shopping and short-video viewing. A scenario token can adjust intermediate reads, not just swap the final classifier.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why tokenize scenario and task in MDL rather than just choosing output heads?
Show answer and example
Answer: Late heads cannot continuously steer feature selection. Tokens let domain queries read relevant fields across layers and task information shape shared representations. Deeper sharing may cause interference; check whether low-traffic settings are overridden by larger ones instead of assuming a unified design helps all tasks.
Illustrative example: Illustration: Home-page clicks and search conversion may need different product evidence. Scenario and task tokens can change early reads instead of splitting one summary at the end.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does MDL's deep domain-feature interaction condition more than a shallow gate?
Show answer and example
Answer: A shallow gate adjusts already encoded features. Deep interaction repeatedly conditions representation formation, expanding joint modeling at higher compute and negative-transfer risk. Local accuracy improved with a clear rise in head exposure, not evidence that every group benefited.
Illustrative example: Illustration: The scenario can emphasize price first and combine it with budget next. A shallow gate can only reweight previously formed interactions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why label feature, scenario, and task token identities in MDL?
Show answer and example
Answer: Equal final dimensions do not make evidence, conditions, and predictions interchangeable. Keep attention layout and readout positions aligned. Treating scenario count as batch size or reading the wrong task position silently changes the objective. The local 19 scenarios are proxies, not 19 production businesses.
Illustrative example: Illustration: A purchase token cannot supervise viewing just because both are two-dimensional; verify identity mapping as well as shape.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why are Representation Memento and Data Memento different memory stores?
Memento: Personalized RAG-Style Long-Retention Data Scaling for Online Ads Recommendation
Show answer and example
Answer: Representation memory retrieves long-term user evidence for a current prediction. Data memory finds forgotten examples related to the current batch and updates parameters again. Storage, updating, and evaluation differ, so implementing either does not complete both loops.
Illustrative example: Illustration: Reading old photography history today is representation memory. Replaying training examples to prevent forgetting a user segment is data memory.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
What types of forgetting do Memento's representation and data-replay paths address?
Memento: Personalized RAG-Style Long-Retention Data Scaling for Online Ads Recommendation
Show answer and example
Answer: Truncated inference history loses distant interests; the representation path retrieves relevant nonredundant events and conditions predictions. Single-use training samples can be forgotten through parameter updates; the data path replays old samples related to the current batch. Local history retrieval does not validate both paths.
Illustrative example: Illustration: Retrieve last year's photography click as input; replay old training examples to relearn a rare category. Reading an old vector is not training replay.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does query-conditioned MMR in Memento trade relevance and redundancy against LastN?
Memento: Personalized RAG-Style Long-Retention Data Scaling for Online Ads Recommendation
Show answer and example
Answer: LastN uses recency; MMR picks query-relevant events while avoiding duplicates of already selected history. It can recover distant evidence but depends on query quality, with λ balancing relevance and coverage. Local results cover retrieval, not Ember or data replay.
Illustrative example: Illustration: Ten recent lens views are redundant; MMR can keep one and retrieve an older camera-use event, or choose poorly if the query is wrong.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why can't Memento's local metrics verify Ember transformations and data replay?
Memento: Personalized RAG-Style Long-Retention Data Scaling for Online Ads Recommendation
Show answer and example
Answer: Only Representation Memento's MMR ran. Ember affine/quadratic transforms, production retrieval, and historical replay were not reproduced. Broader claims need learned conditional parameters, actual replay updates, and matched controls. Choose λ on validation, not test.
Illustrative example: Illustration: Better NDCG after recovering an old photography event supports input evidence, not claims about forgetting after SGD without replay updates.
Sections in the Chinese source: 本地复现. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why include both dense fields and behaviors in MixFormer's unified token space?
MixFormer: Co-Scaling Up Dense and Sequence in Industrial Recommenders
Show answer and example
Answer: Separate encoders postpone communication. Joint layers update both representations and scale their capacity together. Heterogeneous fields need proper projection, not treatment as literal words. Serving still separates user and candidate dependencies; not every state can be shared.
Illustrative example: Illustration: A rain field helps interpret outdoor clicks early, and the refined interest feeds back into field interactions before any final concatenation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
How does MixFormer's shared token space change information flow versus concatenating two towers at the end?
MixFormer: Co-Scaling Up Dense and Sequence in Industrial Recommenders
Show answer and example
Answer: Late concatenation finishes field interactions and sequence encoding separately. MixFormer updates both token types together across layers and scales joint modeling. Serving can still decouple reusable user computation from candidate-dependent work rather than rerunning all user history per candidate.
Illustrative example: Illustration: A rainy setting can shape early interpretation of outdoor browsing and reinforce waterproof needs across layers, while late fusion sees two completed summaries.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Why does a parameter-matched stacked baseline isolate MixFormer's architecture better than the old fixed semantic gate?
MixFormer: Co-Scaling Up Dense and Sequence in Industrial Recommenders
Show answer and example
Answer: Actually train stacked and unified models with similar budgets to compare interaction structure. A fixed score gate only proxies fusion. Matching parameters does not remove optimization, feature, or variance differences. The source retired the old proxy; do not mix its numbers with the new experiment or claim stable industrial gains.
Illustrative example: Illustration: Comparing two trained architectures is meaningful; adding a semantic constant to existing scores tests a rule, not unified Transformer training.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why check whether candidates enter the shared backbone before reusing one MixFormer user encoding for all dot products?
MixFormer: Co-Scaling Up Dense and Sequence in Industrial Recommenders
Show answer and example
Answer: If a candidate modifies the unified user state, swapping it invalidates reuse. The local model jointly encodes user fields and history first, then scores separate candidate vectors. Verify actual dependencies, not naming. Public genres replacing private fields also limit fidelity.
Illustrative example: Illustration: Encode four user fields and 32 events once and dot-product with shoes or a camera. If the camera went into the backbone, reusing that state for shoes would change the path.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why mask the cross input rather than the final score in ML-DCN?
Show answer and example
Answer: Selecting channels before multiplication changes which interactions form. Scaling a final score cannot undo bad crosses. Low rank separately controls internal width. Learned mask values are not causal importance and do not prove low-weight features can be removed everywhere.
Illustrative example: Illustration: Suppressing noisy price before interaction prevents bad combinations; scaling the final prediction suppresses useful signals too.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why apply both low-rank projection and a learned mask before ML-DCN's explicit crosses?
Show answer and example
Answer: Full-width crosses increase compute with capacity. Low rank maps to a configurable small dimension and back; the mask selects participating channels. These control cost and information separately. Residuals preserve the original state, but a low-rank bottleneck can still miss relationships outside its subspace.
Illustrative example: Illustration: Rather than fully crossing thousands of age, city, and device features, select useful channels and combine them in tens of dimensions, leaving budget for depth.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How do rank and channel selection trade expressiveness against DCN-V2's full cross matrix?
Show answer and example
Answer: A full matrix expresses more general transformations. Low rank constrains the subspace while masks select channels adaptively and save capacity budget. More layers need not restore omitted relations. The shared local scorer proxy is not a full parameter-matched DCN-V2 control.
Illustrative example: Illustration: Compressing a thousand dimensions to dozens saves compute but can lose rare geography–category relationships that masks cannot recreate.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
What shape relationships should ML-DCN's V, U, mask, and x0 satisfy?
Show answer and example
Answer: Mask the current xl channelwise, map with V to the internal rank, use U to map back, multiply by original x0, then add residual and bias. Do not replace x0 with xl each layer or broadcast the mask across the batch axis incorrectly. Verify gradient flow instead of using fixed selection.
Illustrative example: Illustration: Rank 32 does not make the output 32-dimensional; it must project back before the elementwise x0 product.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why project MLCC's compact-space crosses back into a high-dimensional backbone?
Show answer and example
Answer: Compact z saves interaction cost, while the residual backbone retains old information. Projection P injects new combinations rather than trapping all layers in one reduced space. Compression may omit directions, and residuals do not guarantee learning every high-order cross.
Illustrative example: Illustration: Compute budget–category interaction in a small space and add it back, leaving other fields available at the next layer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why compress before every explicit cross in MLCC instead of compressing only the initial input?
Show answer and example
Answer: Layer states contain different interaction orders. Recompressing the current state lets the next cross use newly formed information. Dynamic crosses in compact space project back into the backbone to reduce high-dimensional cost. Parallel subspaces can cover different relations but need to retain useful directions.
Illustrative example: Illustration: First combine region and category, then compress that combination to cross it with price preference. Reusing only initial fields misses the new relationship.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How do MC-MLCC's parallel subspaces differ from widening one compression space?
Show answer and example
Answer: Separate subspaces can specialize in distinct crosses and combine outputs; one wider projection concentrates capacity. Parallel channels can specialize or duplicate each other. Match parameters and FLOPs; the mechanism does not establish universal cost and accuracy gains.
Illustrative example: Illustration: Price–budget and time–context can occupy different channels, but if both learn popularity, capacity is redundant.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why must MLCC's Cl and Pl match the corresponding layer state?
Show answer and example
Answer: Compress current xl to z, compute its dynamic gate from z, and project back to xl's width. Reusing the initial compression or the wrong layer's weights weakens progressive high-order updates. Check that each layer actually receives the updated state, not just valid residual shapes.
Illustrative example: Illustration: Layer two should compress a state that includes layer one's crosses; compressing x0 again can just repeat the first cross.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why put capacity in MSN memory slots instead of a fully dense network?
MSN: A Memory-based Sparse Activation Scaling Framework for Large-scale Industrial Recommendation
Show answer and example
Answer: Dense models typically activate all parameters. Memory stores more values but reads few slots, so compute need not grow proportionally. Product keys organize addresses across two axes and a gate fuses values. Sparse reads do not remove maintenance cost or guarantee useful knowledge.
Illustrative example: Illustration: Retrieving four files is cheaper than reading an archive, provided the routing finds the right files.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
How does two-axis Product-Key Memory in MSN add capacity while reading little knowledge per query?
MSN: A Memory-based Sparse Activation Scaling Framework for Large-scale Industrial Recommendation
Show answer and example
Answer: A slot combines one key from each axis. Summed similarities rank combinations, only a few values are read, and a gate fuses them into the dense backbone. Stored capacity does not all become active compute, and useful routing still matters.
Illustrative example: Illustration: Two sets of 100 keys address 10,000 slots, but one request reads the four most relevant ones.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does top-k PKM activation differ from a dense soft mixture over every expert?
MSN: A Memory-based Sparse Activation Scaling Framework for Large-scale Industrial Recommendation
Show answer and example
Answer: A dense mixture usually evaluates all experts before weighting. PKM finds high-scoring key pairs and reads selected values, leaving most slots inactive. Search and memory access still cost time, and hard selection can miss evidence. Total-to-active parameter ratio does not predict measured latency.
Illustrative example: Illustration: Reading four of 10,000 slots differs from running 100 experts, but scattered reads can offset lower FLOPs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why preserve unique slot IDs and deduplicate top-k two-axis combinations in MSN?
MSN: A Memory-based Sparse Activation Scaling Framework for Large-scale Industrial Recommendation
Show answer and example
Answer: Both axis addresses identify the slot; the summed similarity is a score, not an ID. Duplicate slots overweight one value. Match value width to the backbone and define short or empty selections. Local genre and transition writes are public proxies.
Illustrative example: Illustration: (2,3) and (3,2) are different slots despite the same index sum, and a duplicate (2,3) is not independent knowledge.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why add NONTP auxiliary modules only during training?
Not Only NTP: Extending Training Signal Coverage for Generative Recommendation
Show answer and example
Answer: TCL and TDL teach long-range and cross-domain signals to the backbone without future teachers or extra branches at serving time. Richer supervision is not uniformly better, especially with weak domain labels and short histories. Future targets are not available online.
Illustrative example: Illustration: Train for a trajectory several steps ahead but serve with the current history and original item head.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Which two kinds of locality do TCL and TDL address in NONTP next-token prediction?
Not Only NTP: Extending Training Signal Coverage for Generative Recommendation
Show answer and example
Answer: TCL predicts a later EMA-teacher state, extending immediate next-step supervision. TDL predicts the target through a shared item head from earlier cross-domain states, helping sparse domains. Both are training-only; inference stays on the original backbone and cannot read future states.
Illustrative example: Illustration: Before a lamp purchase, TCL teaches longer trajectories and TDL uses preceding home-content behavior. Serving sees only events already observed.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How is TCL's EMA teacher more stable than a direct online future-state target?
Not Only NTP: Extending Training Signal Coverage for Generative Recommendation
Show answer and example
Answer: EMA smooths updates and gives a stopped-gradient target rather than two sides chasing each other. Future state remains training-only; use other users as negatives rather than mislabeling nearby same-user interests. Stability does not ensure correctness and can preserve old biases.
Illustrative example: Illustration: A slow teacher changes less than a recent online update, but a consistently popularity-biased teacher is still biased.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why must TDL use pre-target cross-domain history and the same item head?
Not Only NTP: Extending Training Signal Coverage for Generative Recommendation
Show answer and example
Answer: Time order prevents leakage, domain mismatch supplies cross-business transfer, and averaging into the shared NTP head trains the target item space directly. Skip empty groups rather than inventing labels; an independent head is not the same transfer interface.
Illustrative example: Illustration: Predict a shopping target from earlier other-domain states, not videos afterward. No cross-domain history means no TDL evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why map heterogeneous scenario fields into uniform event tokens in OneModel?
OneModel: A Unified Foundation for Platform-Scale Multi-Scenario Ranking
Show answer and example
Answer: Different businesses log different fields and actions. Scene projections establish a common width and attach scene, action, time, and position before the causal backbone. Shared width does not equal shared meaning; gates and hierarchical states preserve differences.
Illustrative example: Illustration: A ten-second view and a purchase should not be two untyped IDs. Event metadata tells the model what each action means.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
What specialization does SAIM retain after OneModel shares a causal backbone?
OneModel: A Unified Foundation for Platform-Scale Multi-Scenario Ranking
Show answer and example
Answer: Scenario projections map heterogeneous events into one history space, but sharing every FFN activation can flatten differences. SAIM conditions shared FFN channels on scene embeddings. Global and recent states separately preserve stable and immediate signals.
Illustrative example: Illustration: Travel browsing and purchases can share history, while ads and content use different gates and recent flight booking remains in the local state.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What time scales does OneModel global/local fusion preserve beyond a pooled user state?
OneModel: A Unified Foundation for Platform-Scale Multi-Scenario Ranking
Show answer and example
Answer: Global attentive pooling summarizes stable patterns, while recent states retain rapid changes. A single pooled vector may average away new needs. Genre-derived scenes lower density in the proxy; report the negative result without extrapolating it to real cross-business logs.
Illustrative example: Illustration: Long-term travel and a new renovation need can coexist without years of travel events dominating a single average.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
What should generate OneModel's SAIM gate, and why not use outcomes after prediction?
OneModel: A Unified Foundation for Platform-Scale Multi-Scenario Ranking
Show answer and example
Answer: Use the scenario embedding known at request time to modulate the shared FFN. Outcomes supervise only. Maintain causal masking and correct domain projections. Local genre-derived scenes are proxies, not real recommendation, ad, and merchant logs.
Illustrative example: Illustration: An ad-request flag is available now; selecting a gate because the user later purchased leaks future information.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why does OnePiece jointly cover retrieval, clicks, and value?
OnePiece: Bringing Context Engineering and Reasoning to Industrial Cascade Ranking
Show answer and example
Answer: Cascade stages have distinct labels and independent training can lose transferable clues. Shared context and latent updates model multiple needs, while progressive weights schedule learning. Objectives can conflict; a 'reasoning' name does not verify language reasoning.
Illustrative example: Illustration: Relatedness, click likelihood, and expected value complement one another, but high clicks need not imply purchases.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
How do OnePiece's latent blocks and progressive task weights differ from simply summing all losses?
OnePiece: Bringing Context Engineering and Reasoning to Industrial Cascade Ranking
Show answer and example
Answer: Residual blocks repeatedly update contextual state before heads read it. Scheduled weights change which objectives dominate training stages so shared representations adapt to retrieval, click, and value. Both path and order matter; a local proxy does not prove verifiable linguistic reasoning in latent states.
Illustrative example: Illustration: Learn relevance before raising purchase-value weight to avoid an early sparse objective dominating basic ranking.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does progressive weighting change learning order relative to fixed equal task weights?
OnePiece: Bringing Context Engineering and Reasoning to Industrial Cascade Ranking
Show answer and example
Answer: Fixed weights exert the same pressure throughout. A schedule lets basic relevance and harder value objectives converge in stages. This can reduce conflicts or cause forgetting, so inspect stage metrics. Proxy results do not make all curricula superior.
Illustrative example: Illustration: Emphasize relevance early and transaction value later; forgetting relevance could yield expensive but unrelated items.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How verify OnePiece's H(k+1) residual update and αt(k) schedule separately?
OnePiece: Bringing Context Engineering and Reasoning to Industrial Cascade Ranking
Show answer and example
Answer: One updates contextual state per block, the other weights losses per training stage. Distinguish the two meanings of k and verify both state propagation and applied loss weights. A fixed scorer proxy cannot establish full cascade training.
Illustrative example: Illustration: More blocks do not imply progressive weights, and changing losses does not establish multi-step latent updates.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why put OneRank task tokens inside the encoder instead of using them only to select output heads?
OneRank: Unified Transformer-Native Ranking Architecture for Multi-Task Recommendation
Show answer and example
Answer: Conditioning inside layers changes how each task interprets fields before prediction. Private channels retain specialization, so unified inputs do not mean complete sharing. The presence of a token does not prove it is used.
Illustrative example: Illustration: Purchase can read budget early and clicks can read freshness; a late head gets one already compressed representation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why retain task-private channels and isolate their gradients inside OneRank's unified sequence?
OneRank: Unified Transformer-Native Ranking Architecture for Multi-Task Recommendation
Show answer and example
Answer: Task tokens condition shared features but cannot prevent conflicting losses from competing for shared parameters. Private channels add dedicated capacity that other tasks cannot update directly. Shared inputs still exchange evidence, and conflict is controlled at specific paths rather than assumed absent.
Illustrative example: Illustration: Clicks value novelty and purchases value reliability. Both share user and item history, but click loss must not overwrite purchase-private features.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What differs between OneRank's private channels and MMoE task-specific gates?
OneRank: Unified Transformer-Native Ranking Architecture for Multi-Task Recommendation
Show answer and example
Answer: A gate mixes shared experts that other tasks still update. Private channels own task-specific parameters and block other tasks' gradients. Both retain sharing, not complete independence. The public proxy does not establish that industrial negative transfer is solved.
Illustrative example: Illustration: A purchase gate changes expert weights but clicks can update those experts. Click loss should not change purchase-private parameters.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How check that OneRank loss Lj cannot update another task's φk without freezing shared θ?
OneRank: Unified Transformer-Native Ranking Architecture for Multi-Task Recommendation
Show answer and example
Answer: Backpropagate one task loss. Other private gradients must be zero; its own private and permitted shared parameters can receive gradients. Detaching the whole backbone loses joint training and detaching only outputs may leave private internal paths. Align parameter groups with tasks.
Illustrative example: Illustration: Click training should preserve purchase-private weights but can update shared fields; zeroing all shared gradients is frozen adaptation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why share S-token weights but use position-specific NS-token weights in OneTrans?
Show answer and example
Answer: Behavior events have similar roles, whereas user and item fields are heterogeneous. Joint causal interaction does not make them homogeneous. Position-specific weights cost capacity and must move with field order.
Illustrative example: Illustration: Clicks share QKV, but age and price may use separate FFNs. Swapping fields without their mapping treats price like age.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why place behavior before non-sequence fields in OneTrans and use different parameter sharing for each?
Show answer and example
Answer: Causal order lets later fields read behavior while keeping history independent of later candidates. Homogeneous events share QKV and FFN, while heterogeneous fields use position-specific parameters. Pyramid shortening saves later query work. Reuse still depends on time window, versions, and dependencies.
Illustrative example: Illustration: Reuse earlier click history when swapping a candidate; candidate fields read that history using parameters suited to price and category.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Where does OneTrans's pyramid of S queries save work compared with keeping full length in every layer?
Show answer and example
Answer: Later layers update fewer recent queries, reducing output and interactions. Older evidence can survive through earlier states or legal K/V; it is not dropped from the input or absent everywhere. The small-data head concentration is not generalization evidence.
Illustrative example: Illustration: Read broad history in shallow layers and focus on recent queries later, unlike deleting distant events first.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why do OneTrans NS order and causal masking determine visibility?
Show answer and example
Answer: S-first lets later fields read history while history cannot read later candidates, enabling shared state. Visibility between NS positions depends on the actual mask. Validate history, time encoding, and model versions before reusing cross-request caches, not just user ID.
Illustrative example: Illustration: Move a candidate before history and later events may see it, invalidating the old cache contract.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why keep three stages' own inputs and targets while sharing OneTrans-V2 user context?
Show answer and example
Answer: Retrieval, preranking, and fine ranking have different candidate counts and features. Shared causal history saves encoding, while stage-specific tokens preserve constraints and controlled losses transfer knowledge. A stage cannot read another's future labels.
Illustrative example: Illustration: Retrieval sees codes while fine ranking sees item details; shared history does not expose posteriors to retrieval.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why share user context but forbid cross-stage token attention in OneTrans-V2?
Show answer and example
Answer: Three stages can reuse costly behavior encoding while retaining distinct inputs and objectives. Each reads only prior behavior, avoiding downstream or future leakage. Fine-to-pre-ranking knowledge travels through controlled distillation, not unrestricted reads. Local work lacks independent three-stage controls.
Illustrative example: Illustration: All stages share observed browsing, but retrieval cannot see later clicks or fine-rank labels. Preranking learns from a stopped-gradient teacher.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Why aren't independent stage attention and fine-to-pre-rank distillation contradictory?
Show answer and example
Answer: Forward isolation controls serving inputs; training distillation transfers stopped-gradient soft supervision into parameters. It need not expose the fine-ranker online. Runnable stages without an independent three-stage baseline do not establish cascade gains.
Illustrative example: Illustration: Teach preranking to mimic fine ranking offline. Running fine ranking first on every request defeats ordering and cost.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why should OneTrans-V2 decision offsets directly change only decision logits?
Show answer and example
Answer: The local test offsets two observable proxy decisions. Given a decision, it should not directly rewrite SID distributions or pre-/fine-rank outputs. This tests interface independence, not unobserved purchase value or ad supply. Proxy thresholds are not CTR/CVR.
Illustrative example: Illustration: Raising the discovery offset can change decision likelihood, not add a constant to all fine-rank scores.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why combine next-, multi-, and future-token objectives in PinFM?
PinFM: Foundation Model for User Activity Sequences at a Billion-scale Visual Discovery Platform
Show answer and example
Answer: The immediate item, future window, and future from a specified position supply distinct time scales, helping learn more than recency before transfer. Future labels are training-only. Cross-scenario pretraining can negatively transfer despite being a foundation model.
Illustrative example: Illustration: Next click teaches immediate intent, a window teaches coexisting interests, and later targets add long-term direction.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
How does PinFM DCAT retain early candidate fusion without re-encoding user context per item?
PinFM: Foundation Model for User Activity Sequences at a Billion-scale Visual Discovery Platform
Show answer and example
Answer: Early fusion lets candidates interact with history internally. DCAT self-attends once over deduplicated context, caches layer K/V, and lets candidate queries read it. Shared state must be candidate-independent. Serving savings and pretraining gains are distinct; retain local negative outcomes.
Illustrative example: Illustration: Encode 500 events once for 100 candidates rather than copying all history 100 times, while candidates still read it inside layers.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Why does scratch versus pretrained DCAT answer a different question from DCAT versus ordinary Transformer?
PinFM: Foundation Model for User Activity Sequences at a Billion-scale Visual Discovery Platform
Show answer and example
Answer: Keeping DCAT fixed isolates pretraining, while changing architecture tests cache decomposition or fusion. The selected local pretrained model regressed on test; mathematical DCAT fidelity does not establish pretraining value. Measure serving efficiency separately.
Illustrative example: Illustration: Training two models with the same architecture differently says nothing about a third architecture that was never tested.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why confirm actual context K/V reuse during DCAT full-catalog inference?
PinFM: Foundation Model for User Activity Sequences at a Billion-scale Visual Discovery Platform
Show answer and example
Answer: Compute context self-attention once and let candidate layers perform only legal candidate-dependent work. Equal scores do not imply equal execution cost, so check call counts and dependencies. Bind caches to history and version and exclude candidate-conditioned states.
Illustrative example: Illustration: Recomputing history for 1,000 candidates can give the same NDCG while yielding no amortization.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
How do POEM ranks differ from raw CTR or GMV values?
POEM: Partial-Order Enhanced Real-Time Sequential Modeling for Recommendation
Show answer and example
Answer: Ranks preserve which item comes first within an objective but drop units and magnitude, allowing cross-objective comparisons. They cannot replace all value information, such as one versus a hundred dollars.
Illustrative example: Illustration: A can have slightly higher CTR but much lower GMV. The two rank directions omit 'slightly' and 'much.'
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why does POEM build a partial order from multiple objectives rather than order training sequences chronologically?
POEM: Partial-Order Enhanced Real-Time Sequential Modeling for Recommendation
Show answer and example
Answer: Event time does not measure present-request preference. POEM aggregates objective-relative candidate ranks into a partial order for supervision. Disagreement need not become a unique rank. Existing objective judgments are not independent unbiased user preferences.
Illustrative example: Illustration: An item viewed later can be preferred by click and purchase objectives. Use that relation instead of mistaking browse chronology for preference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What can POEM majority comparison lose versus retaining all conflicting evidence?
POEM: Partial-Order Enhanced Real-Time Sequential Modeling for Recommendation
Show answer and example
Answer: A majority compresses goals to one direction and can lose strongly opposed minority objectives and margins. A partial-order focus differs from representing both sides. The mechanism comparison does not prove universal performance.
Illustrative example: Illustration: Three weak votes for A beat two strong votes for B, hiding the purchase margin.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why check POEM pairwise majorities for ties or cycles rather than assume a total order?
POEM: Partial-Order Enhanced Real-Time Sequential Modeling for Recommendation
Show answer and example
Answer: A can beat B, B beat C, and C beat A, and ties have no unique direction. If a sequence is needed, apply a defined stable rule and record it. Never use test labels to choose a favorable ordering.
Illustrative example: Illustration: Breaking a cycle by item ID is an additional rule, not evidence that candidates had a consistent true order.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Which attention does RankMixer replace?
RankMixer: Scaling Up Ranking Models in Industrial Recommenders
Show answer and example
Answer: It replaces attention for heterogeneous field interactions, not necessarily independent behavior sequence encoding. TokenMix and specialized FFNs process fields after feature extraction.
Illustrative example: History attention can extract photography interest before mixing it with price and item fields.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
How do independent FFNs learn interactions after fixed TokenMix?
RankMixer: Scaling Up Ranking Models in Industrial Recommenders
Show answer and example
Answer: Split every input by head and concatenate matching-index slices into new tokens, so specialized nonlinear FFNs receive cross-field signals. H=T and divisible width are layout conditions. Field mixing does not forbid sequence attention.
Illustrative example: User, product, and context slices can jointly enter an FFN without similarity weights.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
What do ReLU routing and DTSI solve beyond fixed top-k?
RankMixer: Scaling Up Ranking Models in Industrial Recommenders
Show answer and example
Answer: Sparse ReLU permits variable activated experts. Separate training and inference routers apply sparsity only on inference so experts can learn without starvation. Local sparse code does not validate full dual routing.
Illustrative example: Complex tokens can use more experts while rarely deployed experts still train.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Why not credit TokenMix alone for the dense versus shared-FFN gain?
RankMixer: Scaling Up Ranking Models in Industrial Recommenders
Show answer and example
Answer: Both arms use TokenMix; dedicated versus shared FFNs and different parameter counts change together. The sparse arm also trails dense. Report budgets and popularity bias; local results do not establish paper MFU or A/B.
Illustrative example: Both cars already use the same transmission, so a larger engine's gain does not isolate it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why require H=T and divisible hidden dimension?
RankMixer: Scaling Up Ranking Models in Industrial Recommenders
Show answer and example
Answer: Splitting T tokens of width D into H slices gives H outputs of width TD/H. H=T restores T×D for residual addition; D must divide by H.
Illustrative example: Four 64-wide tokens split into four 16-wide slices; three heads cannot split 64 evenly.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Do RankMixer per-token FFNs attach to original fields or mixed positions?
RankMixer: Scaling Up Ranking Models in Industrial Recommenders
Show answer and example
Answer: Each FFN sees a mixed token containing same-index slices from all fields. Its parameters specialize in that subspace, not one original field. Preserve field/head/output layout.
Illustrative example: The first mixed FFN processes parts of all four fields, not only the first one.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why isn't higher RankUp representation rank proof of useful interests?
RankUp: Towards High-rank Representations for Large Scale Advertising Recommender Systems
Show answer and example
Answer: Rank measures independent directions, not task information. Noise can raise rank and obscure useful structure. Inspect spectra with task and subgroup metrics rather than claiming quality when tokens become less alike.
Illustrative example: Illustration: Random orthogonal user noise can yield full rank without adding any predictive preference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why use permutation grouping and global tokens against RankUp rank collapse instead of merely adding embeddings or depth?
RankUp: Towards High-rank Representations for Large Scale Advertising Recommender Systems
Show answer and example
Answer: Added parameters can reuse the same subspace. Different groupings and tables alter interaction paths, global tokens connect groups, and external features and task decoupling add signals. These aim to turn capacity into richer directions, not guarantee effective rank.
Illustrative example: Illustration: New groupings let fields interact that never shared a group under a fixed partition.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does permutation grouping change interaction opportunities versus fully shared RankUp input?
RankUp: Towards High-rank Representations for Large Scale Advertising Recommender Systems
Show answer and example
Answer: New partitions let previously separated fields meet, independent embeddings add views, and global tokens connect groups. Shared inputs can favor strong fields, while groups can miss or duplicate crosses. Combined local changes do not attribute gains to one component.
Illustrative example: Illustration: Another permutation can put age with category; unaligned parameter shuffling can instead destroy meaning.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why track RankUp permutation matrices and field identities together?
RankUp: Towards High-rank Representations for Large Scale Advertising Recommender Systems
Show answer and example
Answer: XP rearranges inputs; corresponding embedding and group weights must follow the same map. Align global-token residual widths, and do not choose permutations using test labels. A local proxy does not verify production scaling laws.
Illustrative example: Illustration: Swapping price and age while keeping the original lookup can interpret age 18 as a price.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Is averaging RECAP-CTR route logits equivalent to averaging click probabilities?
Beyond Interaction Capacity: Estimator Scaling with Recursive Models for CTR Prediction
Show answer and example
Answer: No. Sigmoid is nonlinear. The source averages logits first, which does not turn shared routes into independent complete models.
Illustrative example: Illustration: With logits 0 and 4, sigmoid of their mean is about 0.88; mean sigmoid is about 0.74.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why share recursive blocks in RECAP-CTR yet average routes and retain a training-trajectory EMA?
Beyond Interaction Capacity: Estimator Scaling with Recursive Models for CTR Prediction
Show answer and example
Answer: Different routes through shared blocks produce multiple logits without independent models. Averaging reduces route fluctuations; EMA smooths weights over training time. Correlated routes limit benefit. Local evidence verifies state and operator invariants only.
Illustrative example: Illustration: Two recursive routes yield slightly different click scores; average those while separately smoothing neighboring training checkpoints.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Why can't route averaging replace trajectory EMA in RECAP-CTR?
Beyond Interaction Capacity: Estimator Scaling with Recursive Models for CTR Prediction
Show answer and example
Answer: Route averaging combines simultaneous outputs; EMA smooths parameters over time. Shared routes can remain correlated and EMA can lag drift. Local invariants do not show CTR lift from smoothing.
Illustrative example: Illustration: All routes can share a high bias, and EMA can smooth an unusual step without adding a missing feature.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why checkpoint RECAP-CTR EMA state separately?
Beyond Interaction Capacity: Estimator Scaling with Recursive Models for CTR Prediction
Show answer and example
Answer: EMA accumulates parameter history across steps, so losing it changes the estimator on resume. Match route count and averaging denominator rather than treating missing routes as zero. Verify recurrence, EMA updates, and export separately.
Illustrative example: Illustration: Resetting EMA to current parameters after interruption loses thousands of smoothed steps even if training resumes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why aren't a historical ht and current preference z interchangeable?
Not All Is Lost: Repairing Lossy User Preference States of Personalization Encoders
Show answer and example
Answer: ht represents a contextual position and z summarizes current preference for the head. Equal dimensions do not imply the same time or coverage. Repair selects evidence while retaining current state rather than replacing it wholesale.
Illustrative example: Illustration: Last year's camera state cannot replace a current profile that includes camping.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why repair preference with cached state differences while freezing Repair-State's encoder and head?
Not All Is Lost: Repairing Lossy User Preference States of Personalization Encoders
Show answer and example
Answer: Cached states retain evidence at different times. Select long-range, recent, or burst segments and add weighted differences to restore compressed directions without retraining. It cannot recover uncached evidence and can reintroduce stale interests.
Illustrative example: Illustration: Retrieve recent baby-product activity from cache to supplement a profile dominated by long-term movie viewing.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does Repair-State differ from retraining the encoder?
Not All Is Lost: Repairing Lossy User Preference States of Personalization Encoders
Show answer and example
Answer: Postprocessing uses an existing forward cache while preserving encoder and head, reducing deployment cost. Retraining can change feature extraction. Residuals cannot recreate omitted evidence, and selection may not transfer to all tasks. The local diagnostic does not establish capability rankings.
Illustrative example: Illustration: A stored burst can be restored, but an event ignored entirely by the encoder cannot.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why check valid history before Repair-State top-k and masked softmax?
Not All Is Lost: Repairing Lossy User Preference States of Personalization Encoders
Show answer and example
Answer: Mask padding, future states, and other users before selection and normalize only selected valid states. Align differences with z. Define empty-history fallback rather than injecting random evidence. Frozen encoders do not prevent data leakage.
Illustrative example: Illustration: With two events and k=5, zero-scored padding must not beat valid negative scores and pull z toward zero.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why combine RoPE and relative time in ReST?
Show answer and example
Answer: Position and elapsed time differ: adjacent events can be a minute or a year apart. RoPE represents order, time encoding adds cadence, and dual gates filter noise. Not all periodicity is captured and large gaps are not always irrelevant.
Illustrative example: Illustration: Camera views five minutes apart suggest one comparison; views six months apart may mark a new need.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why separate heavy user encoding and light candidate interaction in ReST compute-once, decode-many?
Show answer and example
Answer: Long-history gated attention, order, and time intervals are shared user compute. Candidate cross-decoders read relevant evidence without rerunning history. Dual gates suppress noise, but only candidate-independent state is reusable and history/version changes invalidate it.
Illustrative example: Illustration: Encode history once for 200 films, then let each query it; never share one film's interaction result as general user state.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What evidence does ReST gating preserve compared with truncation?
Show answer and example
Answer: Truncation discards events; gating can suppress noise across broader history while retaining distant evidence for candidate decoding. It can suppress important items too and still incurs input cost. A small local regression does not show gating beats recent-window baselines.
Illustrative example: Illustration: Keep a small weight on old photography and downweight accidental news; truncation could remove the useful photography event entirely.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
What dependencies matter before reusing ReST encoder caches across requests?
Show answer and example
Answer: Same-request hU reuse requires unchanged history and version. New events, relative time encodings, or model updates can invalidate cross-request state. Candidate decoder outputs remain private to their candidates. Local structure does not prove production QPS or cache lifecycle.
Illustrative example: Illustration: After a user buys a camera, update history instead of reusing the old summary forever because user ID matches.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Does SAM blacklist same-category items after a purchase?
Show answer and example
Answer: No. It softly suppresses attention to related history and restores it over time, not permanent candidate filtering. Other evidence can still support recommendations.
Illustrative example: Buying shampoo reduces old browsing influence without banning hair products.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why suppress related history after buying and restore it near replenishment?
Show answer and example
Answer: A purchase can complete an intent. Find its related history and estimate personal rhythm; time gates and log masks suppress then restore it. Completion is not dislike. Local ratings proxy purchases, not actual replenishment.
Illustrative example: Reduce detergent repeats immediately, then restore after likely use rather than suppress forever.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does SAM differ from a fixed post-purchase cooldown?
Show answer and example
Answer: It personalizes suppression by history and long-term purchase rhythm. Errors can still hurt, and nonreplenishable products may not fit.
Illustrative example: Coffee and televisions should not share a one-week rule.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Where is SAM's mask applied, and what are the local limits?
Show answer and example
Answer: Add a positive log mask to attention logits with numeric safeguards, not a multiplier on final scores. No real replenishment or censored purchase logs; only mechanisms and failure boundaries.
Illustrative example: Guard near-zero masks without claiming accurate repurchase cycles.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
How does SASRec shift inputs and next-item targets?
Self-Attentive Sequential Recommendation
Show answer and example
Answer: Position i reads history through i and supervises the following item, without exposing future events. Exclude padding; each sequence yields several prefix tasks.
Illustrative example: Use A→B and A,B→C; the representation predicting B cannot read C.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
How does SASRec balance recent and distant interests without reading future?
Self-Attentive Sequential Recommendation
Show answer and example
Answer: Content attention accesses several earlier behaviors beyond fixed-order Markov neighbors, without RNN recurrence. Position representations preserve order and causal masks restrict reads. Next item is supervision only, and attention can still read irrelevant history.
Illustrative example: Recall old sci-fi interest despite recent comedy, while position three cannot read item four.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does causal next-item prediction differ from masked item recovery?
Self-Attentive Sequential Recommendation
Show answer and example
Answer: Causal training uses only history; masked recovery can use both neighbors. Right context legal for recovery is future leakage in next-item prediction.
Illustrative example: Recovering A,[MASK],C may read C; predicting B after only A may not.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How preserve SASRec embeddings and negative objective?
Self-Attentive Sequential Recommendation
Show answer and example
Answer: Tie input item embeddings with output weights and use per-position positive and sampled-negative pairwise BCE, not catalog CE. Mask padding and future positions.
Illustrative example: Never sample the true positive as its own negative.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Does a shared Semantic-Native SID prefix mean identical videos?
Beyond Item IDs: Scaling Short-Form-Video Recommendation via Semantic-Native Long Sequence Modeling
Show answer and example
Answer: No. Prefixes encode coarse similarity; identity still depends on the complete path and catalog mapping. Semantic sharing does not merge interaction records.
Illustrative example: Distinct camping videos can share a prefix but keep different logs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why compress both SID vocabulary and temporal sequence?
Beyond Item IDs: Scaling Short-Form-Video Recommendation via Semantic-Native Long Sequence Modeling
Show answer and example
Answer: Atomic ID tables grow with inventory, and full attention grows quadratically. Bounded hierarchical SID handles representation size; event folding reduces time positions and global queries gather interests. Both sacrifice detail.
Illustrative example: Fold 48 events into 12 local tokens and share semantic codes, but do not assume original within-window order survives.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does temporal folding differ from retaining only recent history?
Beyond Item IDs: Scaling Short-Form-Video Recommendation via Semantic-Native Long Sequence Modeling
Show answer and example
Answer: Folding combines neighboring events into features and keeps coarse older information; truncation drops it. Folding and global queries do not preserve full temporal resolution.
Illustrative example: Four events compressed into one token can lose ordering without discarding all evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Does the local folding exactly match paper channel rearrangement?
Beyond Item IDs: Scaling Short-Form-Video Recommendation via Semantic-Native Long Sequence Modeling
Show answer and example
Answer: No. Mean/recent is a proxy; no Google RQ-VAE or production components. Pool always-valid global queries and exclude padding.
Illustrative example: Mean folding tests coarse compression, not the paper tensor layout.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Is a SequenceO1 sketch original distant history?
SequenceO1: End-to-End Ultra-Long (100K) Sequence Modeling in Recommendation with Low-Rank Caching
Show answer and example
Answer: No. A fixed-budget prototype representation summarizes remote behavior while recent history remains finer. It is not event-by-event recoverable.
Illustrative example: Thousands of events can form a few prototypes without preserving exact times.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
What does a fixed remote sketch bound, and what still scales?
SequenceO1: End-to-End Ultra-Long (100K) Sequence Modeling in Recommendation with Low-Rank Caching
Show answer and example
Answer: Candidate–sketch matching uses fixed prototypes; recent behavior remains detailed and STCA connects timescales. Reading, building, and updating the sketch still cost work. The local proxy is not the industrial path.
Illustrative example: Matching dozens of prototypes is bounded, but constructing them from 100,000 events is not free.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does sketch plus recent differ from recent-only truncation?
SequenceO1: End-to-End Ultra-Long (100K) Sequence Modeling in Recommendation with Low-Rank Caching
Show answer and example
Answer: Sketch preserves compressed long-term patterns, whereas truncation drops remote history. Prototypes can still merge different interests.
Illustrative example: An older sports preference can survive a temporary travel streak.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Is the local 0.65/0.35 blend the paper's 100K-history formula?
SequenceO1: End-to-End Ultra-Long (100K) Sequence Modeling in Recommendation with Low-Rank Caching
Show answer and example
Answer: No. It is a public scoring proxy; no production STCA or long-history service. Fix normalization and candidate masks and do not tune blend on test.
Illustrative example: Small-catalog score gains do not show end-to-end latency with 100,000 events.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why run GSU search before ESU attention?
Search-based User Interest Modeling with Lifelong Sequential Behavior Data for CTR Prediction
Show answer and example
Answer: Lifelong histories are long. Cheap candidate-related retrieval narrows them before fine matching, rather than always taking recent events.
Illustrative example: A camera candidate can retrieve older photography behavior.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why use GSU then ESU instead of scoring lifelong history directly?
Search-based User Interest Modeling with Lifelong Sequential Behavior Data for CTR Prediction
Show answer and example
Answer: Most history is irrelevant per candidate. Cheap GSU narrows it, then ESU attends precisely to selected behavior. Fine matching cannot recover evidence omitted by coarse retrieval.
Illustrative example: If category filtering excludes old lens interactions, later camera attention cannot see them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does candidate-related selection differ from one user vector?
Search-based User Interest Modeling with Lifelong Sequential Behavior Data for CTR Prediction
Show answer and example
Answer: Different candidates retrieve different evidence. A shared user vector compresses once and is easy to cache. Quality and cost still need measurement.
Illustrative example: Shoes and books can activate different historical interests for one user.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How prevent future leakage in SIM top-k history?
Search-based User Interest Modeling with Lifelong Sequential Behavior Data for CTR Prediction
Show answer and example
Answer: GSU searches only pre-request history, separate from the target label. ESU sees selected entries only. MovieLens verifies mechanism, not production or online A/B.
Illustrative example: Do not add the next actual click before searching history.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Does not updating all history at each SlimPer layer eliminate relationships between adjacent behaviors?
SlimPer: Make Personalization Model Slim and Smart
Show answer and example
Answer: No. Section 2.5 adds local context when tokenizing events, for example via convolution or windowed attention. Later layers repeatedly read those tokens and update a fixed bank. This avoids deep all-pairs history interaction, not every historical relation. Fixed slots cap intermediate state; storing and reading inputs still grows with history length.
Illustrative example: Search, compare, and add-to-cart events can get local context before candidate-specific slots read them, without rewriting the full history at every layer.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
How does SlimPer keep its knowledge base fixed in size while reading the full history at every layer?
SlimPer: Make Personalization Model Slim and Smart
Show answer and example
Answer: Only a small set of candidate-specific slots is updated. Original user tokens are revisited as read-only evidence. Select retrieves relevant evidence, Match compares it with templates, and Refine updates slots. The full hidden history does not propagate between layers. Candidates share user encodings, but each evolves its own knowledge base rather than using one final interest vector.
Illustrative example: Shoe and camera candidates can read the same purchase records but organize size and photography signals in separate banks. Slots updated for shoes must not be reused unchanged for cameras.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
If SlimPer already selects evidence with attention, why explicitly dot-product it with knowledge templates?
SlimPer: Make Personalization Model Slim and Smart
Show answer and example
Answer: Select builds queries from the current bank and aggregates a few user evidence vectors. Match compares those vectors with templates from the same bank, yielding a query-by-template relevance matrix. Refine updates knowledge from these explicit relations rather than treating the aggregate as final interest. Retrieval and aspect-specific relevance are separate.
Illustrative example: For running shoes, Select finds running distances and sizes. Match evaluates comfort and intended use separately. Finding evidence does not tell which candidate aspect it supports.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Why is SlimPer not just a sequence encoder when compared with HSTU or LONGER?
SlimPer: Make Personalization Model Slim and Smart
Show answer and example
Answer: SlimPer models user–item relevance. Its knowledge base also receives item and cross features, and each Select–Match–Refine layer forms a personalized ranking state. A matched comparison needs candidate interactions and the task head, not only a user-sequence vector.
Illustrative example: Two candidates can produce different knowledge bases from the same history. Do not compare them solely to a candidate-independent encoder output.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Once SlimPer fixes the knowledge-base size, can later layers stop reading long histories?
SlimPer: Make Personalization Model Slim and Smart
Show answer and example
Answer: No. Each layer queries shared original tokens. It saves updates to N history states at each layer but still accesses KN query–history pairs. Short MovieLens runs, AdamW, and analytical element counts do not reproduce production QPS or Shampoo results.
Illustrative example: A small bank must still revisit the evidence; it is not a one-shot compression that discards the history.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why isn't SORT Ranking simply a language Transformer for recommendations?
SORT: A Systematically Optimized Ranking Transformer for Industrial-scale Recommenders
Show answer and example
Answer: Industrial fields, tokenization, attention, FFNs, and hardware layout are co-designed to replace DLRM. Generic residual equations are not the full system.
Illustrative example: Sparse categorical features are not prepared like natural-language words.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why co-design tokenization, attention, and FFNs?
SORT: A Systematically Optimized Ranking Transformer for Industrial-scale Recommenders
Show answer and example
Answer: Heterogeneous fields and long histories need appropriate token interfaces, information flow, and tokenwise capacity. Compression changes resource allocation. Local mechanisms do not establish full DLRM replacement throughput or quality.
Illustrative example: Organize age, category, and history rather than treating them all as ordinary text tokens.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does a ranking Transformer differ from a traditional DLRM?
SORT: A Systematically Optimized Ranking Transformer for Industrial-scale Recommenders
Show answer and example
Answer: Token interaction handles features and histories; DLRMs commonly combine embeddings, explicit interactions, and MLPs. Cost and feature processing affect deployment, not names alone.
Illustrative example: The same IDs can interact at different stages with different compute reuse.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Does local residual MHA/FFN demonstrate industrial compression?
SORT: A Systematically Optimized Ranking Transformer for Industrial-scale Recommenders
Show answer and example
Answer: No. Fixed public candidates exercise core paths without private features, production scale, or kernels. Model structure alone is not measured QPS or uplift.
Illustrative example: A small catalog run does not validate latency over industrial sparse tokens.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why chunk histories using personal time gaps?
Unleashing the Potential of Sparse Attention on Long-term Behaviors for CTR Prediction
Show answer and example
Answer: Event counts do not separate dense sessions from long breaks. Adaptive chunks follow individual pace, with global, transition, and local attention handling different scales.
Illustrative example: A burst today and a revisit next month should not be grouped solely because events are adjacent.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why combine time-adaptive chunks with global, transition, and local attention?
Unleashing the Potential of Sparse Attention on Long-term Behaviors for CTR Prediction
Show answer and example
Answer: Equal event counts may span minutes or months. Time grouping respects personal rhythm; three routes capture overview, shifts, and fine details. Relative time encodes intervals/cycles but can miss sudden or sparse interests.
Illustrative example: Separate a furniture browsing burst from later supply shopping while modeling the transition.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does three-path EvoAttention differ from a sparse local window?
Unleashing the Potential of Sparse Attention on Long-term Behaviors for CTR Prediction
Show answer and example
Answer: Local reads details, global chunks represent cross-period interests, and transition paths model changes. Time biases capture intervals and calendar cues, not causal guarantees.
Illustrative example: Weekend patterns can correlate with interest without causing clicks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Do local time features replace production chronology?
Unleashing the Potential of Sparse Attention on Long-term Behaviors for CTR Prediction
Show answer and example
Answer: No. Fixed MovieLens tests chunks and biases only. Sort pre-request events, align biases with valid masks, and exclude future labels.
Illustrative example: A shared wrong threshold can impose identical chunks on users with different rhythms.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why inspect singular spectra for representation collapse?
Show answer and example
Answer: Concentration in a few directions reduces diversity. Spectral softening changes relative singular values; spectral positional residuals affect attention. A flatter spectrum is not task evidence.
Illustrative example: Nearly parallel user vectors struggle to distinguish interests.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why can spectral softening counter collapse?
Show answer and example
Answer: Dominant singular modes concentrate tokens and limit deep interaction. Softening reduces disparity and spectral residuals give attention structural cues. This is not lossless equalization; weak modes can be noise.
Illustrative example: Reduce popularity dominance so genre directions can matter without assuming every weak direction helps.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does spectral softening differ from L2-normalizing embeddings?
Show answer and example
Answer: Unit norms control individual lengths, not collective directional spectrum. Singular-value transforms redistribute subspace energy.
Illustrative example: Unit-length vectors can still all point the same way.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
What should local SpecFormer SVD tests verify?
Show answer and example
Answer: Validate shapes, nonnegative singular values, reconstruction, positional residual alignment, and stability near zero rank. Public proxies do not reproduce production uplift.
Illustrative example: Correct matrix reconstruction does not show genuine interest recovery.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why let TM20K's teacher keep all tokens while the student merges?
Show answer and example
Answer: Full-token attention and ranking provide detail for distillation. Student compression controls serving budget but does not preserve every event.
Illustrative example: Teacher reads 20,000 events; student learns from grouped summaries.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why merge as sum/√n and distill a full-token teacher?
Show answer and example
Answer: Grouping bounds tokens, and square-root normalization moderates sum growth while retaining group-size effects. Attention and ranking distillation approximate useful fine relations without reversing compression.
Illustrative example: Merge four browses into one; teacher sees internal relationships the student cannot recover exactly.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does sum/√n differ from average pooling?
Show answer and example
Answer: Mean divides by n; square-root normalization retains different energy scaling as group size changes, depending on correlation.
Illustrative example: Repeating a vector changes the two resulting norms differently.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How align attention distillation across different lengths?
Show answer and example
Answer: Map teacher positions into student groups; do not subtract mismatched matrices. Preserve masks and sizes. Local work lacks production long histories and traffic.
Illustrative example: Specify which four teacher entries correspond to one merged slot.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why learn field-specific Q/K/V in TMallGS?
TMallGS: Scaling Unified Feature and Sequence Modeling for Generative E-commerce Search
Show answer and example
Answer: Queries, user history, and item fields have different semantics and noise. Field projections, saliency, and gates shape interactions by reliability.
Illustrative example: A search term and years of browsing need not use one projection.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why combine field-specific Q/K/V, noise gates, and layer supervision?
TMallGS: Scaling Unified Feature and Sequence Modeling for Generative E-commerce Search
Show answer and example
Answer: Separate projections address heterogeneous distributions, saliency and gates control noisy interactions, FiLM injects content, and intermediate loss guides deep learning. Gains cannot be attributed just to more heads.
Illustrative example: Downweight an accidental click while preserving an explicit search term and train intermediate relevance.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does FiLM differ from concatenating content features?
TMallGS: Scaling Unified Feature and Sequence Modeling for Generative E-commerce Search
Show answer and example
Answer: FiLM generates per-coordinate scale and bias; concatenation passes features onward. Progressive supervision also assigns intermediate targets, not just one MLP.
Illustrative example: Content can emphasize hidden directions rather than merely add columns.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Does a very large relative NDCG gain establish a stable win?
TMallGS: Scaling Unified Feature and Sequence Modeling for Generative E-commerce Search
Show answer and example
Answer: No. Absolute hits are few and one seed with a low denominator inflates percentages. The run exercises field, gate, FiLM, bias, and intermediate training paths.
Illustrative example: A tiny absolute improvement can have a huge percentage.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Does parameter-free mixing make the whole TokenMixer-Large parameter-free?
TokenMixer-Large: Scaling Up Large Ranking Models in Industrial Recommenders
Show answer and example
Answer: No. Mix/revert are fixed rearrangements; headwise and tokenwise SwiGLU, deep residuals, and auxiliary losses still learn.
Illustrative example: Channel shuffling is fixed, but the FFNs processing channels train.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why use headwise SwiGLU after mixing and tokenwise SwiGLU after revert?
TokenMixer-Large: Scaling Up Large Ranking Models in Industrial Recommenders
Show answer and example
Answer: The former learns cross-token combinations from rearranged channels; the latter refines restored per-token states. Residual intervals and intermediate supervision support depth. Rearrangement moves information but does not learn interactions.
Illustrative example: Mix price and preference channels, learn interactions, then restore fields for token-specific processing.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What does mix–headFFN–revert add over tokenwise FFN?
TokenMixer-Large: Scaling Up Large Ranking Models in Industrial Recommenders
Show answer and example
Answer: Rearrangement exposes slices from different tokens to a nonlinear head transform, then restores layout. It is not dynamic attention, and one small gain cannot replace all attention.
Illustrative example: Each token can receive transformed evidence from other tokens after reverting.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
What can the local RankMixer comparison establish?
TokenMixer-Large: Scaling Up Large Ranking Models in Industrial Recommenders
Show answer and example
Answer: Match width, depth, negatives, and steps, but the result is small single-seed and lacks full attention controls. Bind FFNs to mixed positions and preserve revert order.
Illustrative example: The first mixed head need not be the first original feature slot.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why combine lifelong and recent histories in TransAct V2?
TransAct V2: Lifelong User Action Sequence Modeling on Pinterest Recommendation
Show answer and example
Answer: Candidate retrieval finds relevant older evidence, while recent history captures immediate intent. One Transformer fuses both with candidates early.
Illustrative example: A current browse and a similar older interest can both support a candidate.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why keep raw recent events and add Next Action Loss?
TransAct V2: Lifelong User Action Sequence Modeling on Pinterest Recommendation
Show answer and example
Answer: Candidate matching retrieves old evidence but can miss emerging intent. Keep recent events and fuse with retrieved history. NAL learns next positive action beyond CTR; missed remote events and serving performance remain separate.
Illustrative example: A tent candidate finds old camping while raw history shows new family-travel interest.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does guaranteed recent retention differ from full-history top-k?
TransAct V2: Lifelong User Action Sequence Modeling on Pinterest Recommendation
Show answer and example
Answer: It keeps fresh behavior even with low similarity, while other positions come from candidate search. Extra context costs compute; gains depend on data.
Illustrative example: A new interest may not rank highly under an old similarity model.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Can Next Action Loss labels enter the current input?
TransAct V2: Lifelong User Action Sequence Modeling on Pinterest Recommendation
Show answer and example
Answer: No. Next positive behavior is supervision only; input reads the past, and negatives must be valid. Local model scope excludes production SKUT/request dedup and throughput.
Illustrative example: Adding the future click to history trivially leaks the auxiliary target.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why split behavior and request streams?
TransX: Scaling Transformer-based Recommendation via Behavioral and Serving Stream Crossings
Show answer and example
Answer: Cache global/local historical states asynchronously, then let short candidate requests cross-attend. Share only candidate-independent portions.
Illustrative example: Several candidates read one user cache but retain separate matching.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why cache global/local behavior and cross-attend only from the short service stream?
TransX: Scaling Transformer-based Recommendation via Behavioral and Serving Stream Crossings
Show answer and example
Answer: Behavior changes slower than requests, allowing historical encoding amortization. Global represents overall interests, local recent detail. Cache building/refresh still cost work, and candidate-dependent states are not public user caches.
Illustrative example: Reuse long history across opens and refresh local data without re-encoding years per candidate.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does shared multi-cutoff pooling compare with independent pooling?
TransX: Scaling Transformer-based Recommendation via Behavioral and Serving Stream Crossings
Show answer and example
Answer: Merge softmax statistics to reuse prefix work, with equivalent forward and gradients. This is compute reuse, not a changed score. Base, Retrieval, and Ranking are distinct production tests.
Illustrative example: Shared pooling should match independent computation on the same input.
Sections in the Chinese source: 补充核心公式, 补充报告效果与本地验收. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Why combine duplicate sparse row gradients before squaring?
TransX: Scaling Transformer-based Recommendation via Behavioral and Serving Stream Crossings
Show answer and example
Answer: Squaring is nonlinear: sum logical-step gradients first. Small A100 tensors verify this rule, not the 32-H200 system or an upgraded adapter grade.
Illustrative example: Opposite gradients should cancel before the square.
Sections in the Chinese source: 补充核心公式, 补充报告效果与本地验收. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why serialize cross-page activity as a Tubi user story?
TubiFM: Unified Item, Carousel, and Search Ranking for Streaming Discovery
Show answer and example
Answer: Item, carousel, and search activity relates across the journey. One history and task prompts share evidence while preserving task-specific ranking.
Illustrative example: Search then browse a collection to explain what film the viewer now wants.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why use one cross-page story and task prompts for item, carousel, and search?
TubiFM: Unified Item, Carousel, and Search Ranking for Streaming Discovery
Show answer and example
Answer: Separate page models miss links between events. One sequence shares the journey, while prompts identify scoring targets. Shared input does not equal shared objectives or calibrated candidate types.
Illustrative example: Use suspense search and carousel browsing as common context, then specify whether ranking films, rows, or results.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What is the trade-off between one task-prompted model and separate page models?
TubiFM: Unified Item, Carousel, and Search Ranking for Streaming Discovery
Show answer and example
Answer: Sharing can transfer signals and simplify maintenance, but may cause interference. Evaluate each task under matched budgets.
Illustrative example: Long-term home-feed taste should not override a precise search intent automatically.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How does local task serialization differ from the production foundation model?
TubiFM: Unified Item, Carousel, and Search Ranking for Streaming Discovery
Show answer and example
Answer: Fixed public candidates test model state and task-conditioned scores, not private features, scale, or traffic. Align task tokens, histories, and labels.
Illustrative example: Three prompts must change execution, not just log labels.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why compress lifelong history into sized virtual items?
TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou
Show answer and example
Answer: Per-event matching over millions is expensive. Offline hierarchy summarizes groups and tracks size, then online retrieval and ranking read relevant clusters.
Illustrative example: Many repeated events should differ from one isolated event.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why build sized virtual items before candidate-related retrieval?
TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou
Show answer and example
Answer: Offline clustering compresses repeated events. GSU retrieves relevant clusters and ESU performs size-aware interactions without naively duplicating large groups. Rare details may disappear.
Illustrative example: Represent years of similar shoe browsing with a count rather than recomputing attention on thousands of events.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does cluster retrieval differ from SIM raw-behavior search?
TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou
Show answer and example
Answer: Cluster search filters prototypes before finer interaction; raw retrieval selects original behaviors. Compression changes coverage, accuracy, and cost, with possible rare-detail loss.
Illustrative example: A large cluster may hide a niche event that raw search can still find.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Does public full-catalog ranking verify million-event serving?
TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou
Show answer and example
Answer: No. It tests candidate search and hierarchical compression without private lifelong logs or serving. Align memberships, sizes, and time masks.
Illustrative example: Small MovieLens retrieval is not production latency evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why add landmarks beyond the local ULTRA-HSTU window?
Bending the Scaling Law Curve in Large-Scale Recommendation Systems
Show answer and example
Answer: Local attention preserves detail; sparse landmarks connect distant behavior. LBSL widens deeper layers. Sparse access is not lossless full-HSTU equivalence.
Illustrative example: Read recent events individually and use representative remote positions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why combine local windows, landmarks, and increasing layer reach?
Bending the Scaling Law Curve in Large-Scale Recommendation Systems
Show answer and example
Answer: Fixed local masks lose remote paths and full attention is expensive. Sparse landmarks provide global connections, wider deeper windows integrate more context, and mixture/system changes support scale. Efficiency depends on windows, landmark counts, and kernels.
Illustrative example: Shallow layers read recent clicks, while landmarks and deeper layers connect older interests without recovering every event.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How do semi-local attention and deeper wider windows differ from one fixed window?
Bending the Scaling Law Curve in Large-Scale Recommendation Systems
Show answer and example
Answer: Deeper layers gain wider direct access, while fixed windows retain identical local connections per layer. Effective reach also depends on stacking and landmarks; one formula does not cover mixture, precision, and systems changes.
Illustrative example: Read a visit closely in shallow layers, then relate cross-visit interests deeper.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Can a 48-event ablation prove the 16K production scaling law?
Bending the Scaling Law Curve in Large-Scale Recommendation Systems
Show answer and example
Answer: No. It lacks 18-layer production models, dynamic topology, FP8 kernels, and private logs. Scaling needs matched runs across lengths and hardware measurement.
Illustrative example: A max-length setting of 16000 does not test long history if only dozens are loaded.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why tokenize feature roles and tasks in UniFormer?
UniFormer: Efficient and Unified Model-Centric Scaling for Industrial Recommendation
Show answer and example
Answer: One interaction interface carries candidate-independent, candidate-dependent, and task-specific states. Tokenization preserves role distinctions rather than flattening all fields.
Illustrative example: User history, current product, and click objective each get appropriate tokens.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why distinguish candidate-independent, candidate-dependent, and task tokens before unified interactions?
UniFormer: Efficient and Unified Model-Centric Scaling for Industrial Recommendation
Show answer and example
Answer: They encode common user state, current match, and prediction objective respectively. Explicit roles ease scaling, but joint interactions can make states candidate-dependent; reuse needs masks and architecture.
Illustrative example: A shared age token is reusable, but a state containing one candidate's price is not.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
What is the trade-off versus separate models per task?
UniFormer: Efficient and Unified Model-Centric Scaling for Industrial Recommendation
Show answer and example
Answer: Shared interactions can transfer signal and ease maintenance; task tokens condition predictions. Parameter sharing can still cause interference.
Illustrative example: Viewing features may help clicks but conflict with commerce goals.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How avoid token misalignment locally?
UniFormer: Efficient and Unified Model-Centric Scaling for Industrial Recommendation
Show answer and example
Answer: Match role, position, and valid mask to fields, and have each task head read its own state. MovieLens lacks industrial fields and serving.
Illustrative example: Task A reading task B's token is wrong despite matching shapes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why keep two separate pre-ranking and ranking towers?
UniRec: Cross-Stage Multi-Task Fusion with Preference Optimization for Cascaded Recommender Systems
Show answer and example
Answer: Their features, candidate scales, and latency differ. Share some embeddings and align downstream preference through training, without exposing expensive online ranking input upstream.
Illustrative example: Pre-ranking learns fine-ranker preference from supervision, not its full feature set.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why preserve both directions for one pair in CPPA?
UniRec: Cross-Stage Multi-Task Fusion with Preference Optimization for Cascaded Recommender Systems
Show answer and example
Answer: Objectives can disagree, and a majority loses strength and minority signal. Both softplus penalties remain, CAPP selects confident directions, and grouped KL limits drift. Downstream supervision is training-only.
Illustrative example: Clicks favor A while purchases favor B; majority voting can hide the purchase signal.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does CPPA differ from majority-voting among objectives?
UniRec: Cross-Stage Multi-Task Fusion with Preference Optimization for Cascaded Recommender Systems
Show answer and example
Answer: Opposing evidence contributes to two softplus terms rather than discarding the minority. CAPP chooses confident directed pairs and AGRR controls group drift separately.
Illustrative example: Clicks can prefer A and watch time B; retain both directions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
How keep cross-stage gradient isolation and AGRR evidence separate from pruning?
UniRec: Cross-Stage Multi-Task Fusion with Preference Optimization for Cascaded Recommender Systems
Show answer and example
Answer: Cross-stage alignment uses survivors and freezes the fine-ranker direction. AGRR builds grouped KL targets from unpruned inputs, not centered scores. Document variance and quantile ties.
Illustrative example: Computing evidence after pruning changes the distribution AGRR should see.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why expand both data space and model capacity?
UniScale: Synergistic Entire Space Data and Model Scaling for Search Ranking
Show answer and example
Answer: Larger models still face exposure bias; more data can also exceed current capacity. ES³ expands beyond exposure, HHSFT fuses sources, and capacity must remain deployable.
Illustrative example: Adding unexposed examples differs from adding layers to old exposure data.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why expand exposure-external data and train hierarchically rather than only grow the model?
UniScale: Synergistic Entire Space Data and Model Scaling for Search Ranking
Show answer and example
Answer: More parameters can overfit limited exposure. Broader data expands coverage and hierarchical fusion treats heterogeneous quality appropriately. Verify synthetic/unexposed reliability; more data need not reduce bias.
Illustrative example: Add unseen categories without giving weak labels the same weight as verified clicks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Are unexposed examples equivalent to random negatives?
UniScale: Synergistic Entire Space Data and Model Scaling for Search Ranking
Show answer and example
Answer: No. Sources and label reliability differ. Not exposed is not disliked, and full-space expansion does not automatically remove selection bias.
Illustrative example: A user who never saw an item cannot be assumed not to click it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Does local UniScale gated FFN represent the full ES³ data pipeline?
UniScale: Synergistic Entire Space Data and Model Scaling for Search Ranking
Show answer and example
Answer: No. It tests heterogeneous branches, not private full-space sampling, production models, or cheap deployment. Audit source labels, splits, and mixing weights.
Illustrative example: Working FFNs do not prove reliable exposure-external data generation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why share UxSID memory by user and target SID?
UxSID: Semantic-Aware User Interests Modeling for Ultra-Long Sequence
Show answer and example
Answer: Per-item retrieval is expensive and one user-only summary is too coarse. Similar first-level SID targets share memory, trading cache cost for targeting without assuming identical preferences.
Illustrative example: Several shoes can reuse interest memory while retaining item differences.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why share memory by first-level target SID instead of per-item or all-item compression?
UxSID: Semantic-Aware User Interests Modeling for Ultra-Long Sequence
Show answer and example
Answer: Grouping amortizes full history while retaining target sensitivity. Global SID response gates anchor probing; independent PFFNs and orthogonality mitigate collapse. Within-group details can still be flattened.
Illustrative example: Camping items share one camping memory while cameras use another.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
Why are global and local probes not duplicate attention?
UxSID: Semantic-Aware User Interests Modeling for Ultra-Long Sequence
Show answer and example
Answer: Query full history for a global response, gate the target SID, then read compressed anchors. The inputs differ; anchor-only loses global modulation.
Illustrative example: Determine overall category interest before inspecting individual anchors.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
What details matter for per-anchor PFFN and orthogonality?
UxSID: Semantic-Aware User Interests Modeling for Ultra-Long Sequence
Show answer and example
Answer: Give anchors independent PFFNs and normalize orthogonality by the whole matrix's squared L2 norm, not row cosine norms. Local two-level k-means/residual quantization is not LETTER.
Illustrative example: Sharing one FFN changes anchor specialization even at matching shapes.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why exchange information between Wukong and HSTU at each layer?
WHALE: A Scalable Unified Model for Recommendation with Wukong-HSTU Architecture
Show answer and example
Answer: Sparse high-order fields and behavior sequences co-evolve instead of meeting only at final concatenation. Coupling may also cause interference.
Illustrative example: Attributes affect sequence interest and sequence states contextualize feature interactions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why exchange Wukong and HSTU state bidirectionally at every layer rather than only at the end?
WHALE: A Scalable Unified Model for Recommendation with Wukong-HSTU Architecture
Show answer and example
Answer: Field interactions model high-order combinations and sequences capture evolving behavior. Cross-attention lets each guide later processing by the other. This adds dependencies and cost, not reusable user-only caching; local evidence is scaled public evaluation.
Illustrative example: Weekend context can focus sequence attention on camping, then updated interests inform price/category interactions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does bidirectional cross-attention differ from late fusion?
WHALE: A Scalable Unified Model for Recommendation with Wukong-HSTU Architecture
Show answer and example
Answer: Late-fusion paths stay independent until the end. Intermediate cross-attention changes both state streams and adds dependency costs.
Illustrative example: Layer two can already read features and change later behavior modeling.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
What update order matters for WHALE's two streams?
WHALE: A Scalable Unified Model for Recommendation with Wukong-HSTU Architecture
Show answer and example
Answer: Follow same-layer query/key/value inputs as specified; accidental in-place overwrites change what the other stream reads. Public code lacks industrial features and scaling evidence.
Illustrative example: Updating x before h can differ from both reading old layer states.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why keep linear and deep paths together?
Wide & Deep Learning for Recommender Systems
Show answer and example
Answer: Wide memorizes sparse observed crosses; deep embeddings generalize to unseen combinations. One jointly trained logit combines them, rather than alternating independent services.
Illustrative example: Memorize frequent combinations and generalize related new ones.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why train crosses and deep features jointly rather than average independent scores?
Wide & Deep Learning for Recommender Systems
Show answer and example
Answer: Both contribute one logit and share objective gradients, allowing complementary learning. Handcrafted crosses have limited coverage and deep generalization can be wrong.
Illustrative example: Wide memorizes a frequent user–category cross while deep transfers to related new combinations.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How can wide memorization and deep generalization each fail?
Wide & Deep Learning for Recommender Systems
Show answer and example
Answer: Memorization misses unseen crosses, while similarity can overgeneralize to incompatible combinations. Capacity should match sparsity and feature engineering.
Illustrative example: Semantic similarity does not establish accessory compatibility.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Does local Wide & Deep use production Google Play crosses?
Wide & Deep Learning for Recommender Systems
Show answer and example
Answer: No. MovieLens replaces private logs and distributed training/serving. Add logits before sigmoid and train jointly, rather than averaging post-sigmoid probabilities.
Illustrative example: sigmoid(a+b) generally differs from averaging sigmoid(a) and sigmoid(b).
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Basics
Why fuse many features into a few Prime Tokens?
Zenith: Scaling up Ranking Models for Billion-scale Livestreaming Recommendation
Show answer and example
Answer: Fewer wider tokens reduce interaction count and retain capacity; differentiated attention and independent enhancement counter homogenization. Compression can still lose detail.
Illustrative example: Put sparse fields in several wide slots without making every slot identical.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Core mechanism
Why compress heterogeneous features into wide Prime Tokens and then enhance each separately?
Zenith: Scaling up Ranking Models for Billion-scale Livestreaming Recommendation
Show answer and example
Answer: Token Fusion trades fewer positions for wider channels; RSA/TMHSA maintain differentiation and SwiGLU or MoE adds role-specific transformations. Check information loss and matched compute.
Illustrative example: Reduce pairwise interactions over 100 fields while preserving meaningful user/item roles.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Method comparison
How does token-specific TSwiGLU differ from shared FFNs?
Zenith: Scaling up Ranking Models for Billion-scale Livestreaming Recommendation
Show answer and example
Answer: Independent transformations let roles specialize, but add parameters. Match budgets before attributing gains to architecture.
Illustrative example: User and item slots may benefit from different nonlinearities.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Search, ads, and recommendation / User histories and ranking backbones
Implementation details
Does local Zenith establish a scaling law?
Zenith: Scaling up Ranking Models for Billion-scale Livestreaming Recommendation
Show answer and example
Answer: No. Small fixed data exercises fusion, differentiated interaction, and enhancement without production hardware or size. A scaling slope needs matched multi-size curves.
Illustrative example: A win at one scale does not imply the gap grows with size.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Topic guide and reading order
Start with this question¶
Should compute go to history–history, candidate–history, or heterogeneous field interactions?
Takeaway¶
Identify what interacts. DIN uses candidates to activate relevant history. HSTU exchanges history states across layers. RankMixer uses fixed TokenMix and independent FFNs for heterogeneous features. Long-history compression, mixed sequence and field tokens, sparse attention, and transfer choose what state to keep and when to exchange it. Backbones can complement each other; “Transformer” is not one mechanism.
Example¶
A user has a long gardening history and recent cat videos. Cat-food ranking can read recent cat evidence, while garden tools require older interests. Price, context, and profile still need feature crosses. History and fields serve different purposes.
Common pitfall¶
Field-token count is not history length; do not compare their complexity tables directly. A cache can be shared only if it is candidate-independent. Public short-history results do not establish industrial long-history gains or serving throughput.
Reading order¶
Read DIN for candidate-conditioned interest, HSTU for evolving history states, and RankMixer for field interaction. Then read the topic deep dive for LONGER's short-query path versus SlimPer's fixed user pool. Interpret results within their data and fidelity.
- Deep Interest Network for Click-Through Rate Prediction · Original paper
- Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations · Original paper
- RankMixer: Scaling Up Ranking Models in Industrial Recommenders · Original paper
Check your understanding¶
Can a user state that already saw candidate A be reused unchanged as candidate B's shared cache?
Show suggested answer
Usually no. It depends on A and can contaminate B. Keep a candidate-independent shared path or recompute the candidate-dependent part. A module named user tower is not necessarily reusable.