Slates and business control: reranking, bidding, and long-term value¶
简体中文 · English
Search, ads, and recommendation / 148 questions · 37 papers
Try answering each question before opening the explanation. The bank grows as papers and questions are added.
Read this topic's comparisons, analysis, and cross-paper Q&A
148 questions · 37 papers
Basics
Why optimize per-request ad quantity, not just order?
Adaptive Ad Load Design for Sponsored Search Markets: Evidence, Theory, and Deployment
Show answer and example
Answer: Ranking fills given slots; load chooses their number and affects experience, revenue, and conversion. Estimate marginal response over feasible counts.
Illustrative example: Identical ad candidates can support different loads for shopping and exploratory requests.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why not always choose the ad count maximizing immediate ad revenue?
Adaptive Ad Load Design for Sponsored Search Markets: Evidence, Theory, and Deployment
Show answer and example
Answer: Count affects organic experience and conversions as well as revenue. Randomized field experiments estimate count responses; constrained decisions use Lagrangian trade-offs. Correlation logs or ranking proxies are not causal or market evidence.
Illustrative example: Shopping and navigation queries can react differently to adding one ad.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Why use randomized field evidence instead of comparing high-revenue logs?
Adaptive Ad Load Design for Sponsored Search Markets: Evidence, Theory, and Deployment
Show answer and example
Answer: Existing policies choose load based on intent, which confounds revenue differences. Randomized load helps identify intervention effects. Public ranking without interventions cannot establish causal count effects.
Illustrative example: Many ads on already high-intent requests does not justify increasing all loads.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why enumerate feasible counts with consistent revenue and conversion definitions?
Adaptive Ad Load Design for Sponsored Search Markets: Evidence, Theory, and Deployment
Show answer and example
Answer: Score each legal k under inventory and business limits, using comparable revenue and weighted conversion terms. Tune coefficients on validation, not test.
Illustrative example: Do not output five ads when zero to three are legal or compare inconsistent per-request/per-ad metrics.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why does CDM view diversity as marginal value given the selected prefix?
Contextual Distillation Model for Diversified Recommendation
Show answer and example
Answer: Value depends on what the list already contains: the first science-fiction item can add variety while the fourth similar one is redundant. MMR combines relevance with redundancy penalties; CDM learns the contextual gain for low-latency serving. Diversity is not a permanent per-item bonus.
Illustrative example: A third movie by a director may add little if two are already selected, yet be valuable when no work by that director appears.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why must CDM's student see same-request candidates when distilling an MMR teacher?
Contextual Distillation Model for Diversified Recommendation
Show answer and example
Answer: MMR selection depends on other candidates and redundancy, so isolated item features cannot recover it. CDM learns selection propensity relative to context and combines it with the original recommendation score, transferring costly iterative redundancy reduction to a serving-time scorer. Student approximation need not reproduce the list exactly, and a small local run does not establish production diversity gains.
Illustrative example: A travel clip may be redundant among ten similar clips but distinctive in a cooking list.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
What does CDM distillation save compared with online MMR, and what error can it introduce?
Contextual Distillation Model for Diversified Recommendation
Show answer and example
Answer: MMR recomputes redundancy against the selected set at each step. The student approximates those decisions quickly, reducing iterative serving cost. It can drift on new candidate distributions or longer lists, so test precision and diversity as well as latency. Proxy gains do not prove exact imitation.
Illustrative example: One student score can replace repeated comparisons, but the student may underestimate redundancy in a near-duplicate set absent from training.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why update similarity penalties after selecting each item in the CDM teacher?
Contextual Distillation Model for Diversified Recommendation
Show answer and example
Answer: MMR penalizes maximum similarity to the current selected prefix. Every addition changes remaining marginal scores; one fixed ranking is not the same contextual decision. Student targets must correspond to the relevant prefix rather than conflating stages in a static item label.
Illustrative example: After selecting a camping tutorial, reduce the score of its near-duplicate. An empty-prefix score could put both next to each other.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
What decisions does CGR unify by putting generation and reward estimation in one network?
Constraint-Aware Generative Re-ranking for Multi-Objective Optimization in Advertising Feeds
Show answer and example
Answer: The model proposes the next item and estimates its business value after insertion, conditioned on the prefix, ad positions, and load rules. Feasibility determines what can be chosen; reward compares legal options. A high reward does not override the rules.
Illustrative example: Once the screen reaches its ad cap, even a high-value ad cannot be generated; rank eligible organic items instead.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
How do CGR's decoding-time ad-load constraints differ from deleting surplus ads afterward?
Constraint-Aware Generative Re-ranking for Multi-Objective Optimization in Advertising Feeds
Show answer and example
Answer: Post-hoc deletions change context and remaining slots, invalidating value estimates for the original prefixes. CGR maintains the feasible set at each generation step and combines it with estimated reward to prune infeasible branches. A high-scoring but unusable list is not a valid result.
Illustrative example: With at most two ads in five positions, mask ads after using both slots. Generating four ads and deleting two changes the list the system scored.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does CGR's constraint-aware pruning differ from fixed item ranking in prefix dependence?
Constraint-Aware Generative Re-ranking for Multi-Objective Optimization in Advertising Feeds
Show answer and example
Answer: Fixed item scores do not depend on selected items. CGR updates feasibility and reward at each step, so count and position rules directly change later choices. This reduces repairs but requires accurate value estimates and search budget; one proxy comparison does not prove global optimality.
Illustrative example: An ad in slot one can force organic content in slot two under spacing rules without changing any ordinary item score.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
When should an implementation of CGR update `F(y<t)`?
Constraint-Aware Generative Re-ranking for Multi-Objective Optimization in Advertising Feeds
Show answer and example
Answer: After each selection, update the prefix, load, and position state, then construct the next feasible set. Do not generate the whole sequence and assume every prefix was legal. An empty set must terminate or use a defined legal fallback, never silently fill with an invalid item.
Illustrative example: If the final slot has no eligible organic items and the ad cap is reached, report or terminate as defined instead of adding an ad.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why does CohortMix-TS construct a mixed cohort prior before sufficient personal data arrive?
Show answer and example
Answer: Few personal observations give unstable success estimates. Infer cohort memberships from metadata and mix historical item-arm means to guide initial exploration. Fixed strength limits borrowed confidence so individual feedback can gradually revise it. Shared membership does not imply identical preferences.
Illustrative example: Initialize a new user's movie success rates from two cohorts, then let a few personal outcomes move the posterior away from the group average.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why does CohortMix-TS use a mixed Beta prior with fixed strength instead of inheriting all cohort counts?
Show answer and example
Answer: Similar cohorts can warm-start a new user, but copying their sample sizes would make personal observations ineffective for too long. Mix cohort means, use controlled pseudocounts, and update on individual exposure feedback. Thompson sampling retains exploration, and local evidence only checks the posterior mechanism.
Illustrative example: With 100,000 cohort events and three personal events, full inheritance drowns out the new user. Fixed prior strength allows personal preference to take over gradually.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does CohortMix-TS explore differently from always choosing the highest posterior mean?
Show answer and example
Answer: Greedy selection ignores uncertainty. Thompson sampling draws from each posterior, giving uncertain but promising arms a chance; more evidence naturally reduces uncertainty. It does not improve every round, and the local three-seed state-transition diagnostic is not recommendation capability evidence.
Illustrative example: A slightly lower-mean, rarely observed arm can sometimes beat a well-known arm in sampled scores, while greedy selection may never try it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why keep independent Beta counts per new user and exclude previously exposed slate items in CohortMix-TS?
Show answer and example
Answer: The cohort prior only initializes state. Subsequent success and failure belong to that user's arm, not every new user. Select from unseen items each round for sequential exploration; unexposed arms do not receive invented failures.
Illustrative example: A comedy click updates only the viewing user's success count. An action movie not shown receives no feedback.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why feed exploration feedback into DEGR's generator rather than just inserting random items at the end?
DEGR: Dual Exploration-Driven Generative Re-Ranking for Adaptive Cross-Request Context Bridging
Show answer and example
Answer: Random insertions collect feedback but do not teach the generator which new content is valuable. Reward-adaptive preference updates retain useful exploration, while same-cohort regularization and inference selection reduce repeats. Lower head share alone is not user benefit.
Illustrative example: Positive feedback on an uncommon outdoor item can inform future generation; random insertion alone starts from scratch each time.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why does DEGR constrain same-cohort representations and adapt preference update strength to exploration reward?
DEGR: Dual Exploration-Driven Generative Re-Ranking for Adaptive Cross-Request Context Bridging
Show answer and example
Answer: High per-request scores can still expose similar items repeatedly across requests. Representation regularization reduces directional overlap, adaptive preference updates avoid reinforcing exploration without gains, and inference diversity selection acts on the final list. Spread vectors do not guarantee perceived diversity; measure relevance and exploration feedback.
Illustrative example: Similar shoes may dominate successive sensible lists. Spread candidate directions and separately verify that a new category provides worthwhile exploration.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Where do DEGR training diversity regularization and inference diversity-aware greedy selection act?
DEGR: Dual Exploration-Driven Generative Re-Ranking for Adaptive Cross-Request Context Bridging
Show answer and example
Answer: Squared cosine regularization spreads same-cohort embeddings; greedy selection limits redundancy relative to items already in the current list. Neither substitutes for the other. Excessive regularization separates truly similar items, and overly strong selection can hurt relevance, so ablate each.
Illustrative example: Spread embeddings do not guarantee variety on one screen; forced screen variety does not fix a representation collapsed around popular items.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
How should local DEGR with unchanged NDCG and lower head share be interpreted?
DEGR: Dual Exploration-Driven Generative Re-Ranking for Adaptive Cross-Request Context Bridging
Show answer and example
Answer: Logs confirm actual preference and diversity-pair updates; report ranking and exposure distribution separately. Zero NDCG gain means no main ranking improvement under this protocol, and slightly lower head share only indicates reduced concentration. Loss presence and update counts are not user value.
Illustrative example: A less popular list without better relevant positions supports wider exposure, not necessarily happier users.
Sections in the Chinese source: 本地复现. The example illustrates the mechanism.
Basics
Why use DeGRe's offline cumulative-return evaluator instead of only the next logged item?
DeGRe: Listwise Generative Reranking with Offline Lookahead Distillation
Show answer and example
Answer: A point label describes one logged path, not the future list value of alternatives. Beam search uses prefix-dependent lookahead values and distills them into the generator. This expands beyond single-path imitation but can carry evaluator errors.
Illustrative example: A basic tutorial can enable useful advanced items despite a low immediate click probability.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why does DeGRe use offline beam lookahead but only a greedy generator online?
DeGRe: Listwise Generative Reranking with Offline Lookahead Distillation
Show answer and example
Answer: Lookahead scores future list value across prefixes but is costly online. Offline search finds valuable lists and distills value distributions over remaining candidates under each prefix into soft targets. Greedy serving approximates future effects; it learns estimated values, not a guaranteed optimum, and may inherit evaluator bias.
Illustrative example: The highest-scoring first item can force redundant later choices. Offline search can find a better opening and teach it without online permutation search.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How are DeGRe dense soft labels richer than distilling only the best beam list?
DeGRe: Listwise Generative Reranking with Offline Lookahead Distillation
Show answer and example
Answer: A winning list supplies one target per step. Dense labels distribute probability across all remaining candidates under that prefix by lookahead value, with list weights emphasizing better trajectories. Denser targets can also spread evaluator bias.
Illustrative example: Two good books can both get high target probabilities rather than one winner; a popularity-biased evaluator can still mis-score both.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why mask selected items and normalize DeGRe's q_t over candidates remaining under the same prefix?
DeGRe: Listwise Generative Reranking with Offline Lookahead Distillation
Show answer and example
Answer: The target represents legal next choices. Repeated items must be excluded, scores must share a prefix, and target softmax and student masks must agree. Greedy serving still updates selected state step by step without a beam.
Illustrative example: After movie A is selected, step two distributes probability among other movies. Keeping A in the denominator can train repeated recommendations.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why does DIRECTOR's dynamic index distribution cover all display positions jointly?
DIRECTOR: Dynamic Index-based Recommendation with Transport-Optimized Retrieval
Show answer and example
Answer: Positions draw from competing candidate sets. Joint transport connects position supply and item allocation, and hard matching creates a feasible slate. Parallel generation reduces sequential dependencies, but still requires relevance and legality checks.
Illustrative example: Five positions can all favor the same movie; joint allocation restricts it to its allowed count and assigns other options elsewhere.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why does DIRECTOR combine Sinkhorn with global matching rather than independently choosing each position's top item?
DIRECTOR: Dynamic Index-based Recommendation with Transport-Optimized Retrieval
Show answer and example
Answer: Independent predictions can select the same item. Sinkhorn constructs a soft transport plan constrained by row and column marginals; global hard matching then creates a conflict-free list. Soft probabilities alone are not a displayable permutation.
Illustrative example: If all three positions favor A, matching assigns it where most useful and places B and C elsewhere.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Why not decode DIRECTOR's Sinkhorn matrix with independent row argmaxes?
DIRECTOR: Dynamic Index-based Recommendation with Transport-Optimized Retrieval
Show answer and example
Answer: Continuous soft constraints do not guarantee that independent maxima choose distinct columns. Discrete global matching trades per-position scores for total feasible assignment quality. Temperature and iteration affect approximation, not rowwise legality.
Illustrative example: If both rows favor A at 0.6 over B at 0.4, independent choices conflict; hard matching must give B to one row.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
How should DIRECTOR align score matrices and final permutations with candidate IDs?
DIRECTOR: Dynamic Index-based Recommendation with Transport-Optimized Retrieval
Show answer and example
Answer: Rows are positions and columns are candidate identities. Maintain column identity through transport and matching, and retrieve by matched global indexes rather than per-row rank. Explicitly handle duplicate IDs or too few candidates. Local movie proxies do not establish production valid views.
Illustrative example: If column two was movie B, indexing a reshuffled candidate table after matching can display the wrong movie.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why screen immediate behaviors with future engagement in Downstream Rewards?
Long-term User Engagement Optimization through Model-agnostic Downstream Rewards Learning
Show answer and example
Answer: Short-term clicks or saves can be fleeting. Find signals predictive of later activity, then train online-predictable auxiliary heads to account for longer-horizon value. Association remains provisional and each product needs independent verification.
Illustrative example: Compare accidental image opens with deeper collection exploration before selecting a reward target.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why screen downstream rewards for future engagement instead of adding all immediate behavior to ranking?
Long-term User Engagement Optimization through Model-agnostic Downstream Rewards Learning
Show answer and example
Answer: Immediate responses can conflict with satisfaction or amplify noise. Test candidate signals for future engagement and connect only selected heads. Predictive association does not imply causal benefit; evaluate each product and do not add short-term gains as retention.
Illustrative example: Opening a large image may be accidental; deeper collection browsing might predict later activity better and is worth checking.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does adding a downstream reward head differ from replacing the ranker?
Long-term User Engagement Optimization through Model-agnostic Downstream Rewards Learning
Show answer and example
Answer: An auxiliary head preserves the backbone and adds a filtered long-horizon proxy to loss or score fusion across surfaces. Replacement changes more features and serving logic. Model-agnostic integration does not imply shared label semantics or improved component metrics.
Illustrative example: A feed can keep relevance and add deep-consumption prediction. Notification weights can differ, and saves are not notification retention.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Where does future-window information belong when training a downstream reward head?
Long-term User Engagement Optimization through Model-agnostic Downstream Rewards Learning
Show answer and example
Answer: Use it for reward screening and supervision, not features beyond request time. Select fusion strength on validation and freeze it. Local future category coverage is a proxy, not platform retention; do not read held-out future events.
Illustrative example: Tomorrow's new-topic exploration can label today's prediction, but cannot be today's input.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why is DREAM a control plane over an existing recommendation pipeline?
DREAM: A Dual-Loop Recommendation Evolution Framework Powered by Large Language Models
Show answer and example
Answer: Intent and Meta Engine infer state and plans. Unified Outlet applies typed bounded parameters to existing retrieval, ranking, and reranking; underlying models still score items. Executability does not establish effectiveness without observed feedback.
Illustrative example: Control adjusts category weights and diversification for camping interest without replacing the model that scores every candidate.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why does DREAM translate bounded LLM semantic levels into parameters deterministically?
DREAM: A Dual-Loop Recommendation Evolution Framework Powered by Large Language Models
Show answer and example
Answer: Free-form text cannot reliably meet service schemas and ranges. Separate intent, planning, and compilation; the model proposes permitted directions and levels, while one outlet enforces whitelists and bounds. Only verified feedback enters policy memory. The LLM cannot arbitrarily rewrite production or amplify weights.
Illustrative example: An increase-diversity level compiles to a bounded diversification count instead of executing a vague instruction to promote novel content.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Why is DREAM's shadow online gate not a real production A/B test?
DREAM: A Dual-Loop Recommendation Evolution Framework Powered by Large Language Models
Show answer and example
Answer: Independent shadow data reduce offline selection overfitting, but the public proxy and fixed reaction environment lack actual user feedback. Production double loops need real experiments. Local relevance improved while concentration rose and novelty fell; report all separately.
Illustrative example: Splitting validation users prevents repeated tuning on one group, but cannot measure adaptation, churn, or week-long live engagement.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
How does DREAM compile semantic levels into legal, bounded parameters?
DREAM: A Dual-Loop Recommendation Evolution Framework Powered by Large Language Models
Show answer and example
Answer: Discrete levels must go through whitelists and deterministic translation. Ranking overrides use original weights and levels; category diversification count is `max(1, default_count + bc)`. Record executed parameters, not an unexecuted proposal, so feedback is attributable.
Illustrative example: A default count of one plus a level of −2 still compiles to at least one, never a negative serving parameter.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why study noise as pairwise preference flips in DrEM?
Show answer and example
Answer: Rankings depend on relative order; small errors can swap close candidates. Logit perturbation, preference-preserving filtering, and risk correction target this instability, while consistency fusion limits drift. Accurate individual predictions do not guarantee stable pairs, and flip estimates can be wrong.
Illustrative example: Scores only 0.01 apart can trade first place under small noise despite small individual errors.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why does DrEM correct preference-flip risk instead of averaging prediction scores?
Show answer and example
Answer: Prediction noise can reverse pair preferences, and averaging can still learn the wrong order. DrEM selects perturbation-stable pairs, corrects the objective for flip risk, and applies consistency fusion to limit ranking drift. Correction assumes a noise estimate and becomes unstable near its boundary.
Illustrative example: A small CTR perturbation can reverse a close video pair; model that risk rather than trusting the observed order as a clean label.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
What assumptions does DrEM risk inversion add beyond perturbation augmentation?
Show answer and example
Answer: Augmentation exposes the model to noise. Inversion combines original and reversed-label losses according to estimated flip risk to recover the clean objective. It needs valid risk estimates and a nonsingular denominator; errors leave bias or amplify variance. Noise alone is not inversion.
Illustrative example: Correcting a true 40% flip rate as 20% remains biased. Random perturbations do not supply the inversion coefficient.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why separately check probability-to-logit conversion and `1−2ρ` in DrEM?
Show answer and example
Answer: Probabilities of zero or one cause infinite logits, while flip rate ρ near one half makes the inversion denominator singular. Document handling rather than silently producing huge gradients, and verify filtering, correction, and consistency fusion separately.
Illustrative example: A coin-flip preference contains no recoverable signal merely because it is divided by a tiny denominator.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why make utility weights the action instead of generating items in DRL-PUT?
Deep Reinforcement Learning for Ranking Utility Tuning in the Ad Recommender System at Pinterest
Show answer and example
Answer: Existing CTR, CVR, relevance, and revenue models score candidates. The policy chooses a controlled mix, and feedback evaluates the induced list, not isolated causal item effects.
Illustrative example: A 7:2:1 versus 4:3:3 mix changes order while leaving item embeddings and the click model unchanged.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Does DRL-PUT learn which items to recommend or how to combine utility scores?
Deep Reinforcement Learning for Ranking Utility Tuning in the Ad Recommender System at Pinterest
Show answer and example
Answer: Actions specify weights for relevance, CTR, CVR, and related scores. A request-conditioned policy combines existing predictors, while offline updates correct action distribution differences using logging probabilities. Rarely logged weights lack evidence for claims of live optimality.
Illustrative example: Keep ad representations fixed but emphasize relevance for exploration and conversion for purchase intent.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does DRL-PUT off-policy updating differ from fitting high-reward logged examples directly?
Deep Reinforcement Learning for Ranking Utility Tuning in the Ad Recommender System at Pinterest
Show answer and example
Answer: Logs are generated by μ and target choices follow π; the probability ratio addresses differing action frequencies. Without it, frequently tried actions dominate. Valid support and logging are required, extreme ratios increase variance, and public proxies are not randomized ad logs.
Illustrative example: If the old system nearly always emphasized CTR, most high rewards may follow that choice without showing alternatives are worse.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why must action identities match logged propensities in DRL-PUT's `π/μ`?
Deep Reinforcement Learning for Ranking Utility Tuning in the Ad Recommender System at Pinterest
Show answer and example
Answer: Keep executed weights, behavior probability, context, and reward tied to one event. With μ=0, no support exists; a tiny replacement does not enable unbiased extrapolation. Baseline b cannot read test outcomes, and final lists alone cannot reconstruct the update.
Illustrative example: Logging the propensity of 4:6 for an executed 7:3 mix gives the wrong gradient multiplier despite a genuine reward.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why does Dual-Rerank pair an AR teacher with an NAR student?
Dual-Rerank: Fusing Sequential Dependencies and Utility for Generative Reranking
Show answer and example
Answer: The teacher learns prefix and order dependencies item by item. The student estimates positions in parallel using distilled distributions and utilities, avoiding teacher calls at serving. Check deduplication and position consistency rather than assuming full teacher capability is preserved.
Illustrative example: The student can learn basic-then-advanced tutorial ordering and score positions together.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
How does Dual-Rerank's parallel NAR student absorb an autoregressive teacher's ordering dependencies?
Dual-Rerank: Fusing Sequential Dependencies and Utility for Generative Reranking
Show answer and example
Answer: The AR teacher models prefix-dependent item interactions at sequential serving cost. Ranking distribution and utility distillation compress its ordering preferences into parallel rank estimates. The student approximates rather than recomputes every prefix-dependent decision.
Illustrative example: Learn that a tutorial before equipment raises list value and emit the list once; novel combinations may still differ from the teacher.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
What trade-off does utility distillation allow beyond matching AR probabilities?
Dual-Rerank: Fusing Sequential Dependencies and Utility for Generative Reranking
Show answer and example
Answer: KL matches distributions, while utility lets quality and serving constraints enter the student's objective so it can deviate where appropriate. Bad utility proxies can harm quality, and a latency term is not proof of actual speed. Measure both contributions separately.
Illustrative example: The student may preserve high-value list relationships without matching tail probabilities exactly; speed alone can reward irrelevant output.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why align candidates, positions, and masks for Dual-Rerank's AR–NAR KL?
Dual-Rerank: Fusing Sequential Dependencies and Utility for Generative Reranking
Show answer and example
Answer: The teacher is prefix-constrained and the student may see all candidates in parallel. Compare the same event space with matching masks for padding and selected items and matching candidate identities. Copying final teacher scores to each position does not teach order.
Illustrative example: A step-two teacher excludes the movie at step one; a full-pool student distribution has a different denominator until explicitly aligned.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why is a full-page GenPage action closer to the homepage objective than per-item prediction?
GenPage: Towards End-to-End Generative Homepage Construction at Netflix
Show answer and example
Answer: Page utility depends on content types, layout, and interactions. Individually relevant items can form repetitive pages. Condition generation on the selected prefix and evaluate overall reward, accepting harder search and credit assignment without assuming global optimality.
Illustrative example: A page of one movie franchise can be individually relevant but provide no exploration entry points.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why does GenPage post-train full-page generation on long-term reward rather than simply imitate past pages?
GenPage: Towards End-to-End Generative Homepage Construction at Netflix
Show answer and example
Answer: Cloning or SFT reproduces logged display behavior, not necessarily long-term effects of positions and combinations. Page-level reward evaluates the full action while post-training respects business feasibility and cold start. Public proxy lists are not production long-term retention evidence.
Illustrative example: Imitation copies rows of popular items. Adding novel content requires real evidence of sustained exploration, not just more diverse titles.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
What evidence do GenPage WBC/SFT and long-term RL use?
GenPage: Towards End-to-End Generative Homepage Construction at Netflix
Show answer and example
Answer: Weighted demonstrations build a usable policy from existing pages. RL adjusts beyond imitation using reliable long-term environments and rewards. Movie proxies do not recover retention, and business and cold-start constraints need separate enforcement.
Illustrative example: Learn a sensible layout quickly from logs; explore more only when long-term feedback supports it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why is GenPage's MovieLens core not Netflix's production full-page implementation?
GenPage: Towards End-to-End Generative Homepage Construction at Netflix
Show answer and example
Answer: Public data lack real layout, multiple content formats, and long-term rewards. The fixed candidate protocol only checks actual implemented proxy states, constraints, and updates; a ranked list is not a homepage. Preserve negative local results independently of paper gains.
Illustrative example: A ten-movie list has no carousel, copy, or multi-day reactions and cannot verify the production UI or retention policy.
Sections in the Chinese source: 本地复现. The example illustrates the mechanism.
Basics
Why separate GESE title exploration from personalized selection?
Show answer and example
Answer: Generate a relevant, diverse, faithful pool, then choose by current user and context. The selector cannot repair a pool lacking faithful options, and catchy wording does not imply satisfaction.
Illustrative example: Title variants can emphasize method, setting, or conclusion, but none may invent results absent from the article.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why does GESE explore diverse faithful titles and then select one for current context?
Show answer and example
Answer: One fixed title lacks personalization; online free generation costs compute and can misrepresent content. Reusable candidate creation and real-time selection are separate, and diversity cannot justify false claims. Fixed-candidate local scoring verifies selection only.
Illustrative example: A documentary can have faithful titles about photography or natural history, but not invented action scenes for clicks.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does GESE allocate compute compared with using an LLM to write each user's title in real time?
Show answer and example
Answer: Reusable candidates amortize expensive generation across requests and leave real-time adaptation to a lightweight selector. Per-user generation is more flexible but adds latency, repeated cost, and consistency risks. Pre-generation limits coverage; deterministic local scoring does not establish production speed or quality.
Illustrative example: Generate five faithful angles once and let a thousand users run only selection. An angle outside the pool cannot be created by the selector.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why is GESE's local candidate scorer not full LLM generation training?
Show answer and example
Answer: It uses deterministic relevance, diversity, faithfulness, and contextual selection to validate the selection mechanism, not full set generation or production post-training. Report the implemented operators and negative results; paper online metrics are not local measurements.
Illustrative example: Selecting among five hand-written titles verifies a selector, not a model's ability to generate those faithful titles.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why can GloRank select items outside the local candidate pool with global SIDs?
From Local Indices to Global Identifiers: Generative Reranking via Global Action Space
Show answer and example
Answer: Global identifiers define actions over the catalog instead of the current list, adding nomination to reranking. This expands coverage while increasing search, legal mapping, and list-objective difficulty. Fixed-candidate assumptions no longer describe all results.
Illustrative example: A niche book omitted from five popular local candidates can be nominated globally if legal and relevant.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why does GloRank move beyond ordinary reranking when it replaces local IDs with global SIDs?
From Local Indices to Global Identifiers: Generative Reranking via Global Action Space
Show answer and example
Answer: Local IDs only address upstream candidates. Global SIDs allow a broader catalog, so training learns nomination as well as list utility. Larger action spaces complicate search and legality and require evaluation different from closed-pool reranking.
Illustrative example: Index 11 is invalid in a ten-item local list; a global code can nominate a new item only if it maps to a serveable catalog entry.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Why state candidate access when comparing global generation and local reranking?
From Local Indices to Global Identifiers: Generative Reranking via Global Action Space
Show answer and example
Answer: Full-catalog generation can improve coverage independently of ordering. Define full or local evaluation, legal mappings, and reward protocols. Different access is not a single-component ablation, and preserve local negative results.
Illustrative example: Finding the right book outside a fixed pool does not show a better reranker; the local model could not choose it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why can't request-local indexes replace global SIDs in GloRank listwise SFT and group rewards?
From Local Indices to Global Identifiers: Generative Reranking via Global Action Space
Show answer and example
Answer: Local indexes change with candidate order and cannot identify catalog items consistently. Train and decode mapped global paths, and score actual generated identities without fixing invalid codes using test answers. Do not claim full global language training from a proxy.
Illustrative example: Index three can mean camera in one request and tent in another, so it is not a stable global token.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why move from classification scores to rank relationships in HarmonRank?
Show answer and example
Answer: Different target base rates and scales can dominate arithmetic fusion. Differentiable rank-sum/AUC first emphasizes each task's internal order, then learns task dependencies. Ranking loses absolute value magnitude rather than preserving every business unit.
Illustrative example: A purchase score can be numerically tiny but still rank items meaningfully, without saying how much more value one brings.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why align task ranks before fusing raw predicted probabilities in HarmonRank?
Show answer and example
Answer: Different base rates and scales make raw fusion sensitive to calibration and sparsity. Differentiable rank-sum learns within-task order; inter-task attention models dependencies such as purchases and comments before unified ranking. Rank alignment discards magnitude and is not a general probability calibrator.
Illustrative example: Purchase probabilities are much smaller than click probabilities; ranking candidates within each task reduces raw-scale dominance.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does inter-objective self-attention handle conflicts differently from fixed fusion?
Show answer and example
Answer: Fixed coefficients impose a global trade-off. Attention learns dependencies and conditional combinations, increasing flexibility but risking domination by frequent objectives. Check by task and example; the local negative result does not establish quality benefits.
Illustrative example: A highly clicked but rarely purchased item can be treated differently from one high on both, instead of applying the same 7:3 ratio.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why require both classes and identify invalid tasks in a soft-AUC batch?
Show answer and example
Answer: AUC needs comparable positive-negative pairs; one-class slices must be masked or reported not estimable, not scored as perfect. Align candidate IDs across tasks. Temperature and sampling affect the differentiable approximation and its gradients, not actual full-data AUC.
Illustrative example: An all-zero purchase batch has no pairs and must not send a perfect score into task fusion.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why choose context-tree granularity dynamically in HCUB?
Hierarchical Contextual Uplift Bandits for Catalog Personalization
Show answer and example
Answer: Fine nodes are relevant but sparse; parents are better sampled but coarse. Reward inheritance handles cold starts and yields to local data. Tree design and mixing affect bias, so more partitions do not ensure accuracy.
Illustrative example: Borrow a parent estimate after two events, then rely more on the child after 100 observations.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why does HCUB inherit parent rewards in sparse contexts while preserving exploration?
Hierarchical Contextual Uplift Bandits for Catalog Personalization
Show answer and example
Answer: Fine-grained estimates have high variance. Parent rewards give a stable starting point, and local data gradually gain weight. Choose by incremental value plus uncertainty to avoid early lock-in. Hierarchical sharing can mask minorities, so do not replace all local evidence with parent averages.
Illustrative example: A sparse device group borrows regional experience until feedback reveals its own preferences, with exploration helping discover that difference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does HCUB incremental reward plus UCB differ from greedy absolute CTR?
Hierarchical Contextual Uplift Bandits for Catalog Personalization
Show answer and example
Answer: Increment measures change relative to a reference, and the exploration bonus addresses uncertainty. CTR greedy favors already responsive groups. Increment needs suitable causal data; confidence bonuses are not identification, and movie proxies do not establish interventions.
Illustrative example: An active group might click anyway, while a less active group may respond more to the action.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
How should HCUB avoid propagating one sparse child's feedback to every node?
Hierarchical Contextual Uplift Bandits for Catalog Personalization
Show answer and example
Answer: Define local statistics and parent prior update scopes. Reading a mixture does not copy one outcome across the tree. Use defined η and uncertainty, retain action identities, and do not invent failures for unobserved arms.
Illustrative example: Update a child's observed success and permitted ancestor aggregates without marking every sibling as a success.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why choose semantic clusters before items in HiGR's page generation?
HiGR: Industrial-Scale Hierarchical Generative Slate Recommendation Framework in Tencent
Show answer and example
Answer: Hierarchy divides a large space into coarse interests and precise identities, enabling composition across groups. Wrong coarse codes restrict later candidates, and apparent diversity does not prove relevance. Evaluate both quantization and whole-page utility.
Illustrative example: Choose tutorial, tool, and example groups before specific items, while recognizing a wrong entertainment branch can remain diverse but irrelevant.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why does HiGR need whole-page preference alignment after choosing semantic clusters then items?
HiGR: Industrial-Scale Hierarchical Generative Slate Recommendation Framework in Tencent
Show answer and example
Answer: Hierarchical codes make identification easier but individual relevance does not ensure useful order or complementarity. The Hierarchical Slate Decoder generates coarse-to-fine choices; ORPO reshapes combinations from preferred and rejected pages. Evaluated preferences do not guarantee long-tail or utility gains.
Illustrative example: Sports clusters can yield only near-duplicate shoes; page preferences can add socks or training content.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does HiGR ORPO preference evidence differ from page-level SFT?
HiGR: Industrial-Scale Hierarchical Generative Slate Recommendation Framework in Tencent
Show answer and example
Answer: SFT imitates positive pages. ORPO compares preferred and rejected page odds to teach relative composition. Preference and quantization biases remain, and local NDCG declined; report it with head-share changes instead of declaring overall improvement.
Illustrative example: An equally relevant but more complementary page can beat a repetitive one; a winner-only target provides no explicit relative signal.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why define HiGR sequence probabilities and numerical boundaries before calculating log odds?
HiGR: Industrial-Scale Hierarchical Generative Slate Recommendation Framework in Tencent
Show answer and example
Answer: Page probability is a product of conditional token probabilities and can underflow; use stable log-space arithmetic for both the joint and its complement. Arbitrary scores are not probabilities. Match hierarchical mappings and repetition constraints to page identity.
Illustrative example: Taking logs after a long-page probability rounded to zero fails; silently replacing the sequence probability with a length average changes the objective.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why does LTV Video Ranking correct position before assigning long-term session value?
A Long-term Value Prediction Framework In Video Ranking
Show answer and example
Answer: Position affects views and interactions. PDQ aligns outcomes across positions before aggregating session and cross-day creator value to avoid confusing exposure with contribution. Quantile correction relies on distributional assumptions and does not remove all exposure confounding.
Illustrative example: Illustration: The first video is easier to see than the fifth. Match position conditions before attributing subsequent consumption to creator quality.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why correct position bias before including session and cross-day creator value in LTV video ranking?
A Long-term Value Prediction Framework In Video Ranking
Show answer and example
Answer: Top positions generate more views, so raw engagement can reward exposure rather than long-term contribution. Correction aims for comparable feedback before session attribution and creator-cycle objectives assign longer-horizon value. Quantile mappings and attribution have assumptions; their combined score is not identified causal retention lift.
Illustrative example: Illustration: A creator always shown first gets more clicks than one shown lower. Align exposure opportunity before judging who drives sustained consumption.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does long-term creator value differ from ranking by seconds watched on one video?
A Long-term Value Prediction Framework In Video Ranking
Show answer and example
Answer: Watch time rewards immediate consumption; a cross-day creator objective may reward consistent supply rather than one attention-grabbing video. Define how to combine the two and check whether the long-term proxy favors incumbent creators. The public proxy lacks real creator cycles and does not establish retention or ecosystem gains.
Illustrative example: Illustration: A sensational video can hold attention once, while a reliable series brings people back for days. Watch time alone can miss the latter contribution.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why must PDQ distinguish the original position distribution from the reference distribution?
A Long-term Value Prediction Framework In Video Ranking
Show answer and example
Answer: Find the outcome's quantile in its original position's conditional distribution, then invert it in a common reference distribution. Inverting in the original distribution would nearly recover the original value. Fit distributions from allowed data and do not tune on future test behavior; without real position logs this is a proxy.
Illustrative example: Illustration: Map a fifth-position video's top-decile watch time to the reference position's top-decile value, not back to the fifth position.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why does MAPS split creative generation into Profile, Scripter, Director, and Reflector?
Show answer and example
Answer: The roles extract needs and style, write product messaging and scenes, execute video, and route feedback. This localizes errors rather than rebuilding all stages. Role names do not confer capability; real semantic models, video generation, and a credible environment are needed. The local scope is review-only for cost.
Illustrative example: Illustration: Fix copy that overclaims, or fix video execution that violates the script, rather than forcing the profiling agent to rewrite a correct preference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why does MAPS's Reflector separate profile errors from script errors and route a minimal edit?
Show answer and example
Answer: Poor feedback can arise from misunderstanding the user or from messaging and factual mistakes. Rebuilding everything wastes compute and can damage correct stages. Role attribution targets the relevant stage and keeps profile, script, and video generation auditable. Mixed rewards cover structure, personalization, and factuality. Large models and private environments make this review-only; there is no keyword substitute.
Illustrative example: Illustration: Fix a script that overstates product performance when the profile is right. If the gift recipient was misunderstood, polishing shots cannot fix targeting.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
What does MAPS's mixed reward measure that keyword counts cannot?
Show answer and example
Answer: Structure includes scene roles and physical plausibility; semantic judges score personalization and product consistency. Keyword coverage cannot detect contradictions or user fit. A proxy can test interfaces but is not algorithm evidence. Reward weights do not guarantee judge quality, and text scores differ from creative outcomes.
Illustrative example: Illustration: 'Durable' and 'perfect for you' can satisfy keywords while calling ordinary shoes fireproof. Product consistency must check facts.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why doesn't MAPS perform effective RL when group feedback never differs?
Show answer and example
Answer: Scripts in the same context need different rewards. Identical rewards yield zero centered advantage and no preference direction. The actual judge and video pipeline also matter. The source explicitly avoids a keyword proxy; evaluating the formula alone does not reproduce training or the environment.
Illustrative example: Illustration: If a crude rule gives all four scripts full marks, an optimizer call cannot teach which is more personalized or establish improved four-agent video creation.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why put objective weights, preferences, constraints, and position rules into typed JSON in MetaStrategy?
MetaStrategy: Generative Ranking with Executable LLM Strategies
Show answer and example
Answer: A fixed scorer is hard to steer on demand, and free text is difficult to execute reliably. Typed parameters separate semantic planning from deterministic ranking. Validity is only a format gate; still check ranges, replay, and feedback.
Illustrative example: Illustration: Raising a freshness weight with a placement limit is executable, unlike 'show more new items,' but it may still be bad for the user.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why does MetaStrategy generate typed policies rather than final item order?
MetaStrategy: Generative Ranking with Executable LLM Strategies
Show answer and example
Answer: Typed objective weights, category preferences, and position constraints are explicit and compiled deterministically against real candidates. This bounds execution and makes failures auditable. The LLM interprets intent and the compiler executes it; evaluate them separately. Valid JSON proves interface compliance, not utility, and self-scores do not replace replay feedback.
Illustrative example: Illustration: For less repetition, emit a bounded category-diversity rule. If matching candidates are absent, expose the limit rather than inventing items.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
What is auditable and reusable in MetaStrategy compared with direct item-list generation?
MetaStrategy: Generative Ranking with Executable LLM Strategies
Show answer and example
Answer: A list is bound to its candidates, whereas a policy can be inspected and reapplied after candidates change. Compiler expressiveness limits which preferences it can encode. The typed-bundle proxy is not full production distillation or equivalent intelligence.
Illustrative example: Illustration: 'At most one per author' works across candidate lists; a fixed ten-book list goes stale. The rule cannot interpret all content semantics.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why must MetaStrategy's compiler reject unknown fields and out-of-range values?
MetaStrategy: Generative Ranking with Executable LLM Strategies
Show answer and example
Answer: Otherwise generated policies can escape the allowed space, breaking equivalence between evaluation and serving. Bind the schema and defaults to actual output and record failures rather than guessing intent. A JSON file does not implement online reward routing or teacher–student OPD.
Illustrative example: Illustration: Reject 'increase purchase weight without limit' and unknown fields rather than interpreting them as arbitrary executable expressions.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why does OGR's TUSID encode semantics and collaboration instead of only quantizing content?
Show answer and example
Answer: Content generalizes to new items; collaborative codes preserve consumption relations. GL2P plans a slate and SPA adapts from feedback, distinct from coding. Richer codes can conflict and do not guarantee better pages.
Illustrative example: Illustration: Visually different items bought together can connect via collaboration; new items can borrow semantic structure.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why fuse semantic and collaborative codes before conservative slate optimization in OGR?
Show answer and example
Answer: Semantic codes share content structure and collaborative codes capture behavior relations; encoding organizes the generative space. Planning treats a slate as one action to model item combinations, and conservative feedback updates limit drift from supervision. Representation, joint action, and stable learning are separate.
Illustrative example: Illustration: Four individually strong shoes can be redundant, while a slate can combine shoes, socks, and a pack without jumping to an unfamiliar combination for one reward.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Why does conservative SPA keep an SFT anchor rather than maximize reward freely?
Show answer and example
Answer: Sparse or misspecified rewards can favor extreme slates. KL limits drift from supervised behavior, trading improvement freedom for stability without fixing bad rewards. Local NDCG and Hit moved in different directions, not universal gains.
Illustrative example: Illustration: Novelty alone may favor irrelevant tail items; SFT preserves interest, but excessive constraint may block exploration.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why must OGR's R(Y) and KL compare the whole slate rather than one item?
Show answer and example
Answer: Y is the slate, so reward, generation probability, and reference must apply to the same list or defined token decomposition with consistent IDs and deduplication. Mean item reward misses interactions and fixed score mixing is not policy updating. State the actual local path.
Illustrative example: Illustration: Five individually relevant duplicates can have high mean item score but poor slate value.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why does PIANO's IAN read the entire candidate list?
PIANO: Personalized Reranking via Information Aggregation Node for Music Search Optimization
Show answer and example
Answer: Choice depends on substitutes and competitors on the page. Aggregation conditions item CTR/CVR on the current alternatives, while query-related history represents needs. This differs from static item value and changes when the list changes.
Illustrative example: Illustration: A mid-priced shoe can look practical among expensive models but not among cheap ones.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why extract query-relevant history and condition item predictions on the entire slate in PIANO?
PIANO: Personalized Reranking via Information Aggregation Node for Music Search Optimization
Show answer and example
Answer: History selection captures current search intent; slate aggregation tells each CTR/CVR prediction which alternatives surround the item. They model different relationships. A change in candidates can change every score, so these are not independent cacheable item values.
Illustrative example: Illustration: A hiking-shoe search reads outdoor history, while the same shoe has different appeal among premium models versus beginner models.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does PIANO list conditioning change caching and evaluation versus pointwise CTR?
PIANO: Personalized Reranking via Information Aggregation Node for Music Search Optimization
Show answer and example
Answer: Independent scores can be reused for fixed user–item inputs; list-conditioned scores also depend on candidate membership. Extra context may help competition modeling but costs compute and is distribution-sensitive. The public proxy was negative, not universal superiority.
Illustrative example: Illustration: Removing the cheapest item can alter all other predictions. Reusing old scores would ignore the new IAN context.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
How avoid leaking gold target position through PIANO list aggregation?
PIANO: Personalized Reranking via Information Aggregation Node for Music Search Optimization
Show answer and example
Answer: The candidate list is allowed, but never order or flag it using real clicks. Labels enter loss only. Mask padding and candidate count, and query only past history. Otherwise IAN can learn a position shortcut.
Illustrative example: Illustration: Always putting the correct product first inflates test results if that layout is repeated; use a serving-equivalent unlabeled list.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why use component-specific towers for images, titles, and layout in PinDCO?
PinDCO: Whole-Page Aware Dynamic Creative Optimization at Scale
Show answer and example
Answer: Components differ in modality and statistics. Separate encoders retain roles before combining them to predict increments for one base ad. Interaction remains necessary to score combinations. A predicted lift is not randomized causal evidence.
Illustrative example: Illustration: Changing a title and changing image shape have different effects; an attractive image can still mismatch the copy.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why predict creative lift over a base ad and penalize pixel use in PinDCO?
PinDCO: Whole-Page Aware Dynamic Creative Optimization at Scale
Show answer and example
Answer: Base scores already capture product–user match; increments isolate predicted image, title, and layout effects. Tall creatives consume more feed space, so a pixel adjustment weighs value against footprint. Exploration gathers new feedback. The local pixel-free proxy is not causal layout evidence.
Illustrative example: Illustration: A taller image can gain clicks but displace content. Prefer it only when added value offsets space cost.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does PinDCO ε-greedy help new creatives compared with giving all traffic to historical winners?
PinDCO: Whole-Page Aware Dynamic Creative Optimization at Scale
Show answer and example
Answer: Greedy exposure starves new creatives of evidence. Limited random exploration provides less-selected feedback for component learning, but costs traffic and does not remove all confounding. Pixel adjustment separately controls space; retain catalog and factual safeguards.
Illustrative example: Illustration: A new title with no clicks may simply lack exposure. Explore modestly without showing clearly misleading copy.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why preserve PAM clipping and the precise definition of relative aspect ratio r?
PinDCO: Whole-Page Aware Dynamic Creative Optimization at Scale
Show answer and example
Answer: P(r)=clip(1−tanh(k(r−1)),0,1) penalizes tall creatives within a legal range and multiplies base objective O. Inverting the ratio or omitting clipping changes the behavior. Local content norm is a pixel-free proxy, not real layout efficiency.
Illustrative example: Illustration: If r means height over width, using width over height reverses the screen-space penalty.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why can primal–dual decoding work without retraining a generator?
Stochastic Primal-Dual Decoding for Multiobjective Generative Recommender Systems
Show answer and example
Answer: Serving search combines original relevance with auxiliary scores and updates multipliers while leaving weights unchanged. It cannot create knowledge missing from the candidates or generator. Measure runtime, relevance loss, and constraint satisfaction separately.
Illustrative example: Illustration: Increase new-author weight when lacking one, without changing the base model's scores.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why adjust primal–dual decoding multipliers dynamically rather than add a fixed diversity bonus?
Stochastic Primal-Dual Decoding for Multiobjective Generative Recommender Systems
Show answer and example
Answer: Fixed weights cannot react to remaining slack. Per-position scoring combines model relevance with auxiliary contribution and adjusts multipliers so later choices can compensate. No retraining is needed, but candidate availability and local search limit results, with possible relevance loss and no guarantee for arbitrary hard constraints.
Illustrative example: Illustration: If four of five slots include only one new author but two are desired, raise the last-slot incentive; reduce pressure once satisfied.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Can primal–dual soft objectives replace hard eligibility constraints?
Stochastic Primal-Dual Decoding for Multiobjective Generative Recommender Systems
Show answer and example
Answer: No. Multipliers shift preferences but unavailable or out-of-catalog items must be explicitly excluded. A larger multiplier cannot create feasible items. Soft optimization trades objectives within hard eligibility; no arbitrary-constraint guarantee is reported.
Illustrative example: Illustration: Encourage new authors softly, but mask out-of-stock items rather than let them win after a penalty.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why keep mirror-descent multipliers nonnegative and control exponent ranges?
Stochastic Primal-Dual Decoding for Multiobjective Generative Recommender Systems
Show answer and example
Answer: Nonnegative multipliers preserve penalty direction, and positive initialization with multiplicative updates preserves sign. Large steps can overflow or become extreme. Match auxiliary units, R/K targets, and relevance scores; follow actual probability/logit conventions rather than interchanging them.
Illustrative example: Illustration: Raw seconds mixed into probabilities can dominate. Repeated exponential increases can pursue quota at the expense of relevance even without an error.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Does Pareto non-dominance in PRL-PUTS require improvement on all objectives?
Show answer and example
Answer: No. A policy remains when none is at least as good on every goal and better on at least one. Different Repin/P2P trade-offs can be nondominated. Suitability still depends on business needs, constraints, and estimation error.
Illustrative example: Illustration: (8,3) and (6,5) trade off while (5,2) is dominated. The first two are not necessarily better than production.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why preselect PRL-PUTS weights via two Q heads and a Pareto sweep?
Show answer and example
Answer: Each head estimates a business utility and weight mixtures choose trade-offs. An offline sweep drops dominated strategies and yields a governable frontier; serving can switch weights without swapping models. Q estimation error and drift make offline non-dominance insufficient for online superiority.
Illustrative example: Illustration: Keep one policy strong in saves and another strong in deeper interactions, and discard one worse in both rather than testing arbitrary weights on traffic.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
What choices does Pareto sweeping retain beyond one highest weighted PRL-PUTS score?
Show answer and example
Answer: One weighting selects one compromise. Sweeping and removing dominated strategies retains options for changing goals. Offline noise can create false frontier points, so uncertainty and online tests matter.
Illustrative example: Illustration: Keep stronger-save and stronger-engagement policies, but reject one worse on both rather than promoting it for an arbitrary score.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why hold Q model and action-space versions fixed when switching PRL-PUTS weights?
Show answer and example
Answer: Weight effects are identifiable only over the same comparable Q values and action set. Keep α in range and record each head's units and reward definition. A fixed local scorer does not reproduce full long-term Q-learning and online governance.
Illustrative example: Illustration: Switching 70/30 to 50/50 while retraining a Q head confounds attribution.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Does lossless PSG expansion mean pair and item models have the same probabilities?
PSG: Pair-Space Generation for Efficient Generative Reranking
Show answer and example
Answer: No. The mapping recovers both items and their order for a given pair; the scorer and search distribution can still change. Fewer steps do not imply equal training error, coverage, or quality.
Illustrative example: Illustration: Packing A B C D as AB and CD recovers it exactly, but the new model can prefer other pairs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why do PSG pair tokens reduce steps but need internal order?
PSG: Pair-Space Generation for Efficient Generative Reranking
Show answer and example
Answer: Each token jointly models two positions and can be losslessly expanded, halving autoregressive steps for even lengths. Pair order affects display and future conditions, while cross-pair duplicates can invalidate the slate. Some complexity moves to the larger pairing/scoring space.
Illustrative example: Illustration: Six items take three ordered pairs. Coffee machine then beans differs from beans then machine; another pair containing the machine needs duplicate handling.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
What dependency can PSG pair scoring capture beyond parallel individual item heads?
PSG: Pair-Space Generation for Efficient Generative Reranking
Show answer and example
Answer: Joint scoring evaluates two items and their order as one action, modeling complementarity. Parallel independent heads may not. The pair action space can be large, and the proxy regressed, so half-length alone is not lossless speedup.
Illustrative example: Illustration: Camera plus lens can be better than two redundant high-scoring cameras, but all possible pairs outnumber individual items.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why deduplicate across PSG pairs instead of only within each pair?
PSG: Pair-Space Generation for Efficient Generative Reranking
Show answer and example
Answer: Local uniqueness does not prevent repeats across a slate. Check new pairs against the selected prefix and preserve internal order when expanding. Define odd lengths, leftovers, and empty legal sets instead of repeating items to fill slots.
Illustrative example: Illustration: Camera–lens and camera–bag are each unique internally but repeat the camera after expansion.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why combine structured property attributes with conversational needs instead of just listing prose?
LLM-Based Re-Ranking for Real Estate Search
Show answer and example
Answer: Descriptions can be vague; structured price, size, and commute specify constraints, while dialogue gives priorities and trade-offs. Set statistics add context. Language reasoning cannot replace factual attribute validation or rewrite facts.
Illustrative example: Illustration: Both listings say convenient; five versus twenty minutes to transit resolves a ten-minute requirement.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why include candidate-set statistics in real-estate LLM reranking rather than score each listing independently?
LLM-Based Re-Ranking for Real Estate Search
Show answer and example
Answer: Conversation needs can trade off budget, commute, and space across structured data and text. Set statistics provide current alternatives for relative decisions beyond vector similarity. List changes can invalidate scores, and reranking cannot recover missing listings.
Illustrative example: Illustration: If none is both quiet and near transit, the set helps identify the best compromise instead of matching each sales description alone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Why can't LLM relative reranking replace vector retrieval, or vice versa?
LLM-Based Re-Ranking for Real Estate Search
Show answer and example
Answer: Retrieval cheaply builds a pool from a large catalog but can miss complex needs. LLM reranking compares a small pool in detail but cannot choose absent items. Set conditioning adds cost; local movies lack real dialogue and housing constraints.
Illustrative example: Illustration: If the best school-and-commute match misses the top 100, reranking explanations cannot restore it.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Why recompute Agg(X) whenever a real-estate candidate set changes?
LLM-Based Re-Ranking for Real Estate Search
Show answer and example
Answer: It summarizes the current pool. Adding or removing listings changes relative budget and feasible options. Align structured fields and descriptions by identity and never choose candidates by real transaction labels. A local pointwise scorer is not full conversational reranking.
Illustrative example: Illustration: An initially cheap listing may cease to look cheap after adding lower-priced homes. The old mean misrepresents context.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Does each SAFRO rollout have one reward or several task rewards?
SaFRO: Satisfaction-Aware Fusion via Dual-Relative Policy Optimization for Short-Video Search
Show answer and example
Answer: rᵢ,ₜ indexes candidate and task separately. A candidate can score differently on immediate interaction and long-term signals; preserve that matrix for comparisons.
Illustrative example: Do not treat two objectives on one item as two items on one objective.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why normalize advantages within groups and across tasks?
SaFRO: Satisfaction-Aware Fusion via Dual-Relative Policy Optimization for Short-Video Search
Show answer and example
Answer: Within-group ranking identifies better candidates for one objective; cross-task terms compare objectives for the same candidate. Relation gates organize search objectives, limiting domination by high-scale rewards without defining long-term value or guaranteeing joint gains.
Illustrative example: Normalize dense clicks and sparse retention separately before balancing them.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Does standardization determine the relative importance of engagement and retention?
SaFRO: Satisfaction-Aware Fusion via Dual-Relative Policy Optimization for Short-Video Search
Show answer and example
Answer: No. It aligns scales, not business values. Relation gating combines goals without turning proxies into causal value. Specify reward definitions, axes, and fusion separately.
Illustrative example: Standardizing two exams does not determine their graduation weights.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Does local SAFRO on MovieLens reproduce short-video retention?
SaFRO: Satisfaction-Aware Fusion via Dual-Relative Policy Optimization for Short-Video Search
Show answer and example
Answer: No. Public full-catalog ranking tests proxy calculations, not private search, retention, or live buckets. Preserve negative results and do not call same-seed gaps online uplift.
Illustrative example: Offline movie hits are not next-day visits.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Does ordered regression predict item IDs or cumulative behavior thresholds?
A Generative Re-ranking Model for List-level Multi-objective Optimization at Taobao
Show answer and example
Answer: It predicts whether a prefix reaches each click/payment count. Threshold and prefix-length axes are distinct; score the full prefix after adding a candidate.
Illustrative example: For three items, outputs can mean at least one or two clicks, not item identities.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why estimate prefix value with ordered regression instead of summing item CTRs?
A Generative Re-ranking Model for List-level Multi-objective Optimization at Taobao
Show answer and example
Answer: Item scores miss list interactions. Predict cumulative behavior thresholds after adding a candidate. Target queues narrow search, batched masks evaluate, and MMR participates during selection. Proxy labels and search bounds do not establish global optimality.
Illustrative example: A third similar phone may add less engagement than headphones despite higher standalone CTR.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does MMR during generation differ from deduplication afterward?
A Generative Re-ranking Model for List-level Multi-objective Optimization at Taobao
Show answer and example
Answer: Choosing with marginal utility and similarity changes later prefixes. Removing duplicates only from the final list can leave holes or miss another generation trajectory.
Illustrative example: Deleting an early duplicate afterward does not recreate the alternative selection path.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Does local batched execution reproduce the paper's 19ms kernel?
A Generative Re-ranking Model for List-level Multi-objective Optimization at Taobao
Show answer and example
Answer: No. It is PyTorch masks without production optimization. MovieLens proxies clicks, trades, and upstream scores, not GMV. Align prefix targets and candidate masks.
Illustrative example: One batch call need not match a specialized kernel's device cost.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why model interference between ad positions?
Online Advertising with Spatial Interactions
Show answer and example
Answer: Neighboring ads affect value. Grouped disks and coverage constraints represent compatible selections, with LP and randomized rounding.
Illustrative example: Two nearby large ads can compete for attention, so values are not independent.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why check conflicts against the entire original sample?
Online Advertising with Spatial Interactions
Show answer and example
Answer: Rounding probabilities derive from independent sampling under geometric and group constraints. Greedy acceptance changes conditional distributions. Local tests verify actual LP/rounding, not advertising uplift.
Illustrative example: Accepting A first would change B's decision; the paper instead evaluates both against one fixed sample.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does LP with two-stage rounding differ from greedily placing ads?
Online Advertising with Spatial Interactions
Show answer and example
Answer: Solve the fractional program, test conflicts on the entire first-stage sample, then thin again. Greedy tests only already-kept items and changes probabilities and guarantees.
Illustrative example: A sampled item discarded later can still create a first-stage conflict.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
How handle disk boundaries and invalid probabilities?
Online Advertising with Spatial Interactions
Show answer and example
Answer: Cover only at strictly smaller distance than radius. Validate metric and thinning probabilities; fail rather than clip. Local HiGHS LP does not reproduce all auctions or the 2D PTAS.
Illustrative example: Probability above one cannot be silently clipped without changing the distribution.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why decide both whether to push and when to wake again?
A Self-Triggered Agentic Push Recommendation System
Show answer and example
Answer: Fixed frequency ignores user state; polling wastes work. Planning, execution, and cheap filters jointly handle timing and cost.
Illustrative example: A no-send decision should still specify when to reevaluate.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why use separate wake and send agents plus a filter?
A Self-Triggered Agentic Push Recommendation System
Show answer and example
Answer: Ordinal planning decides when to observe, trajectory reward drives sending, and a light filter rejects low-value or abnormal plans. This combines observation cadence and actions, without equating waiting value to message value or local ranking with retention.
Illustrative example: Delay after an ignored notification, then still decline sending; filter one-second wake loops.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
Are planning, execution, and filtering one threshold?
A Self-Triggered Agentic Push Recommendation System
Show answer and example
Answer: No. Planning predicts wake interval, execution decides send, and filtering controls cost and anomalous plans. Combining them into one probability removes scheduling and guardrails.
Illustrative example: Choosing tomorrow morning differs from choosing to send now.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Does local STEPS have real permissions and wall-clock wakeups?
A Self-Triggered Agentic Push Recommendation System
Show answer and example
Answer: No. MovieLens verifies decision decomposition, not push permissions, reach cost, clocks, or delivery. Actual service needs contracts and real rewards.
Illustrative example: Updating a simulated timestamp does not notify a phone.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Are SWAG-Bid episode boundaries and seven-day windows the same?
Show answer and example
Answer: No. Episodes bound local interactions; the sliding window carries longer goals/state across episodes, not seven fixed actions. Do not reset all long-term context at every boundary.
Illustrative example: A new bidding day does not restart a seven-day objective.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why use a seven-day plan across episodes and inject it through per-step gates?
Show answer and example
Answer: Local rewards can miss multi-day budgets. Sliding plans carry long goals; gates let immediate markets override guidance when appropriate. Masked future supervision is training-only, and a small zero-change result does not validate bidding gains.
Illustrative example: Save budget during a crowded day but act on a valuable current opportunity using only available information.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does masked future planning in training differ from deployment future access?
Show answer and example
Answer: Training can construct masked plan signals under protocol. Serving sees only then-available state and estimates, not realized later GMV or spend.
Illustrative example: Tomorrow's spend can supervise offline training but cannot be an input to today's bid.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Does local SWAG use real bids, budgets, and GMV?
Show answer and example
Answer: No. It ablates structure, not a production Decision Transformer. Maintain causal masks and timestamps, without filling from future outcomes.
Illustrative example: A movie action score is not an auction price.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why use both semantic and collaborative-ID rewards in TAIJI?
Show answer and example
Answer: Semantics explains content, while IDs reflect platform interactions. They can conflict. Activate recommendation reasoning and balance goals dynamically.
Illustrative example: Similar descriptions need not imply co-preference.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why dynamically balance semantic and collaborative rewards?
Show answer and example
Answer: Different examples need different content/behavior trade-offs. Fixed directions can miss useful nonconvex solutions. Reverse preference reasoning and rejection sampling precede multiobjective alignment, without guaranteeing true preferences.
Illustrative example: A cold item may rely on semantics while a popular one has rich interaction evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How do dynamic Pareto weights differ from fixed linear weights?
Show answer and example
Answer: A fixed mixture covers one trade-off and can miss nonconvex frontiers. Dynamic weights adapt, but without comparisons they do not establish global Pareto optimality.
Illustrative example: One user may value semantic novelty, another stable behavioral evidence.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
How isolate rejection sampling and weight fitting?
Show answer and example
Answer: Use permitted training preferences and validation; never use test feedback to choose reasoning or weights. Local MovieLens does not reproduce the real LLM, private codes, or serving.
Illustrative example: Selecting reasoning after reading a test click leaks the target.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why predict a mean and variance in UAME?
Show answer and example
Answer: Multiple proxy scores can conflict. A Gaussian pair probability uses uncertainty during training, while serving reads only the mean.
Illustrative example: High clicks and low watch can make relative preference less certain.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why learn mean and variance, emphasize conflict in training, and serve only means?
Show answer and example
Answer: Gaussian score comparisons express proxy disagreement. Train using analytic wins, weight uncertain pairs, and regularize variance. Mean-only serving keeps cost low. Modeled uncertainty is not measured satisfaction.
Illustrative example: A click-friendly and a long-watch video create an uncertain pair; train on it but serve one mean score.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does uncertainty weighting differ from averaging pxtr scores?
Show answer and example
Answer: It emphasizes conflicting pairs and models comparison probability, unlike a fixed average. Variance is not observed satisfaction, and larger weights do not solve every conflict.
Illustrative example: Two candidates strong on different metrics should not be collapsed without showing uncertainty.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Does UAME sample Gaussian noise online, and how interpret local negatives?
Show answer and example
Answer: Serving uses mean only. Proxy losses do not reproduce author gains or refute private tests. Handle variance and zero-range weighting explicitly.
Illustrative example: Equal uncertainties give a zero min–max denominator; do not produce NaNs.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Why separate training debiasing from serving exploration for freshness?
Breaking the Loop: An Empirical Comparison of Strategies for Novelty and Freshness in YouTube Music
Show answer and example
Answer: Historical exposure favors old items. IPS/bias towers change learning signals, while uncertainty exploration changes actual exposures. Age features alone do not fix feedback loops.
Illustrative example: Old songs have more data because they were shown; new songs need actual opportunities.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why separate debiasing and exploration instead of relying on freshness features?
Breaking the Loop: An Empirical Comparison of Strategies for Novelty and Freshness in YouTube Music
Show answer and example
Answer: Content age does not fix logged exposure selection. IPS and removable bias towers adjust training; SNGP uncertainty gives unknown items exposure online. Both depend on assumptions, and less head concentration does not prove satisfaction.
Illustrative example: A new video may lack clicks because it has never been shown; explore and collect feedback rather than boosting all new videos blindly.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Method comparison
How does a removable bias tower differ from retaining recency in serving?
Breaking the Loop: An Empirical Comparison of Strategies for Novelty and Freshness in YouTube Music
Show answer and example
Answer: A tower can absorb training exposure bias then be removed. Recency remains a serving signal. Do not keep adding learned exposure bias as quality.
Illustrative example: Learning old items' logging advantage need not reward them again in deployment.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Implementation details
Do local SNGP proxies or negative results explain YouTube's live study?
Breaking the Loop: An Empirical Comparison of Strategies for Novelty and Freshness in YouTube Music
Show answer and example
Answer: No. There is no full SNGP or continual training. Report interventions, logging propensities, and budgets separately. Popularity is not a measured IPS propensity.
Illustrative example: Low popularity does not establish a low historical impression probability.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Basics
Does RLBP use a neural network to bid in every auction?
Reinforcement Learning for Controller Parameters in the Auto-Bidding System at Pinterest
Show answer and example
Answer: No. It chooses daily PID multipliers and writes a fixed daily lookup table; the original PID controls high-frequency feedback. Production varies P and I over three factors each with D=1, giving nine actions.
Illustrative example: Example: Set P to 1.2 times its default in the morning. Individual bids still respond to PID errors.
Sections in the Chinese source: 背景与主要改动, 核心公式. The example illustrates the mechanism.
Core mechanism
Why subtract the CPC head if CPC improvement is negative lift?
Reinforcement Learning for Controller Parameters in the Auto-Bidding System at Pinterest
Show answer and example
Answer: Lower CPC is better and below-baseline change remains negative. In Qbu−beta×Qcpc, subtracting negative CPC raises the score. A multi-step CPC backup also uses min rather than max.
Illustrative example: Example: For equal budget score 0, CPC heads −0.1 and 0.2 with beta=1 give fused scores 0.1 and −0.2.
Sections in the Chinese source: 核心公式. The example illustrates the mechanism.
Method comparison
Why can deployed RLBP behave like a contextual bandit despite the DQN name?
Reinforcement Learning for Controller Parameters in the Auto-Bidding System at Pinterest
Show answer and example
Answer: Deployed gamma=0 regresses same-day outcomes with no future Q bootstrap. The proposed multi-step extension independently maxes/mins heads before fusion and need not equal optimal Q of the combined reward. It is not the deployed long-run optimum.
Illustrative example: Example: Reward 1, future Q 100: gamma 0 yields 1, whereas gamma 0.9 changes the objective to 91.
Sections in the Chinese source: 背景与主要改动, 核心公式, 论文离线与线上效果. The example illustrates the mechanism.
Implementation details
Does implementing top-K softmax reproduce production action probabilities?
Reinforcement Learning for Controller Parameters in the Auto-Bidding System at Pinterest
Show answer and example
Answer: No. Production also needs trained Q, beta scale and post-fusion preprocessing. The component externalizes beta and preprocessing; identity is just a reference. Small tensor tests cannot supply logged propensities or actual outcomes.
Illustrative example: Example: Doubling scores often preserves top-K but sharpens softmax, changing IPS propensities despite the same argmax.
Sections in the Chinese source: 核心公式, 本地复现与边界. The example illustrates the mechanism.
Topic guide and reading order
Start with this question¶
Why is sorting items by individual score not necessarily a good slate?
Takeaway¶
Slate value depends on neighbors, position, and selected prefix. Redundancy-aware methods estimate marginal contribution. SORT-Gen uses prefix-conditioned multi-objective slate value. Policy methods trade off exploration, ad load, revenue, and long-term feedback. Titles, creatives, and layout affect exposure and belong in the evaluation. RLBP instead chooses bounded daily PID multipliers; two Q heads learn BU and CPC, then lifecycle weighting combines them. Similar objective weights do not imply identical states, actions, or rewards.
Example¶
If three identical ski clips score 0.90, 0.89, and 0.88 and a camping clip scores 0.85, pointwise ranking chooses the first three. A slate model may choose the camping clip once duplicates have low marginal value. Whether this trade-off is worthwhile depends on objectives and constraints.
Common pitfall¶
Weighted objectives do not guarantee hard ad-load or coverage constraints. Diversity does not guarantee satisfaction. MovieLens clicks, payments, or GMV proxies are mechanism evidence, not revenue. Long-term value needs a corresponding observation horizon.
Reading order¶
Read CDM for relevance and redundancy, SORT-Gen for prefix-dependent value, and PRL-PUTS for prediction versus multi-objective policy. Check gaps between local proxies and production goals.
- Contextual Distillation Model for Diversified Recommendation · Original paper
- A Generative Re-ranking Model for List-level Multi-objective Optimization at Taobao · Original paper
- A Production-Ready RL Framework for Personalized Utility Tuning with Pareto Sweeping in Pinterest Recommender Systems · Original paper
Check your understanding¶
If item-level CTR predictions stay identical, can reordering still change page clicks?
Show suggested answer
Yes. Position exposure, duplicate neighbors, and prefix effects can change slate outcomes despite unchanged pointwise scores.